3.41am SG time - TR is up and running...
http://www.temasekreview.com/2010/10/12/corrupt-database-caused-prolonged-site-downtime/
Corrupt database caused prolonged site downtime
October 12th, 2010 | Author: Han Molina
Temasek Review went offline from Monday 2200 hours to Tuesday 2300 hours due to a corrupted database caused by a failed upgrading of wordpress, which powers our site.
We had originally scheduled an upgrading of our wordpress installation on Monday 2200 hours – expecting it to be completed within an hour, but unfortunately it did not.
The upgrading failed due to compatibility issues with our customised theme and certain plugins which were outdated. This caused a serious security breach, allowing any registered users full access to our administrative backend. Our tech had to shut the site down immediately.
* We later learnt that the failed update had corrupted the database. Since our servers were sync in real time, our mirror server suffered the same fate.
As it was already past midnight, we were unable to grab hold of the programmer who customised the theme used on our site to work with support personnel from our service provider to rectify the problem.
Although our service provider had originally suggested restoring the site from the previous day’s backup but we decided against it because hundreds of posts would be lost. Instead, we decided to salvage the corrupted database and try to restore it as fully as possible.
Fortunately for us, the large chunk of data corrupted were subscriber’s information and previous site archives for 2008. Other than that, all data remains intact.
We would like to take this opportunity to thank our host RTG (Asia) Network and our programmer Alan Goh for their professional assistance. Most importantly, our readers for their patience and understanding.
Molina Han
P/S: If you are a registered subscriber of our site but no longer able to login with your existing username, try executing a password retrieval and if it fails, please re-register yourself. Contributors, Authors and Editor’s data are all intact.
http://www.temasekreview.com/2010/10/12/corrupt-database-caused-prolonged-site-downtime/
Corrupt database caused prolonged site downtime
October 12th, 2010 | Author: Han Molina
Temasek Review went offline from Monday 2200 hours to Tuesday 2300 hours due to a corrupted database caused by a failed upgrading of wordpress, which powers our site.
We had originally scheduled an upgrading of our wordpress installation on Monday 2200 hours – expecting it to be completed within an hour, but unfortunately it did not.
The upgrading failed due to compatibility issues with our customised theme and certain plugins which were outdated. This caused a serious security breach, allowing any registered users full access to our administrative backend. Our tech had to shut the site down immediately.
* We later learnt that the failed update had corrupted the database. Since our servers were sync in real time, our mirror server suffered the same fate.
As it was already past midnight, we were unable to grab hold of the programmer who customised the theme used on our site to work with support personnel from our service provider to rectify the problem.
Although our service provider had originally suggested restoring the site from the previous day’s backup but we decided against it because hundreds of posts would be lost. Instead, we decided to salvage the corrupted database and try to restore it as fully as possible.
Fortunately for us, the large chunk of data corrupted were subscriber’s information and previous site archives for 2008. Other than that, all data remains intact.
We would like to take this opportunity to thank our host RTG (Asia) Network and our programmer Alan Goh for their professional assistance. Most importantly, our readers for their patience and understanding.
Molina Han
P/S: If you are a registered subscriber of our site but no longer able to login with your existing username, try executing a password retrieval and if it fails, please re-register yourself. Contributors, Authors and Editor’s data are all intact.
