Slashdot Mirror
Power Outage Takes Wikimedia Down
Baricom writes "Just a few weeks after a major power outage took out well-known blogging service LiveJournal for several hours, almost all of Wikimedia Foundation's services are offline due to a tripped circuit breaker at a different colo. Among other services, Wikimedia runs the well-known Wikipedia open encyclopedia. Coincidentally, the foundation is in the middle of a fundraising drive to pay for new servers. They have established an off-site backup of the fundraising page here until power returns."
And here is the data Paris Hilton photos and phone numbers
People should talk. They should talk about his work and his contribution to American culture. They shouldn't be making fun of him. He deserves better. What if someone you knew killed himself? Would it be funny if people joked about that?
You know not what you ask. Trust me.
But then again, I could be wrong.
Here are the ingredients to this solution:
A completely designed, 100% empty database.
A COMPLETE log of all the SQL statements that were applied to it IN the order they were used. This is obtained by the application logging the SQL statements to the SQL log file AFTER the SQL statement is succesfully executed.
When a data base failure occurs, stop everything, 'replay' the backed up SQL logfile (thats on a separate backup system) on a copy of the empty DB there. TADA! you are back in business back to the point of failure!
The downsides....
Redesigning the database will screw everything up unless the SQL statements used during the redesign are logged as well.
All sql requests must be funneld through 1 and only 1 db connection. Otherwise the sql statements in the logfile stand a chance of being recorded 'out of sequence'. Here is a brief example:
With one db connection, user 1 edits record x two separate times in succession then user 2 comes along behind user 1 and modifies the record with no problems. Without record locking or with indiscriminant multithreading, record x will be corrupted if user 2 edits record x between user 1's two consecutive edits. See the downside?
The SQL logfile gets corrupted due to storage media failure. The only way around this would be to copy the log file to a backup mirror system on a periodic basis and verify it is a good backup copy using a strong cryptographic hash such as SHA-512 or for the utterly anal and paranoid, a byte-for-byte comparison.
The EXTREME volume of data may/will make this approach unfeasable due to time constraints -- too much data to restore via 'replaying'. 'Checkpointing' from a known good database state will cut down the size of the SQL log file but introduces the possibility of database corruption by simply using the wrong checkpoint database when replaying the sql statements.
Speaking of 'tar' in the parent post, I 'cowrote' a simple, high-performance freeware Windows file archiver that combines file aggregation with data compression. If you want to try it out, it is here.
I think Hunter Thompson would probobly rather his death not be taken so seriously.
"It is not how things are in the world that is mystical, but that it exists." -Ludwig Wittgenstein