Slashdot Mirror
Ma.gnolia User Data Is Gone For Good
miller60 writes "The social bookmarking service Ma.gnolia reports that all its user data was irretrievably lost in the Jan. 30 database crash that knocked the service offline. Ma.gnolia founder Larry Halff recently discussed the crash and the lessons to be learned from Ma.gnolia's experience. A lesson for users: don't assume online services have lots of staff and servers, and always keep backup copies of your data. Ma.gnolia was a one-man operation running on two Mac OS X servers and four Mac minis."
Crashing Macs? That's unpossible!
Facebook was recently brought down when their hamster keeled over and ceased powering their Amiga.
Bored at work? Play Game!
This bad news is delicious food for Stallman's argument against "cloud" services.
Colin Dean Go a year without DRM
Argh, why not just add a backup or replication database on one of the spare Mac Minis?
That way you would have needed a complete server farm disaster to mess things up irretrievably.
Couldn't resist.
And how can they be slashdot worthy when they are a social networking site with ONLY a half a terabyte of data? In short, who cares?
Feed the need: Digitaladdiction.net
lesson #2, trust no-one with your data
lesson #3 disaster recovery capability only exists after it's been tested
lesson #4 backups are useless unless you can prove you can recover from them
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Good backup strategies are critical to any operation, regardless of platform. I've seen similar things happen with MSSQL server databases as well as Oracle running on the most powerful Sun box you can get (circa 2001).
One database backup strategy I've seen used rather successfully is doing a straight SQL dump every night and then copying the sql file over to somewhere else; even if the database became hopelessly corrupted there's still a way to re-import everything.
Of course, this is in *addition* to mirroring, tape backups, etc.
Friday, February 20, 2009, 10:01 AM PST:
WE'RE FUCKED!! WE'RE FUCKED!! GAME OVER MAN, GAME OVER!!!
Tuesday, February 17, 2009, 10:50 AM PST:
Oh. Shit. Unfortunately, database file recovery has been unsuccessful and I won't be able to recover members' bookmarks from the Ma.gnolia database. This means that the public bookmark recovery tools are the only source for recovering your bookmark collections.
If you are interested in hearing more about what happened, the history of Ma.gnolia in general, and future prospects, you can watch the latest Citizen Garden podcast below, which was recorded last week. As I mention in this podcast, I am working on relaunching Ma.gnolia as a private service on a more robust infrastructure in the coming months. I'll update this page and the twitter account with those and any other developments.
Friday, February 13, 2009, 7:00 PM PST:
The data recovery folks let me know that they're still working, but I should hear more from them by Tuesday. Everything's cool, no worries, man!
Monday, February 9, 2009, 3:10 PM PST:
Just posted a new bookmark recovery tool for our members who used the scheduled blog posting system. Relax...you'll get your stinkin' data back. Whiners.
Sunday, February 8, 2009, 9:50 PM PST:
There are now more than twice the number of bookmarks in the Web Cache recovery tool than there were on Friday. I can't guarantee results, but members may want to try again.
Friday, February 6, 2009, 6:00 PM PST:
I've been able to extract a set of public bookmarks from a cached copy of the Ma.gnolia web site, and these are now available via the Web Cache recovery tool. I'm working on expanding the size of this cache and the number of bookmarks I'll be able to provide and will update this page, the bookmark recovery tips thread at Get Satisfaction, and Twitter when I do so.
Additionally, I'm working with data recovery specialists to recover the Ma.gnolia data store; but, unfortunately I won't hear from them until next week. As with the other data recovery efforts, I'll keep you posted as to their progress I as get any updates.
Dear Ma.gnolia Members and Visitors,
So far, my efforts to recover Ma.gnolia's data store have been unsuccessful. While I'm continuing to work at it, both from the data store and other sources on the web, I don't want to raise expectations about our prospects. While certainly unanticipated, I do take responsibility and apologize for this widespread loss of data.
In this past year, many of us have seen much loss around us. While bookmarks seem small on the national or global scale, I know that many of you had built intellectual and social capital through the bookmarks, groups, and connections you made here. For those who had shown their support for Ma.gnolia by buying one or more premium feature subscriptions, that's one thing you won't be losing: refunds will be issued for those purchases within two weeks from today.
Ma.gnolia was approaching the third anniversary of its public launch; for me, it was the project and people to which I'd devoted most of my time, energy, and love for nearly four years. It's still a little too soon to give word about the return of Ma.gnolia the service and the future of the M2 project, but I will keep this site and our Twitter account updated as those decisions are made.
In the meantime, I can provide a few pointers to some resources that can help:
If you've been publishing your bookmarks through any RSS feeds or aggregation services like FriendFeed, you can re-capture some of them before those feeds expire.
We've set up a recovery tools page with several options. We're still adding more.
If you're looking for a place to start a new collection, I think Diigo is a good option to check out for its groups, cross-service posting features and attentive staff.
Further tips for recovering bookmarks can be found or posted in a thread at Ma.gnolia's page on Get Satisfaction.
Sincerely, Larry
Well, at least they now know how not to do it properly. The probability that it will happen again is quite small, especially because no one will trust them anymore in the first place and it will be really hard to start anew. Their best bet is probably to start a new service under another name and another look.
discussed the crash and the lessons to be learned
Lessons such as "Regularly monitor and maintain backups like and business should?"
Apparently they were waiting on the port of the drive imaging software.
Like frickin' having a backup? Isn't that one of the first things you ever learn if your business relies on computers + userdata?
ACK
You shouldn't use shiny plastic ornaments for serious business.
It's food for any argument against any web service that doesn't publish it's reliability information or publicize the data for what types of mechanisms it has in place in case of disasters like a corrupt database, fried motherboard, or busted hard drive.
There's a design methodology that's used by NASA for manned missions: Any individual component should be able to fail without compromising the mission. Of course, in the last few decades we've seen 2 out of 5 Shuttles go ka-boom! so obviously this NASA guideline isn't enough and it's *REALLY* hard to prevent failure when a perfect storm of multiple systems experience failure at the same time.
So if anything, I'd say this is an argument that supports robust, reliable, fault-tolerant design rather than just kludging a half dozen systems together and calling it a "web service".
Support the 30 Hour Work Week!!!
One man operation which doesn't make backups, sounds fairly typical to me, remind me again why this is Slashdot worthy?
My Mac servers run snapshots to external drives every hour. When something goes badly, it's back up in a few minutes. Not sure why that wouldn't have been done here.
I mean, just because a few medium-profile sites running on Macs have experienced a failure causing data loss doesn't make them unique. Every OS and every type of hardware will, at some point, experience a failure. It's the PEOPLE that make the failure a problem, and it sure looks like this tard was a problem.
Who the hell doesn't back up their data? Seriously? This is "Slashdot worthy" because some hapless Mac user lost their data. BOO FUCKING FAIL. Move on.
Honestly, how sad can I get that some people lost their social bookmarks. Start over. All us geeks are perfectly happy to reformat our hard drives and reinstall our OS every 9 months or so... so quit crying over your beloved social bookmarks! They're for girly-men anyhow.
"Gee, Bob, we have the proof that this thing works. Why don't we sell it already?"
"Well, Bill, nobody wants to buy it and grandfather in all the whining freeloaders and their data."
"It's too bad we can't just drop all the data and start fresh."
"Well, why not, Bill? All we have to do is say it's been lost and can't be recovered. We can tell the buyer what's actually happening so they don't think we're total IT rejects who couldn't figure out a data retention policy."
"That's why I like working with you, Bob. You always have a way around the problem."
Have fun with it. The names have been changed (one changed anyway and one added), well, because it probably has nothing to do with reality. It sure is fun to ponder, though.
Rather than watch the video or download the 23MB MP3, you can read the full transcript here:
http://ratafia.info/post/78915439/transcript-and-commentary-for-whither-magnolia
I can read much faster than I can listen.
All right, let me get this straight: First you people bitch and moan when Facebook says they'll save user data forever. NOW you people bitch and moan when this site loses user data forever! You're never happy, are you?!?
Demanding constant attention will only lead to attention.
Since the file system and database were corrupted, it wouldn't matter if it was hardware RAID or software RAID. That's not the problem at all, the problem is there was no archival backup, and their only backup was a file sync... that replicated the database errors on the backup.
To backup a database, you dump it in a serialized form, or maintain a serialized form of the data in parallel with the database.
And the users got what they paid for.
Simple as that.
The flip side is that this guy's service will probably be the MOST reliable going forward.
Of course he should have had reliable backups; now he is the poster child for backups. Remember, nobody pays you for backups, only for restores.
Users: if you're trusting your data to someone else, you need to insure one of two things. Either you need a signed, iron-clad written contract guaranteeing service with nasty penalty clauses requiring the service to compensate you fully for all the costs of data loss (and sufficient insurance and/or confidence that the service has the wherewithall to pay those penalties and not just flee into bankruptcy leaving you holding the bag anyway), or you need a backup of your data under your own control and in a form where you can upload it to another service if you need to change services. If you don't have one or the other, you will end up being caught in something like this. The only question is when it'll happen. And I can pretty much tell you that no service you can afford will give you the first option. So you'd better evaluate the cost of backing up your data yourself and the costs of losing it and decide which one you're going to pay.
Time for ZFS
He kept all on one hard disk? Even I know that it is wrong. I presented my spouse a PC on her birthday with the hard disk of 500 GB, I mean it s not that hard to back up 500 GB nowadays.
Ouch... Isn't part of a backup strategy to sometimes attempt a recovery from a backup, on a test system?
I just assume that the FBI or NSA have a back door into my server and are making copies of everything for me.
What the hell is a "social bookmarking service"? Since the site is dead, going to their webpage didn't help clear that up at all. Is it seriously a social networking site where people share _bookmarks_?
Didn't almost the exact same thing happen to some blog hosting service within the past month? Wow.
This is why I'm reluctant to trust "the cloud" with my data. ALWAYS keep a local copy and only use "the cloud" for backup.
"Just because you do not take an interest in politics doesn't mean politics won't take an interest in you." --Pericles
Seriously, all hardware will eventually die, unless it joins the Q continuum or part vampire, demon or god... Really the summary should be. Mac user is retarded and doesn't backup. But then that would be redundant given that hes running a server on macs...
After all, it's the new paradigm.
NASA guideline isn't enough and it's *REALLY* hard to prevent failure when a perfect storm of multiple systems experience failure at the same time.
I'm not saying that saving Apollo 13 wasn't hard, or an extremely great accomplishment, however I am going to say "slick and pretty" (the shuttle) is generally the opposite of "robust" or "fault tolerant." Slick and pretty is also usually more expensive.
The basic, non-pimped xserve is $2999. An identically configured node from eRacks, running your choice of BSD (the default on these quad-core Xeons seems to be OpenBSD) or Linux, $1894 -- leaving you with plenty of room in the budget to build a bigger, badder node, or replace one when it fails.
I suppose the point i'm trying to make is that if you're going for function over form (Apollo capsul), it easier to plan for more contingencies on the same budget you'd otherwise be spending on gee-whiz factor (shuttle).
You have a 5 digit UID and you are just realizing this now?!?
DEMETRIUS: Villain, what hast thou done?
AARON: Villain, I have done thy mother.
Shakespeare invents 'your mom'
Software raid. Profession web app. Brilliant!
Poof goes the cloud bank.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Google lost about 4 years worth of my emails with no explanation and no response to my inquiries. And nobody can say they don't have enough servers or enough experienced stuff. I guess it's just that "you get what you pay for". The lesson here is don't rely on outside vendors for stuff that really matters. Especially if the service is free.
If con is the opposite of pro, is Congress the opposite of progress?
Should have paid the LifeCall dues on time.
Here's a funny thing. I was using Epiphany under GNOME, and discover that the mis-named "Epilicious" extension has not worked with Delicious for some time, but it did work with Magnolia. So I go to Magnolia only to discover that the service had just gone down (this was on or about 30 Jan. 2009 -- what timing!). So I'm still on Delicious, and now using Iceweasel (that's Firefox to most of the world) because it's the only thing that works with Delicious.
My question is: What other options are there? I have found some good links and some very useful information (especially in the area of *nix and OSS tutorials) on Delicious, but strictly speaking I don't need an account there to continue using it as a search engine. I do like having my bookmarks available from any computer I happen to find myself on and I'm aware that there are other ways to achieve this, but I'm not sure how or what the best methods are. When I say "hapless *nix user," I mean I can get by in Debian just fine, but I tend to keep things pretty simple ... no advanced server set-up, no tricky configurations. I no longer use GNOME, preferring a simpler Openbox set-up. I've gotten pretty good at navigating around my system in the terminal, using mc for file management, even writing a few very basic bash scripts, but networking especially remains something of a black art to me. I'm not a programmer or web designer and have no desire to be; I use Debian because I like the philosophy of OSS.
So what are the easier more-or-less "point-and-drool" methods of achieving remote access to one's bookmarks, without having to mess with SAMBA or NFS or similar complexities? Do they exist? If cloud services like Delicious and Magnolia are inherently risky and soul-suckingly closed, are there any safer, more open alternatives that are as easy to use?
"No live organism can continue for long to exist sanely under conditions of absolute reality;..."
Isn't this pretty much a repeat of this story: http://hardware.slashdot.org/article.pl?sid=09/01/02/1546214 ? All the excuses and "lessons" just try to hide the same root cause - incompetence.
End anonymous moderation and posting on
Seriously. No backup? At all?
Of course, it's not like it matters. I've never heard of ma.gnolia, and I doubt very many other people have. This dumbass gets what he deserves.
Pillock.
Once you get past one server (if not before), you need to start backing up. Once you start offering a service to people who are giving you money by visiting (whether through ad revenue or otherwise), you need to start backing up. The number of servers is an indicator to the amount of work you're putting into something and the popularity of it. Even if you consider your user's data unimportant, the work that's gone into the servers and their various configurations by that point makes it worth backing up the WHOLE thing. Once you start maxing out a server, you REALLY need to look closely at what happens if it all goes wrong. This is especially true if you're going to start doing replication or load-balancing because in the first case you get the problem of your mistakes replicating across every one of your servers, and in the latter case you have the problem of what happens if one half of your load-balance cuts out (because, presumably, your remaining server won't be able to cope for long on it's own).
My brother runs a very popular website for a specific niche... The only money it makes is Google Ad revenue (enough to pay for hosting plus a bit more). There aren't millions of users who would cry foul if the site was down for a few days. But even I have a backup of the content that he gives me to store on my "offsite location" (back bedroom) on a regular basis, we have a seperate domain on a seperate host that isn't advertised and which mirrors all the content, there are local copies on not just one computer but every computer we own, there are copies of most stuff on several, seperate FTP hosts, plus we burn off DVD's of the content on a regular basis to provide a historical record.
We *have* had to restore the backup several times (which isn't fun when you have Gb's of data to be uploaded from a broadband connection over FTP, and even better when the new configuration doesn't run PHP or have the same paths and Unix permissions, etc.) because of mistakes at the hosting company... but we don't buy backup from them, so it's fair enough. We've never yet lost a byte of anything important - a few forums entries at most, but even the MySQL databases were safely backed up. We don't have RAID, we don't have fancy rsync'ing, scheduled backups or anything else. It's just slapping some stuff on a different computer / different media every now and then.
Now, this is 500Gb of other people's data... that alone tells you that you should be backing it up... the schools I work for don't have that much total storage if I add every server's data together (including Ghost images of client machines) but we still back it up on a regular basis (disk, tape, offsite, etc.). I've *never* had to restore a server in that case but I know that all the backups work just fine.
Don't be an idiot... if it takes you until you lose data before you consider backups, then you're a fool. Just do it for yourself - turn off your machine and pretend the hard drive is corrupt - how much of that data do you have stored elsewhere? How much of it would you be sad to lose? Don't forget to include configurations (thank God, Linux configurations are so easy to back up, being mostly plaintext), PHP scripts, passwords stored on it, login emails, scripts, serial numbers, text files of instructions on how to do certain things, before you even get to your actual data (photographs, personal emails, documents, etc.).
Some well-known sites are just hanging on, with almost no staff. Tribe is down to two employees.
The most active "tribe" is "Tribe.net bug reports".
I can't even tell from the website and searching doesn't turn up much of anything. What did it do? What is a social bookmarking service?
The ratio of people to cake is too big
Enpossiblized, the way things can be "embiggened"...
But, i am sure that while the databased was "enlightened", Halff was fully "enlightened"...
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
Wow and I was using this, because Jeffrey Zeldman told me to. Time to switch to Delicious and never use anything this company ever makes again. Fail.
I've had enough, I'm backing up all my Slashdot comments. The world can't afford to lose them!
Oh they knew perfectly well that the Challenger disaster would happen. The breathtaking silliness of using o-rings to seal a solid-rocket booster was discussed more than enough times. The problem, there and everywhere, was politico-economic concerns shifting the risk thresholds farther than we (afterward) feel they should be shifted.
NASA can't win though. If anything, we should fault them for taking too few risks, especially given the very large number of people who would line up to volunteer for even a crazy risky mission. (The are "lined up" in the sense that NASA has stacks of applications even though the prereqs are currently higher than Mount Everest.)
The worst thing NASA did to space exploration was make it boringly safe and cautious. Why do you think everybody got so excited about Rutan's cheesey nonorbital craft?
FATMOUSE + YOU = FATMOUSE
All right, let me get this straight: First you people bitch and moan when Facebook says they'll save user data forever. NOW you people bitch and moan when this site loses user data forever! You're never happy, are you?!?
Sorry but it wasn't the fact that Facebook was saving user data forever, it was the fact they the original person gave up rights to the data and Facebook basically "took" your stuff.
So if you decided to close your account, Facebook could turn around and sell your pictures to a stock photo house for example.
Again, it was that they were keeping the data... it's what they could potentially and legally have done with the data that made it so bad.
How would you really KNOW how large a service is when you sign up? The simplest indicator is a slick interface, but really that only tells you the difference between a newbie and someone with the smarts to download a better one. After that, you only have indirect indicators like response speed, and apparent reliability. Anything else could all just be a Massive Hoax.
How do we know kdawson, samzenpus, soulskill, and the others aren't just Cowboy Neal's alternate personalities?
Behold, this dreamer cometh. Come now, and let us slay him... and we shall see what will become of his dreams.
...Seriously, do people still not realise that OS X is just UNIX with a pretty UI?
Ah, ask the average 17-year old girl going off to college that just HAD to have a Macbook (with a pink cover no less) what the operating system is based on. What percentage of Macbook owners would be able to answer that accurately regardless of gender?
Apple is 1% hardware and 99% Marketing. Not too much they do can't be done on a Dell or HP. They just make it appear to do it better/slicker/faster, that's all.
I hope this happens to intelligence services too from time to time ...
NB: The message above might reflect my opinion right now, but not necessarily tomorrow or next year.
That can't possibly be true. If one booster fails, you aren't getting to orbit. If one one windshield fails I suspect you're in a world of hurt as well. :D
Mac OS X Server runs a host of services, particularly for managing Mac OS X clients, that you won't find on any other OS, so there are reasons to get a Xserve in particular; web serving just is not one of them.
"I like systems, their application excepted", George Sand (French)
And we'll probably be cheering if Facebook suddenly lost all of its data forever. That or coming up with fancy conspiracy theories.
obviously this NASA guideline isn't enough and it's *REALLY* hard to prevent failure when a perfect storm of multiple systems experience failure at the same time.
Neither the Challenger nor the Columbia represented simultaneous multiple failures. They *did* represent cascade failures that should have been planned for, but weren't.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
Oh they knew perfectly well that the Challenger disaster would happen. The breathtaking silliness of using o-rings to seal a solid-rocket booster was discussed more than enough times.
Particularly that morning, when it was cold enough that the engineers were saying, "I sure hope the O-rings work". I must say, it was entertaining being present at one of the Rogers Commission hearings and watching Feynman lay into the Thiokol guys about the O-rings.
Please stand clear of the doors, por favor mantenganse alejado de las puertas
Comment removed based on user account deletion
The extra $ is for the software. Mac OS Server comes with a lot of stuff that's not available elsewhere. Whether or not you need that stuff is a separate question (most people don't).
You are a dumbass.
Actually, I wouldn't mind if all of the articles accepted by kdawson were "lost". I also wouldn't mind "losing" the entire Idle section.
I'm backing up all my YouTube comments. Yes, I'm Rocckir!
Couple of points here:
1) The second shuttle was not destroyed by a component failure. Ceramic tiles were damaged by debris; they did not fail to function, their integrity was compromised by a outside influence.
2) There will always be some components which are single points of failure. Both of the shuttles where destroyed by something going wrong in one of those areas.
I must say, it was entertaining being present at one of the Rogers Commission hearings and watching Feynman lay into the Thiokol guys about the O-rings.
My favorite part of Feynman's report was how he pointed out that they claimed that the O-rings had "a safety factor of 3" because they were regularly only 1/3 eroded (!) by blow-by gas leakage, when any leakage of combustion gases was already an anomalous event according to the design. Perfect illustration of the exact kind of boneheaded idiocy that has gone into the effort to keep the boondoggle shuttle flying.
If a job's not worth doing, it's not worth doing right.
Don't say mean things. She may become ... furious. But, if you send her a McFlur(r)y, she might be ensweetened.
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
a Confibulating Anular (vs annular) Confinement Beam to tap into your data stream via their ODN Junction/Relay frazzletrap... Instead of "A Fistful of Data", one might doodle to "An Assload of Data Streams"...
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
The second shuttle was not destroyed by a component failure.
The foam on the external tank is a component.
http://michaelsmith.id.au
It's a one man show and the guy that actually runs it is called founder? Precisely another example of the pretentiousness of the Mac community. A backup on a piece of $50 hard drive maybe? Oh who has time to think of unpretty things like that?
http://hardware.slashdot.org/article.pl?sid=09/01/02/1546214
Per your own numbers.
Why would I ever use tape? Granting tape can scale (with a changer) you still have the throughput of the drive as a limit.
I could attach 8 2TB USB drives to a backup server and backup critical data to that. Granting the USB throughput is weak, multiplying by 8 (or more) fixes that.
But unless I misunderstand the mess the problem was database integrity and lack of old backups.
I wouldn't look forward to hauling 8 drives home in a box every day.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Since the file system and database were corrupted, it wouldn't matter if it was hardware RAID or software RAID. That's not the problem at all,
Why isn't this easy for people to understand? Faith in hardware replication as backup seems incredibly prevalent, even here on /. where you'd think everyone would get it.
Very strange.
"And the meaning of words; when they cease to function; when will it start worrying you?"
It's particularly bad in the Mac world, because the leading desktop-level "backup" tools just do disk mirroring.
I know. As modern-day techies, the very concept of using magnetic tape to hold data is anathema to us. We remember the old TRS-80 or Commodore cassettes, or maybe even reel-to-reel, and it just seems so outmoded.
But here's the thing: tape really is pretty much the absolute best backup solution going right now.
Of course, I am making a big assumption here: that you're going to want to take your backups out of the server room. In that case, do you really want to send off hard drives every day? Tapes are built with this specific use-case in mind, while hard drives aren't -- I can't imagine what you'd be doing to the MBTF for a drive by sending it on a road trip once a week.
In fact, if you watch the video here, you see that's effectively the solution they had, just Firewire instead of USB. This was precisely the point of my original post: people seem to think that this is at least theoretically viable, but it's really not. If you think it is, then I challenge you to try it -- go out and buy even just 4 cheap 2GB flash drives, plug them all in, time how long it takes to fill all 8GB, then extrapolate that to 800GB and beyond. Note that the limiting factor in the speed here will not, surprisingly, be the speed of the flash drives but will be that of the USB bus.
Nope, you misunderstood half of it -- the data integrity was a problem, yes, and they did have backups, but they never bothered to make sure that the backups worked. They were backing up garbage data, so the backups were just as useless as the original corrupt data.
Whether they had a stack of USB drives or a stack of tapes or a stack of Blu-ray discs wouldn't have mattered in the slightest, as they never checked to make sure they could restore in the event of a catastrophe.
Which brings me full circle: backups are very, very easy to do wrong. Most people think of them as an afterthought. Even once you really sit down and think the entire process all the way through, from backup to restore, they are still astoundingly difficult to do right. And, at least for now, none of the "right" answers are anywhere near cheap, while all of the wrong answers are quite cheap and obtainable.
eRacks website error
eRacks has encountered an error while publishing this resource.
Error Type: KeyError
Error Value: 'Xeon Systems'
Troubleshooting Suggestions
This resource may be trying to reference a nonexistent object or variable 'Xeon Systems'.
The URL may be incorrect.
The parameters passed to this resource may be incorrect.
A resource that this resource relies on may be encountering an error.
For more detailed information about the error, please refer to the HTML source for this page.
If the error persists please contact the site maintainer. Thank you for your patience.
So either they don't have Xeon Servers (so no identically configured node) or...
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
http://eracks.com/products/General%20Purpose/config?sku=QUADPREM
or you're doing it wrong?
Time for ZFS
So how does ZFS deal with corrupted MySQL files?
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
I love whens someone has a replicated DB as a "backup". I like to say okay.. "Drop table users". And then it dawns on them that the drop command would replicate.... ;)
Damn, I knew that I shouldn't have recommended Ma.gnolia to little Bobby Tables.
Serves them right for demanding users register with their full name, though.
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
that has been hosting close to 200 clients with 300+ sites along with ecommerce ones, and high traffic ones.
i didnt lose anything. every day whole server gets backed up with rsync to some new york based backup service, automatically.
costs 20/month.
is it TOO hard to set up an arrangement like that ?
Read radical news here
When I had a major photo sharing site on the 'Net circa '98-'01, we hosted a lot of user data, to the tune of 50M photos. They were all backed up, regularly and religiously. To multiple media. They were stored on RAID to start with, so any hardware failure was dealt with. Then we had mag tape as a primary backup, stored offsite. And we even did a dump to DVD now and then. With a site that lives and dies on user's content, backup (and restore!) is critical to your survival.
Screwing this up is gross negligence, IMHO. Especially where RAID and redundant servers are so incredibly cheap nowadays. (And we were stashing entire photos from digital cameras; these guys were storing bookmarks... Tiny!)
Love many, trust a few, do harm to none.
Hey, us 5 digit people are old. Give the guy a break will ya?
I know what the Internet is, what the hell is this Interweb business?!
If you can't afford to have backups, you can't afford to have data. Period, end of story.
They don't realize that your data is worth more than all of your hardware. Hardware is something that anyone can buy, your data (including your code) is all you've got.
Even on the tight budget at my shop, a backup is created every night, kept locally, and also copied to a pair of machines (yes, two of them) in another location. Once per week, a copy of the backup is taken to yet another location for storage.
A lot of small-time shops think that something like that is too costly... but really, if they lose all of their data, the cost of backups is nothing.
And... lots of beginners don't realize that even though the probability of TWO of your backups being bad at once may seem astronomical... it happens a whole lot more than many folks would think.
Look at the monkey! Look at the silly monkey!
Chas - The one, the only.
THANK GOD!!!
I had to do some research recently which backup software to use for my Debian server. And the most flexible and at the same time easy to use solution seemed to be Backup-Manager. One other program I also took a close look at was Backup2l. There were other progams, but those either seemed VERY big and complex (like Bacula) or did not even some of the features I needed and those features were very important ones.
http://backup2l.sourceforge.net/
http://www2.backup-manager.org/
Really it's easy.
1. Put your data in an encrypted blob.
2. Name that blob something like [Natlie.Portman.Hot.Grits.REAL][][][XvIdd.AVI.TGZ.Bz2].avi
3. Upload to a tracker.
Done
The real problem is that the one man ignored the fundamentals of system administration. He was totally ignorant of what backups actually meant, was too dumb to know that he had no expertise in the administration of systems, and thus did not belong in the position he was in. This is the Darwin award for companies composed of stupid people. In the end, he got what he deserved, and a whole bunch of people are wiser now.
"Beware of he who would deny you access to information, for in his heart, he dreams himself your master."
I had no idea anyone actually used Mac's as servers.
Huh? Mac's what? And who is Mac?
Is this Mac's Milk in Toronto, where they offer a complete Win-7 DVD for only $12 when you buy a quart of milk? Wow!
Lies.
Is that they never did any proper restore testing....
Why would you put your data that you haven't proven is safe?
Hey boss, I spent $3,000 instead of $1000 on our backups. Same order of magnitude, quit bitching.
Tapes are about $0.30/GB (raw space, since you can also compress data to drives). Hard drives are around $0.09/GB. 10TB = $3,000 for tape vs $900 for disk. Not including tape drive.
So you're paying about 3x for tape.
Tape drives have moving parts. Oh the tapes themselves - here's a fun scenario: Write your backup out to tape, then verify it was written correctly. It was, so ship it offsite. The tape warps in the back of a hot delivery van, then sits in storage a while. Your database fails, so you get the tape back from storage. While restoring you get data errors (from the warping), then the tape snaps, gets chewed up in the tape drive, which then jams.
Tape has failure scenarios too.
Here is the the one argument tape has in its favor. There are rugged enclosures and connectors for hard drives, though.
Of course, as networking gets cheaper, it may become more attractive to have a local disk backup, and a remote disk backup that is copied over the network. Depends on how much data you produce every day vs your network speed/costs.
Portable drive vs. tape arguments aside. (Consider that USB drives can run in parallel and have decent throughput. This dude never changed his firewire drive, you can write to the same tape over and over as well.)
This wasn't a backup problem in the first place.
This was a database integrity problem and should have been caught by routine or built in validation checks on live data before it ever came to going to backups.
The best you can expect from a backup is a snapshot of your database from an arbitrary point in time.
If you hang your hat on a database known to blow indexes etc you had better be on top of its day to day operation. Preventive re-indexing was my preferred solution the last time I was in such a situation. Also batch data integrity checks run nightly (with new checks added every time new ways to screw the data up are found.)
I expect this database was fucked for some time. A two month rewind wouldn't have made the customers happy ether.
This problem was in the operator wetware.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'