Slashdot Mirror
SSD Failure Temporarily Halts Linux 3.12 Kernel Work
jones_supa writes "The sudden death of a solid-state drive in Linus Torvalds' main workstation has led to the work on the 3.12 Linux kernel being temporarily suspended. Torvalds has not been able to recover anything from the drive. Subsystem maintainers who have outstanding pull requests may need to re-submit their requests in the coming days. If the SSD isn't recoverable he will finish out the Linux 3.12 merge window from a laptop."
No backup?
"If any question why we died, Tell them because our fathers lied."
He should listen to Steve Gibson and run Spin Rite.
That's all that Ballmer needs to stop Linux? Just find Torvald's SSD?
"No freeman shall ever be debarred the use of arms." -- Thomas Jefferson
Maybe Linus needs to create a backup program like he did when he wanted a better version control system and created git? Also, why is the only copy of the changes on his local workstation and not a server with redundancy? This seems rather amateurish.
today is spelling optional day.
Linux said "So I don't want to necessarily blame the harddisk, since it's just ten
days since I upgraded the rest of my machine, after it worked years in
the previous one. That just makes me go "hmm". As far as I know, all
the fans etc were working fine, but.."
There's his problem: "after it worked years in the previous [machine]."
His SSD died a natural death of old age.
IMarv
Trusting software vendors is no smarter than trus
Bet you regret knocking those platters of spinning rust now, don't you Mr. Torvalds?
There's no -1 for "I don't get it."
Surely he's not working on a single drive system?
Was he too busy treating people horribly to audit his DR procedures?
Fuck Ajit Pai
....I hear rsync is lovely this time of year....
So Linus got bitten by the same Intel SSD bug that bit me and many others?
Why is this news... is this our version of People magazine, where instead of hearing about all the details of the Kardashians' lives, we hear about every email or event that happens to Linus?
I find it amazing to consider that he is not working on a redundant and well backed up machine. Where's last hour's backup? Yesterday's backup? Even pig farmer's know to backup their data.
I'm no kernel maintainer but...
If his workstation is so important why doesn't he mirror the disks?
Back them up regularly?
Run a remote desktop to a server with the above conditions
Is to send a profanity-laced e-mail to the hard drive. Perhaps then it will see the error of its ways and begin working properly.
Rsync to a second drive with a cron job?
First, there's RAID, so that the death of a single drive doesn't make you lose any time. Just replace the dead disk, and go about your work. Everybody likes to say that RAID isn't a backup, but it's better than nothing. If you have RAID, and pair it with nightly backups of everything that's changed, which is pretty easy to set up, then you're pretty well covered. The only important part being that you actually have to be connected to another machine with a reasonably fast connection in order for the backup to be done. For laptops, this often isn't possible, especially if you are on the road. But even then, you can make sure you copy the really important stuff off to a USB hard disk or some other medium. It's better than nothing.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
If 30 minutes is near enough to real time for you, you could use rdiff-backup on a cron job. That's what I do at home, though I only run it once per day.
What kind of an idiot uses a year-old SSD for critical work? Oh, yeah, the kind of idiot who routinely flames other people for trying to get him to make code changes he doesn't like, rather than explaining his position in a calm, rational fashion.
Karma's a bitch, no matter how famous you are.
...for over an hour when Torvalds had to make an emergency run to Albertson's for some toilet paper and hostility medication.
I've owned several hundred hard drives over the last 30 years. I've never had an active hard drive drive just blank out. I have had drives that had not been powered for a couple of years refuse to ever come back. But if I did not feel the need to even power the thing on for years, you can imagine how little I cared for what was on it.
In the last four years, I've owned around 20 SSDs. I've had five failures. Every single one was the drive just instantly lost everything. Amazingly, in four of the five cases, the drive still worked fine! It had simply lost all the data on it and believed itself to be a blank drive.
That said, the speed of SSDs makes them worth the risk to me. But I take backups far more seriously than I used to. I need them far more often.
They have. You use something like inotify or a DMAPI enabled file system generate a queue of things that need to be backed up and constantly run through it. To get good performance however is going to cost $$$
Looks intriguing, although I see there hasn't been a release in over 4 years. Is it just that good, in your experience?
Well without any pushing to a remote repo it's as mortal as any other source control.
In debates about Christianity, there are two groups: those looking for answers, and those looking to just ask questions.
That is correct. In fact he wrote the code that is the industry standard and uses it every day. How else do you think he is going to continue completion of the project on his laptop.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
Now there a new meaning for Kernel Panic!
I could write something witty for my sig, but instead wrote this...
You beat me to it. Anyone with a vague clue about unix would have thought of that. Obviously vague clues are a rare thing for the parent poster.
We're not complaining because we know full well that Linus has backups. It's called the Git source code repository. All he's lost is time, and assuming that he's saved a whole load by not having to run backups, he's probably ahead on average.
He just needs to switch to his other machine, and re-download the sources from Git. Problem solved, system working as intended. Once he gets a new drive for his old machine, it's a case of re-install the OS, re-sync with Git, done. Maybe copy a few config files off his laptop to save time getting his preferences the way he wants them.
Just how exactly does "that disk was too old" constitute defending Linus?
Sounds like he should have known better. That's probably the way most of the rest of us will take it.
A Pirate and a Puritan look the same on a balance sheet.
I haven't had any problems with it, but I also haven't tested the backups much. I know I've seen it get updated in Ubuntu more recently than four years ago, though. The lack of frequent updates are probably more because of how simple it is. There might be some more advanced features that it doesn't have.
backintime. I am using it, works great, and the restore functions are quite easy to use as well.
Clearly he's just a cheap bastard and didn't want to pay for the extra drive to run mirrored.
A Pirate and a Puritan look the same on a balance sheet.
Having a single point of failure disrupt something as essential as Linux kernel development doesn't instill confidence in the business world. Why are those pull requests not filtered through a separate system running GitHub, and one with some redundancy?
I learned long ago after some close calls to back everything up. In my case for my desktop I store my data on a XFS partition stored on a RAID 5 hard drive array. I also am using Crashplan to back up all of my data, both to a removeable hard drive and to the cloud with over 3TB of data backed up. The nice thing about Crashplan is that it continually backs up, taking periodic snapshots so I can restore a previous version of a file if I wish. The main drawbacks of Crashplan are that it runs on Java and can be a memory pig. I pay $6/month for unlimited backup of up to 10 machines and have several computers backed up with them now. With the proper settings on my router I don't even notice all the backup traffic running in the background.
Since I have had sudden SSD failures in the past I also dump my root XFS filesystem weekly onto my RAID array (it takes under a minute to run xfsdump) and incremental backups nightly and those dumps get backed up on the cloud as well.
I have found the XFS tools to be quite good at recovery when things go really bad. When running software RAID 1 I had problems where drives would drop out of the array for apparently no reason and I have had several occasions where while rebuilding the other drive would pop out of the array. Switching to an Areca hardware raid controller with battery backed DRAM ended those problems (besides seeing a big performance improvement).
I have found the RAID controller to work well when drive failure occurs and it even recovered after human error (I accidentally disconnected one of the active drives while it was rebuilding and reconnected it).
I won't use btrfs yet. The last time I tried it about 6 months ago it was quite slow and I have a lot of concerns about the storage filling up due to COW that have not been adqeuately addressed as far as I could tell. I tried setting it up for a Cyrus IMAP server on an Intel SSD and it was unusably slow just untaring all the files so I ended up going back to XFS.
SSDs are still relatively new. I have had issues with some firmware versions and had one fail catastrophically after only 2 weeks of use. I have also had compact flash and SD devices suddenly fail. My experience is that usually mechanical hard drives give some warning (i.e. SMART) and they tend to last years. I have a server I just retired where the hard drive had 10 years on the clock according to SMART.
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
I don't feel anything but shame for someone losing data in a hard drive crash who has or should have network backups available to them. If this happened to anyone but Linus the majority of the comments would be calling the coder a n00b. If it was Balmer there would be an absolute riot of anti-MS venom....
I guess the great Linus has fallen into shadow.
As someone who's taken over server administration from very talented developers a number of times, I've found that being a great developer doesn't mean that you're a great sysadmin. Developers may understand conceptually that RAID and backups are important (but sometimes think that RAID is a backup), but that doesn't mean that they actually set them up.
I think I speak for everyone here when I say... this
I'm a good cook. I'm a fantastic eater. - Steven Brust
And is back at work Jesus h Christ everyone on the planet has a copy.
Does this mean all Linux Development will stop until they come up with a way to prevent or recover quickly from SSD failures?
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
I'm not nearly as much of a believer in RAID for the home environment. If you (accidentally) delete something on one drive it's gone from both. Better to buy two drives and do a daily rsync. That way you have a window of opportunity to recover data. Personally, I use rsync without --delete until the 2d drive starts getting full, then I use the --delete flag to clean up.
Competition Good, Monopoly Bad.
... to protect him from this kind of calamity, for example by using mirrored disks (even with SSDs) whenever possible and by ensuring that regular backups are made of all of his important data. Of course, privacy-wise it would be better if he did it himself, but apparently it's just not a priority. I've long found it curious that so many excellent programmers are comparatively inept when it comes to looking after their own machines and data.
I have a mirrored set of SSD's on all my important machines, and RAID 6 for bulk storage.
Unlike Linus, I can't afford to lose work.
"To those who are overly cautious, everything is impossible. "
rsync to the backup drive with the --backup switch. That way if the file system decides to overwrite a bunch of key files with 0 length files, your backup keeps the originals. It might take you a while to restore, but at least the originals are still there.
The backup heirarchy:
- Natural disasters occur rarely.
- Hardware failures occur infrequently.
- Software failures occur more frequently.
- User failures occur often.
- The unexpected happens all the time.
http://www.wired.com/wiredenterprise/2012/10/linus-torvalds-hard-disks/
"Let us raise a standard to which the wise and honest can repair" - George Washington
So buy a new drive with the same rev boards and swap them out. Problem solved.
Only the State obtains its revenue by coercion. - Murray Rothbard
According to a speech of his, that's how Linux got started. He accidentally wiped his MINIX partition.
I run smartctl regularly to check on my disks (SSD or spinning) but I find the info difficult to interpret. Is there a service where I can upload the reult and it distills it to: fine OR dying ?
Non-Linux Penguins ?
As someone who's taken over server administration from very talented developers a number of times, I've found that being a great developer doesn't mean that you're a great sysadmin. Developers may understand conceptually that RAID and backups are important (but sometimes think that RAID is a backup), but that doesn't mean that they actually set them up.
And as a sysadmin, I'm tired of hearing that. RAID1,5,6,10,Z is a backup. It's not an archive. An archive is what you go to when you want the old version. A backup is generally one of two things:
1) Something that lets you keep chugging through a failure (raid5, a backup generator with automatic cut-over, etc)
2) A standby spare (tape, NAS/usb drive, secondary location with desks/computers/etc.
RAID (other than 0) is absolutely a backup. It's not the perfect backup but it is a backup. What it is NOT is an archive - last night's/week's/month's/quarter's data.
No, RAID is *not* a backup, RAID's only purpose is to improve reliability/uptime by letting you ride through hardware failures, but it does nothing to protect you from all of the rest of the things that can destroy your data, like file corruption, fat fingering a "rm -rf / home/someuser", a virus, a website hack attack, etc. That's what your backups are for, but you can call them archives if you like, but don't call RAID a "backup" because it's not. Depending on what the problem is and when you discover it, you may need to go back through several archives before you find the data you're looking for.
That is a ridiculous statement. Work is lost every time a drive fails unless it happens to fail immediately after a backup. Full backups take lots of time. If you understood git better you would realize that a lot less work is lost the git way than with old school backups. I'm sure that every time Linus does a successful merge he pushes it to a git repo elsewhere. All history is in the git logs. I am certain the work he lost is minimal, and is much less than if he was relying on nightly backups and the failure happened near the end of the work day. Just the effort of trying to determine what was done and what has been lost would be far more time consuming without git.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I'm starting to look like a shill in the comments for this story, but what the hell. I've been happy with Crashplan. They charge you to store on the cloud, but it's free to your own computers, or friends computers. It also runs a hash against everything it backs up, so data integrity is supposedly "assured". Another commenter has been going on about Bacula, but I've never tried it.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I don't care what the mods say - this is funny as hell.
"Ignorance more frequently begets confidence than does knowledge"
- Charles Darwin
This might be [electrolytic] capacitor or some other component-level magic-smoke release. There is also the dreaded, much-discussed "wear" from re-writing flash memory -- worse than you think because blocks of 64 KB [typically] have to be erased and re-written to change any byte therein.
Linus, of all people, ought to know his kernel has options to minimize the re-writes, many of them developed to optimize laptops (like delaying writes). Another thing is to mount partitions (/etc/fstab anyone?) with `noatime` as an option (maybe 'nodiratime` too). Un*x and other Linux-like systems by default will re-write the access time for any disk inode read. Turning it off reduces disk write load (and seeks on slow disks). I've had it off for over ten years an not noticed any malperformance, althrough there are rumored to be some, somewhere.
It is pretty clear you haven't thought this through. What happens to the performance of git, or any software for that matter, when the data it accesses is somewhere else on the internet? How much work do you suppose is lost every day doing it that way?
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
As someone who's taken over server administration from very talented developers a number of times, I've found that being a great developer doesn't mean that you're a great sysadmin. Developers may understand conceptually that RAID and backups are important (but sometimes think that RAID is a backup), but that doesn't mean that they actually set them up.
And as a sysadmin, I'm tired of hearing that. RAID1,5,6,10,Z is a backup. It's not an archive. An archive is what you go to when you want the old version. A backup is generally one of two things:
1) Something that lets you keep chugging through a failure (raid5, a backup generator with automatic cut-over, etc)
2) A standby spare (tape, NAS/usb drive, secondary location with desks/computers/etc.
RAID (other than 0) is absolutely a backup. It's not the perfect backup but it is a backup. What it is NOT is an archive - last night's/week's/month's/quarter's data.
So much fucking wrong.
RAID provides redundancy. It is not a backup.
A spare (hot or cold) is not a backup. It's a spare.
When you're talking about DATA, a backup is a COPY of that DATA on a physically-separate device. Ideally, all backups should be UNPOWERED and MOVED THE FUCK AWAY after being created. For my home use I don't do either of these, but for work we have 3 or 4 layers of backups and separation depending on how you want to count it.
It not like the GIT repo is hosted on his home machine...
Tomorrow is another day...
So an SSD in a desktop computer died. So what? Just run the array in degraded mode until the damaged drive is replaced.
Who uses hard drives (SSD or otherwise) in desktops in anything other than a RAID configuration? In Linux where software RAID-1 is trivial there's even less of an excuse.
In a laptop I can understand it, since there's often only space for one drive and even then you expect everything on it to be volatile.
I don't get it.
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
You're confusing the English with technology. A backup hamburger -- in case the hamburgler steals my first one -- is not a backup of my data. A backup is a short term copy of my data. An archive is a long term copy of my data. Neither one changes after they've been created. A good backup has left the building. RAID, UPSes, generators, etc. are availability measures, not backups.
> Hasn't Apple's Time Machine sort of set the bar a little higher on frequency of backup?
No, not really.
A Pirate and a Puritan look the same on a balance sheet.
trying to desolder 100 pins spaced 0.01" apart then resoldering them, unless you have a 0.1 mill precision soldering robot it is impossible, you can't even buy wire thin enough to do it by hand.
SMT rework by hand isint rocket science, but takes more tools than the average garage has.
Desoldering you use a custom tip for that socket/package type (one tip per package & they're not cheap). It's essentially a metal ring that heats the solder on all the pins at once. In the center of the assembly is a vacuum probe. You heat all the pins, melting all the solder & hit the button on the handpiece to suction the chip up off the board. Then clean up the pads on the board. Careful with the heat because you dont want to lift pads off the board, if you do then you have to either fix them, or make a new pads. And then if you manage to trash a via (conductivity path to a different board layer), then you've got to drill out a new one and you have to use a esd safe conductive drill with a resistance cutoff. You put a clip from the drill in contact with the layer you're trying to get to, drill down and when the drill tip makes contact with that layer the drill turns off because the circuit is complete. But it still sucks and if you don't know how all the board layers are put together you may end up trashing a trace a couple layers into the board and wrecking the whole thing.
Soldering it down you do this. Align all the chip legs on the pads. Then you can either run a small bead of solder paste across all the pins or use a wave soldering tip (small cup, uses surface tension to hold the solder in place) and drag the tip over all the pins. Heat on the pin & pad draws the solder down into the joints. If you put too much solder you might have to vac it back up and redo it if you've made bridges etc. Alignment is key, and keeping the part in position is key. I used to try and avoid using glue underneath because that made it difficult to get it back off if you needed to down the road.
Doing hand rework on that kind of stuff the hardest thing for me was dealing with smt chip caps, little bastards will crack if you heat em to fast, so you have to get a temp regulated hot plate, heat em up slow, then pick and place em quick with tweezers/needlenose & solder em down quick.
01:36AM up 426 days, 2:46, 1 user, load average: 0.14, 0.11, 0.05
You'll notice that I singled out frequency. :)
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
You have a software feature in a server OS that supports certain client OSes to do backups to the server. RAID may be a software feature, but even if it's "software raid", you often have BIOS bootable raids that even work with one of the drives missing. This essentially means that you can work OS agnostic on a lower level than "I have a backup system that works". For Linux, you can have a backup system too that will restore from a LiveCD/USB stick and stores on a remote server. The same amount of time roughly will be needed to backup and restore, differential, incremental, full backups, the works. The solution you are providing is really nothing comparable to RAID. It's fundamentally different because it works on a totally different layer, doesn't prevent downtime and it's not OS agnostic. RAID should prevent downtime, making working backups should prevent data loss. Maybe WHS is the shizniz, you rock for making actual backups, but other than that, your post is totally offtopic in this context and doesn't even begin to solve a problem that Linus was facing with his desktop.
I'm not modding you down, even though I have mod-points, but I'm telling you exactly why I think you shouldn't have posted this. I hope you learned something from it and in the future will implement both backups and RAID when unscheduled downtime is important. Maybe you would even implement a system that works for all relevant OSes in the environment you have to do it for, without relying on a single vendor that offers a closed source product. It's a risk that means you'll have to support their product and licencing and other requirements until the data isn't relevant anymore, even after you have migrated to a competing product.
I was promised a flying car. Where is my flying car?
Modern drives for the last five years at least, have calibration factors for platter/head packs on the EEPROM on the controller board. If you swap boards, the board most likely won't be able to read the data on the disk, since it's not calibrated to the head/platter kit.
I was promised a flying car. Where is my flying car?
Huh? Dropbox is a local folder. A daemon monitors the folder for changes and then uploads those changes to the Dropbox servers. It keeps a version history for each file, and so you can restore to a point in the past. In my experience, it does not use much in the way of system resources. Sparkleshare does the same thing, but to a server of your choosing and it is completely open source - and in fact based upon git.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
RAID is NOT a backup.
Raid does only a few things...
Depending on level, allows you to survive $X number of disks failing.
Allows you to 'stripe' a file system across multiple drives for increased performance or size.
It does not protect any other problem, such as raid card failure, filesystem corruption, rm -rf, or any other number of problems that can occur.
So you are saying why not use a wrapper around git. Well, we've really come full circle now haven't we. Me: git already solves this problem. You: why not use git to solve it! (Sparkleshare) Now, let me tell you why asynchronous use of git is a bad idea. git is making changes on the drive, and commits are atomic. You can easily set up git hooks to automate the process synchronously. There is no reason, and no advantage to, using a separate git repo and storing your whole git repo in a git repo. It is at best going to slow things down, and at worst it will hose your repo. I can never decide if it is hilarious or sad that people think they could teach Linus a thing or two about software development. The likelihood that it is true is very close to 0x00;
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I agree with everything you said, other then "RAID's only purpose'. RAID also allows you to create volumes of much larger sizes then the original disks and allows greater read/write and IOPS by treating many disks as a single device.
Linus needs a personal sysadmin to protect him from this kind of calamity ...
All newbs do. And newbs don't usually learn to appreciate backups until they suffer a catastrophic loss. This just proves, yet again, that brilliance in one domain does not imply brilliance in any other domain. Of course the clueless don't recognize this and think a very successful person must have all the answers, when in fact they are equivalent to newbs in many domains.
This is retarded. In this day and age to have somebody the "master of the universe" for doing merges on one machine is not only retarded but it's bad practice. Can you imagine a large company shipping a release and suddenly having to stop because a system or a drive failed? People would be answering up the food chain in a hurry. Shit, forget all of the best practices in terms of backup and redundancy to protect your business in the "real" world that would apply here but we have to account for Linus' imaginary world where he rails on and on over crap that really doesn't matter and pontificates about how "dumb" other people are.
What we have here is a Nelson moment... To all that have been belittled by his rude, arrogant nature in the past you have the right to say "Ha! Ha!"
Harrison's Postulate - "For every action there is an equal and opposite criticism"
I agree with everything you said, other then "RAID's only purpose'. RAID also allows you to create volumes of much larger sizes then the original disks and allows greater read/write and IOPS by treating many disks as a single device.
Yeah, sorry, I was speaking only in the context of redundancy, and misspoke when I declared that the *only* purpose of RAID is redundancy.
Maybe Linus needs to create a backup program like he did when he wanted a better version control system and created git? Also, why is the only copy of the changes on his local workstation and not a server with redundancy? This seems rather amateurish.
Brilliance in one domain does not imply brilliance in any other domain. Only the clueless think very successful people have all the answers. Only the delusion-ally arrogant think they have all the answer. Only the foolish think it can't happen to them.
Yep, Linus was acting as a true newb if anything more than a small number of hours worth of work was lost. Especially so given the known characteristics of SSD drives. Not very far removed from the person who types in a word processor for hours without saving.
There is no reason, and no advantage to, using a separate git repo and storing your whole git repo in a git repo.
Except, you know, that it would be backed up if the drive died.
I can never decide if it is hilarious or sad that people think they could teach Linus a thing or two about software development.
I don't think "running a backup application" is part of "software development", and it's clearly something he needs help with. The man is clearly not as infallible as you seem to think he is. His mistake now requires work from other people to clean up.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I hope that all of these resubmitted patches are exactly the same as they were before. I would hate to see that this was used as a vector to add a backdoor into the kernal.
Blah Blah Blah.
Do you work for me, because if you do, you're fired.
Wow. I mean seriously. Wow. You are just hell bent on being a moron. You offered "solution" after "solution" that is no solution at all, and seem to have missed the first line of his post: "I had pushed out _most_ of my pulls today, so realistically I didn't lose a lot of work." I have to assume you don't work in the real world, because if you did you would realize how truly asinine you sound.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
Who's going to be the one to rant at *him* on the mailing list for this?
Err... nearly no one does contact rework anymore professionally, and even half-serious hobbyists nowadays at least have a AOYUE and a high temp vac pickup.
Then just clean the site, apply mini stencil, squeegee paste, remove stencil, place new part, reflow.
Yeah, personally I haven't done SMT rework in about 15 years, Aoyue sure has brought the price down on rework stations, that's less than I paid for my Metcal, and that's just a basic iron. I don't want to remember how much we paid for some of the larger Metcal & Pace hand rework systems back in the day.
01:36AM up 426 days, 2:46, 1 user, load average: 0.14, 0.11, 0.05
You offered "solution" after "solution" that is no solution at all,
You can say that all you want, but if he were running Crashplan, he'd have lost maybe 20 minutes of his working day. Switch to one of his other computers, restore his working directory to a new directory and keep on truckin'. Instead he's struggling to recover data from a dead drive. Dropbox or Sparkleshare wouldn't even require a restore step - just walk over to the other computer and pick up where you left off. Do you dispute this fact?
and seem to have missed the first line of his post: "I had pushed out _most_ of my pulls today, so realistically I didn't lose a lot of work."
And you seem to have missed the part where he's been trying to recover the dead drive and told people that they will need to re-push their changes over the next couple of days. Does that sound like a trivial amount of work lost to you? Days of recovery?
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
Because doing a git push takes so much effort? They should still have all the code in their local repos.
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Okay yes, I have to admit git can get gnarly. But if they do it in the right order it should theoretically be fine.
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Do you work for me, because if you do, you're fired.
Wow. I mean seriously. Wow. You are just hell bent on being a moron.
if you did you would realize how truly asinine you sound.
Pot: Kettle.
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
You need to learn about ZFS, as it does protect against raid card failure (you don't use one), filesystem corruption (multiple levels of checksuming AND verification), rm -rf (hourly snapshots)
What else do you have?
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
And that is still nothing like Time Machine.
TimeMachine is not just a copy of your latest files, its snapshots of your machine. Let it 0 byte a bunch of files, just go find the last snapshot before they were 0 bytes and restore.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
I'm sure all of the data still exists somewhere - git is very nice for that. However, there is time and work involved with corralling all of it and getting back to where he was. Had he been using even the most brain-dead, consumer-level backup schemes like Dropbox he wouldn't have lost any work at all. If you read through those kernel mailing lists, the git pushes often seem quite eventful :)
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
This is always the story with SSDs. They suddenly die, even though all the literature from the manufacturers says this is not a mode of failure that ever happens. Supposedly, the drives enter a read-only state where they can only be read, because they are out of space to wear-level. Has anyone ever had a drive actually do this? I want Linus to send this drive back, and get an answer from the SSD manufacturer as to what happened.
Doing hand rework on that kind of stuff the hardest thing for me was dealing with smt chip caps, little bastards will crack if you heat em to fast, so you have to get a temp regulated hot plate, heat em up slow, then pick and place em quick with tweezers/needlenose & solder em down quick.
Maybe your iron doesn't have good enough thermal control, and/or is at too high a temperature? I used to swear by Weller irons, until I was coerced into trying a Metcal. I was amazed at what a difference a really good iron makes. I routinely hand-solder down to 0402 size components without any problems at all. I often have to rework 0201 sized ones, and those are harder. But with a good iron, appropriate use of flux, good minimum-size tips that you don't use for anything larger than 0201, treat carefully, and definitely don't let anybody else borrow, it's not that bad. When I need to tune up an RF path at work, I'll end up changing the same few 0402 or 0201 passives a dozen times or more, without ever cracking a component or lifting a pad.
Oh, and one other thing: If the board was built with lead-free solder, wick that crap off and do your rework with proper 63/37 tin/lead solder! It'll make better joints, and it melts at a lower temperature.
No, RAID is *not* a backup, RAID's only purpose is to improve reliability/uptime by letting you ride through hardware failures,
This, RAID is high availability and in no way a backup.
Why?
Because if data gets corrupted or accidentally deleted it gets corrupted or accidentally deleted across the entire array. RAID will keep you going if a disk dies (most of the time) but it wont recover data. Also remember RAID is vulnerable to the RAID controller dying (had that happen to an Adaptec years ago, it was a good thing we had backups).
Calling someone a "hater" only means you can not rationally rebut their argument.
I know I'm less than careful about backups myself, and only push an incremental backup to cloud storage about once a week, but then again my laptop is not a single point of failure for the development of the largest operating system in the world.
(As for desktop machines, it's pretty trivial to set up a RAID-1 and not have to rely on periodic backups at all.)
Why are you using an iron? Use hot air and an oven, it makes SMT a completely different game. Just make sure you use lots of flux.
Help I am stuck in a signature factory!
Old habits die hard, I guess. I rework existing boards a lot more often than I do assembly. For changing out discrete resistors/caps/inductors, a pair of good irons works very well. An iron is also preferable for tacking wires onto test points.
If you read the detail you would see he lost a grand total of three commits ... that he then took from LKML and promptly reapplied to his new instance.
"Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)" - Linus Torvalds[1]
Pfff... That's soooo last century!
Let me fix that for you, Mr. Torvalds
"Only wimps use tape backup: real men just upload their important stuff on git, and let the rest of the world clone it"
Now that sounds more typical for the current decade.
Oh, and for the MasterCard-Ads like finish:
"For everyone else, there's the NSA."
----
The funniest part is that he is the actual author of the git scm system which served him as backup this time.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
rsync is nothing at all like time machine.
Look a little harder into the versioning aspects of time machine and let me know how to make rsync do that.
And when you've done that, please let me know how to get rsync to manage space and remove the oldest backup first when a full backup cannot complete.
Not sure how to raid my laptop without voiding the warranty, opening it up and replacing the DVD drive with a second SSD...
I looked into crashplan early this year and decided I didn't want it because they can't guarantee that my data will be safe from US authorities.
With some of what we've learned this year, I'm glad I chose not to do that ...
Until the day when an obscure bug* triggers a cascade of events flagging you as a "Pedo-Terrorist Pirate Pronographer".
*: Or a fly lands in a typing machine
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
If only he'd been using a centralised version control system... ;-)
Not just Linus himself, but also his sole hard drive!
This is what concerns me about Linux: a single egg-basket so threadbare as to be nearly see-through.
Just so this is totally clear - SSDs die too!
Now we all know we can see the point of RAID-1 (inclusive)OR frequent backups in any critical system.
PlusFiveTroll was dead on when he said RAID isn't a backup, that ZFS isn't just RAID, it's a whole lot more. ZFS>RAID.
How can anyone make such a claim and be taken seriously? A warrant will always let the government have your data.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
Where does it say that? I see in the summary and linked article where it says he's trying to recover the drive and if he can't then people will need to re-submit over the next couple of days.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
RAID is not a backup BUT it can help when you absolutely must recover data. For mission critical work, you need both.
Remember kids, if you're not paying for the service, YOU ARE THE PRODUCT THAT IS BEING SOLD.
How many minutes of his working day did he lose?
Where did you get that idea?
Oh my God! They will each have to execute a single push command! The horror!
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
How many minutes of his working day did he lose?
I have no idea - what is your estimate? I can only go by the information in the linked article, that it will take a few days to sort out:
"Torvalds is attempting to recover the dead hard disk but at the moment it doesn't appear easy and subsystem maintainers who have outstanding pull requests may need to re-submit their requests in the coming days."
Oh my God! They will each have to execute a single push command! The horror!
Making other people pay for your own mistakes is bad karma.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
Considering the importance of what he does, why didn't he use a multi-drive RAID or ZFS system?
Rack Mounted Server. I just gotta know, why is all this mission-critical operational stuff taking place on a workstation with workstation grade hardware and no backups or raids? Everyone's talking about oh raid at home isn't good, just use backup drives. Look: This is LINUX. If there's need for additional hardware and compile farms, people will probably donate. To have a single SSD failure cause so much calamity for any project, least of all *THE* open source project, is just embarrassing. Worse than swearing at your devs on a mailing list read by the whole world.
Sadly, a Libertarian cannot force his views on another, and freedom cannot spread as does the cancer known as religion.
Lenovo has some laptops that support changing out the optical drive for a second hard drive. They have some really nice stuff. I'm considering them for my next laptop purchase.
Anthropic principle: We see the universe the way it is because if it were different we would not be here to see it.
RAID is a backup scheme for hardware, not for data.
I have found there are just two ways to go.
It all comes down to livin' fast or dyin' slow. -REK, Jr.
No, RAID is *not* a backup, RAID's only purpose is to improve reliability/uptime by letting you ride through hardware failures, but it does nothing to protect you from all of the rest of the things that can destroy your data, like file corruption, fat fingering a "rm -rf / home/someuser", a virus, a website hack attack, etc. That's what your backups are for, but you can call them archives if you like, but don't call RAID a "backup" because it's not. Depending on what the problem is and when you discover it, you may need to go back through several archives before you find the data you're looking for.
And yet... If your drive fails just before your scheduled "Backup" starts then if it was part of a redundant RAID then guess what? Your RAID just saved that data yes? It acted as a back-up for at least an entire day's work where-as your official "Backup" did nothing for you in regards to that data.
So yes, RAID "by itself" is not a reliable back-up system in every case. But then, neither is back-up software a 100% reliable back-up system in every case. Clearly both together are actually required in order to have a truly effective back-up system, not just back-up software by itself.
That's why you should tune your Recovery Point Objective appropriately. There are failures that can take out an entire server and/or RAID array, so if you can't stand a day of lost data, you should be making backups more frequently.
RAID is not part of a backup strategy, only backups are a backup strategy. RAID protects you only against a particular type of hardware failure (i.e. hard drive failure) and is part of your high availability strategy. Don't count on RAID to replace a sane RPO for your backups.
I have multiple backups in seperate locations and online redundancy (saving 50 past versions of the file, in case of damage, deletes or errors). One offsite backup is a passive one (in case of a systemwide hack) that cannot be accessed from the outside through the net. And this is just for my personal stuff...I'd expect an important person like Linus Thorvald to have a minimum of a basic backup or at least a NAS or online storage.
I don't want to know the tirade that drive is receiving right now.
I know that - I use git. It's great, but it is not a complete computer backup system. By its nature, it does a good job keeping copies of Linux all over the world, so that even when the very smart but also apparently very cocky maintainer drops the ball, it only costs the project a few days. However, had he run any of the very, very simple to setup backup applications available out there, we would not have even heard about his SSD dying. No one should know that Linus's drive died - it should be a non-event.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
FYI, in a study Google did of thousands of drives, they found that while certain models of drive were good and some bad, all manufactures had similar failure rates. Western Digital makes some good models and some bad ones, as do all of the other manufacturers .
That said, I run 40 drives at a time. In my environment, at least, HGST (formerly Hitachi) has had the best track record for me.
That is a ridiculous statement. Work is lost every time a drive fails unless it happens to fail immediately after a backup. Full backups take lots of time. If you understood git better [SNIP]
Full backups? LOL, son.
It's not a ridiculous statement. Our backup system backs up 350+ machines every 15 minutes by default, as long as they have a working network connection, anywhere in the world that can reach our server; the client works by watching what files are changed, and periodically (every 2-3 days) doing a full scan in case it missed something. We dialed it back to once an hour based on user feedback - people felt an hour was more than acceptable in terms of lost productivity. We retain those revisions for about a week, and they're progressively paired down. Restores take seconds and are self-service, as is adding another machine to your account.
Furthermore, we use IMAP for email, so even if your workstation or laptop dies in a big puff of smoke, your email isn't lost.1995 called, wants to know why the fuck Linus is apparently using POP3.
If I had a dollar for every prick developer who thinks they know how to do IT, I'd be rich (and a lot saner.) Programmers are the worst to support by far because they have absolutely zero humility. Everyone else generally either asks how something should be done, or at least has the graciousness to ask if what they have in mind will work. Programmers just charge ahead and assume they know what they're doing because they've got a Mythbuntu box and a Linux NAS box at home...
Please help metamoderate.
find it ironic that the biggest and loudest hater of ignorance himself was victim. if anyone else in that circle had that happen it would be message after message of "what a stupid cocksucker you are". what a moron/asshole
short hint: search for 'chip quick' (mouser and similar places sell it).
its like solder that stays liquid for a long time and lets you remove chips without special equip.
(you're welcome).
--
"It is now safe to switch off your computer."
Nowadays I actually prefer SMD over thru-hole... with one exception, BGA.
Hand reworking of them is seldom rewarding, especially the now more common FBGA.
And, It's likely that most of the chips on a device like an SSD are FBGA.
The pitch really isn't within the reach of human precision, so you *need* a pick & place machine.
You also need an IR heater, and X-ray to check the connections.
We can all hear the frustration in your message for having lost data, and for that I am sorry you are experiencing that. You made some interesting statements that I have not seen myself. All the current SandForce SSDs out there have been clear of firmware bugs like this for more than a year from what I see. Mine have all been fine. Are you running with some old FW maybe? The problems I see are more often physical issues that are unlreated to the controller. It is important to select an SSD manufacturer with high quality manufacturing, and don't just go for the cheapest solution. Intel uses SandForce almost exclusively now, so I find it hard to believe what you are describing is 100% true.
I'm fine with information being gained under a warrant because then I know someone is looking at me.
With the US warrantless searches and gag orders where warrants are used, I can't be told that I'm being investigated. That doesn't sit well with me, so I'm avoiding US based services for new stuff and moving my existing services out of the USA.
Ah, now I see your point. Crashplan has an option where you can supply the encryption key. That means the government would have to come to you (or compromise the Crashplan app on your computer) in order to snoop your stuff.
W..w..W - Willy Waterloo washes Warren Wiggins who is washing Waldo Woo.
I wasn't aware of that... That makes crashplan much more appealing actually. Thanks - I'll read up on it!