Slashdot Mirror
Man Deletes His Entire Company With One Line of Bad Code (independent.co.uk)
Reader JustAnotherOldGuy writes: Marco Marsala appears to have deleted his entire company with one mistaken piece of code. By accidentally telling his computer to delete everything in his servers, the hosting provider has seemingly removed all trace of his company and the websites that he looks after for his customers. Marsala wrote on a Centos help forum, "I run a small hosting provider with more or less 1535 customers and I use Ansible to automate some operations to be run on all servers. Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line. All servers got deleted and the offsite backups too because the remote storage was mounted just before by the same script (that is a backup maintenance script)." The terse "rm -rf" is so famously destructive that it has become a joke within some computing circles, but not to this guy. Can this example finally serve as a textbook example of why you need to make offsite backups that are physically removed from the systems you're archiving?"Rm -rf" would mark the block as empty, and if the programmer hasn't written anything new, he should be able to recover nearly all of the data. Something about the story feels weird.
Does he use --no-preserve-root by default? I think that it is there for many years. Of course, if his servers are running on something from 2004, then his rm might be without this safeguard...
I saw the post on ServerFault, and while the original scenario could have happened, the OP's follow-up blunder to reverse the input and output parameters of dd when trying to preserve the disk seemed just a wee bit too unlikely. I looked at the article to see if there was any additional data to suggest this was real, but it seems entirely based on the SF thread. Until corroborated, I'm going to call bs.
While this guy was most likely using traditional HDDs where block level recovery is a possibility, for those of you using SSDs that have TRIM properly enabled, don't expect to be able to recover deleted files from the same drive unless you are really really fast.
TRIM automatically zeros the blocks of deleted files and they are GONE aside from vague sci-fi and probably nonexistent NSA-type forensics.
AntiFA: An abbreviation for Anti First Amendment.
Addendum - just checked a CentOS server, and rm --help says that --preserve-root is enabled by default, and has to be overridden with --no-preserve-root.
"When information is power, privacy is freedom" - Jah-Wren Ryel
Offsite, offline BACKUPS
Would not have helped in this situation. His typo resulted in this command:
"rm -rf --no-preserve-root --write-zeroes --shred-mbr --exec-all-ssh-hosts --douse-hydrofluoric --high-velocity-eject-removable-media --carpet-bomb-offsite-backup --salt-earth"
Which, I mean, who hasn't accidentally done that? The keys are like right next to each other.
Nothing posted to
I have this aliased to 'sl'. Keeps me on my toes.
I make it a point to lump people into the category of "everyone". Then I can despise them all equally without picking and choosing favorites.