Slashdot Mirror
Internet Archive Gets 4.5PB Data Center Upgrade
Lucas123 writes "The Internet Archive, the non-profit organization that scrapes the Web every two months in order to archive web page images, just cut the ribbon on a new 4.5 petabyte data center housed in a metal shipping container that sits outside. The data center supports the Wayback Machine, the Web site that offers the public a view of the 151 billion Web page images collected since 1997. The new data center houses 63 Sun Fire servers, each with 48 1TB hard drives running in parallel to support both the web crawling application and the 200,000 visitors to the site each day."
I have no idea how much 4.5 PB is until it's given in units of Libraries of Congress.
Does lusting after all their space make me a peta-phile?
Life==Jeopardy. All the answers are right in front us - the hard part is coming up with the correct question.
so all one need to do to "own the internet" is to drive a big rig and ... lift the container off their parking lot?
I can now theoretically steal "the internet" with a flatbed truck and a lift. There's something to be said for conventional data centers: They're rather hard to load onto a truck and drive off with.
#fuckbeta #iamslashdot #dicemustdie
one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot, could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system
floppy disks.
lots of floppy disks.
liqbase
They'd better have it backed-up. Last time the Alexandria library burned-down, we lost about one thousands years of collected information from ancient Greece and Rome. Ooopsie.
"I disapprove of what you say, but I will defend to the death your right to say it." - historian Evelyn Beatrice Hall
You can ship 4.5 petabytes over a single OC-192 link in about 71 days.
yeah, but just at the 70th day, someone will pick up the phone and the whole thing will have to be resent.
--
"It is now safe to switch off your computer."
Can you say, Parallelism?
Parallelogram.... crap
Parallellellell... dammit
Parapalouza... >
Why did you have to point that out to everyone? :(
I'd suggest also using stone slabs. Water can do serious damage to paper, and don't get me started on fire hazards. Good old Stone Slabs resist both of those really well. I'm not sure what the write speed is, however, so you'll probably need to hire many stonecutters to work in parallel.
A math problem. My favorite. I don't know much about stone cutters but lets assume they can write one bit every 2 seconds. Thats 1 byte in 16 seconds. The internet archive is (4.5 x 1,125,899,906,842,624) 5,066,549,580,791,808 (5 quadrillion) bytes. That works out to 81,064,793,292,668,928 (81 quadrillion) seconds or about 2,570,547,732 (2.5 billion) years. That is far to long for their stringent 2 month backup cycle. They would need 15,423,286,395 (15.4 billion) stone cutters to keep schedule assuming they had unlimited stone. Last time I checked there were only between 6 and 7 billion people with only a small fraction of them being stone cutters. That leaves but one solution. Force the web developers to become stone cutters. This would not only increase the work force but also reduce the amount needed to backup because fewer people will be making more web pages to backup.