Internet Archive Gets 4.5PB Data Center Upgrade

Lucas123 writes "The Internet Archive, the non-profit organization that scrapes the Web every two months in order to archive web page images, just cut the ribbon on a new 4.5 petabyte data center housed in a metal shipping container that sits outside. The data center supports the Wayback Machine, the Web site that offers the public a view of the 151 billion Web page images collected since 1997. The new data center houses 63 Sun Fire servers, each with 48 1TB hard drives running in parallel to support both the web crawling application and the 200,000 visitors to the site each day."

Comments

Where do they store 4.5TB off site

[wjh31]

one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot, could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system

Re:Where do they store 4.5TB off site

[fuzzyfuzzyfungus]

TFA indicates that they have a mirror at the library of Alexandria. Unless things have changed since last I read about them, the mirroring is pretty much it. The Internet Archive does very impressive work; but they don't have that much money. No Real Big Serious Enterprise tape silos here.

Re:Where do they store 4.5TB off site

[LiquidCoooled]

one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot, could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system

floppy disks.
lots of floppy disks.

Re:Where do they store 4.5TB off site

[MichaelSmith]

Its like the two USB hard disks I use for backups. Pick up the container and swap it with the container from secure storage,

Re:Where do they store 4.5TB off site

[MrEricSir]

It's simple, the backups are compressed -- they simply remove all those useless zeroes from the binary data.

Re:Where do they store 4.5TB off site

[DigiShaman]

Umm, how many forklifts and 18 wheelers does it take to swap out 4.5 petabytes worth of data each day?

Re:Where do they store 4.5TB off site

[CannonballHead]

[subject correction]
PB, not TB... hehe.

Re:Where do they store 4.5TB off site

[clarkkent09]

It's 4.5PB, which is a whole different thing, and TFA says it's mirrored at the library of Alexandria, Egypt. I guess that counts as off-site :)

Re:Where do they store 4.5TB off site

[Bearhouse]

Not reliable enough.
I suggest that this important resource be backed up to punched cards.
This would also enable handy comparisons in units that us oldies understand, such as ELOCs
(Equivalent Library of Congress).
I'd calculate it myself, but seem to have mislaid my slide rule...

Re:Where do they store 4.5TB off site

[commodore64_love]

They'd better have it backed-up. Last time the Alexandria library burned-down, we lost about one thousands years of collected information from ancient Greece and Rome. Ooopsie.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

They have Charlie Babbitt on their staff. No need to replicate.

Re:Where do they store 4.5TB off site

[houghi]

If they need much more, I have some AOL disks laying around that they can use.

Re:Where do they store 4.5TB off site

[TheGratefulNet]

one would assume that something like this does regular off-site back-ups

there are BIG fat cables you connect, wait 3 seconds, then do a massively parallel 'dd if= ...'

Re:Where do they store 4.5TB off site

[__aasqbs9791]

I'd suggest also using stone slabs. Water can do serious damage to paper, and don't get me started on fire hazards. Good old Stone Slabs resist both of those really well. I'm not sure what the write speed is, however, so you'll probably need to hire many stonecutters to work in parallel.

Re:Where do they store 4.5TB off site

[TheGratefulNet]

It's simple, the backups are compressed -- they simply remove all those useless zeroes from the binary data.

in music today, there is a so-called 'loudness war' and I think I've discovered what it is: they're removing the zeroes, thinking that 'all ones' will make the music even louder!

I wonder if its reversable? where do the zeroes go? can they be unzeroed? we should try to find them.

Re:Where do they store 4.5TB off site

[Wingman 5]

Jeeze how much bandwidth would you need to fill that thing in one day... runs to google.
(4.5 petabyte) / (1 day) = 54.6133333 GBps
According to List of device bandwidths the closest things to filling it in one day are:

• HyperTransport 3.1 (3.2 GHz, 32-pair) 409,600 Mbit/s 50 GB/s
• PC3-16000 DDR3-SDRAM (triple channel) 480.4 Gbit/s 48.4 GB/s

Re:Where do they store 4.5TB off site

[Sponge Bath]

Followed by run length encoding of the remaining ones.

Re:Where do they store 4.5TB off site

[pedrop357]

4.5TB isn't that bad. Heck, we have 1TB tapes right now. 5 of them can be carried in a small bag.

It's the 4.5PB that the Internet Archive could use that's hard to store offsite. 4500 1TB tapes can be pretty unruly.

Re:Where do they store 4.5TB off site

[jd]

When you get right down to it, any hard-coded data on silicon is just data on a stone slab. Since you can compile SystemC into a hardware spec, you can write stone slabs as fast as you can generate C.

Re:Where do they store 4.5TB off site

[wealthychef]

Can you say, Parallelism?

Re:Where do they store 4.5TB off site

[medelliadegray]

i find it impressive they have all that hardware for a mere 200k users a day.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

In Brewster Kahle's December 2007 TED talk he mentions a third mirror in the Netherlands.
http://www.ted.com/index.php/talks/brewster_kahle_builds_a_free_digital_library.html

As he puts it, the Archive is mirrored on 'a fault line, a flood plain, and in the Middle East'.

Funny thing is I can't find another reference to the Netherlands mirror. The Bibliotheca Alexandrina site mentions a plan to eventually have four sites (California, Alexandria, Europe, and Asia), but that's it. Anyone know what happened with the Netherlands site?

Re:Where do they store 4.5TB off site

[Anonymous Coward]

one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot, could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system

Create snapshot of zpool (think LVM VG):
# zfs snapshot mydata@2009-03-24

Send snapshot to remote site:
# zfs send mydata@2009-03-24 | ssh remote "zfs recv mydata@2009-03-24"

Create a new snapshot the next day:
# zfs snapshot mydata@2009-03-25

Send only the incremental changes between the two:
# zfs send -i mydata@2009-03-24 mydata@2009-03-25 | ssh remote "zfs recv mydata@2009-03-25"

Now this looks a lot like rsync, but the difference is that rsync has to traverse the file system tree (directories and files), while ZFS only has to look at the 'birth time' (think ctime) of each block of data (not even the full file metadata) to see if it's newer than the first snap shot. If you're talking about tens (or hundreds) of thousands of directories, and an order of magnitude more files, that's a lot of overhead if nothing has changed. For 48 TB raw (what a Sun X4500 can have), ZFS can see nothing has changed in a few minutes.

Creation of snapshots is instantaneous and there is no overhead in them (except that the space from deleted files isn't reclaimed / reused). There are people who create them every five seconds, and sync with a remote server--so at most you would lose five seconds worth of data if your disk died.

All changes are also ACID, so if you start your send-recv, and the transmission dies part way through, the receiving end won't have a partial copy of the data latest snapshot--it's all or nothing of the last good change.

Re:Where do they store 4.5TB off site

[rackserverdeals]

one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot, could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system

Dude, the Internet Archive IS the offsite backup.

At least mine anyway. Tape drives be damned.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

I'm surprised that you got no serious answer.

In that scale, you don't backup anymore, you save the data tripple (or more) redundantly in the first place.

In essence, all the machines form sort of a peer-to-peer network with a distributed hashtable (DHT) to store and lookup data.

The 'petabox' had a pretty cool website once which explained all that in detail, but it seems to redirect only to a page on archive.org now...

Freenet or YaCy may also be good examples of that technique which offer background (basic concepts understanding) material if someone is curious.

Re:Where do they store 4.5TB off site

[dziban303]

(The truth about AIG and Congress.) http://www.foxnews.com/video2/video08.html?maven_referralObject=3833532 - Watch now

I can't take anyone seriously who puts "truth" and a link to Fox news in the same signature.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

"In that scale, you don't backup anymore, you save the data tripple (or more) redundantly in the first place."

Hypothesis: nonsense
Demonstration: Your container gets fire and fluf! all your tripple (or more) redundant data is lost in a burst.

Surely you are one of those that believe RAID is a valid backup strategy.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

Can you say, Parallelism?

Parallelogram.... crap
Parallellellell... dammit
Parapalouza... >

Why did you have to point that out to everyone? :(

Re:Where do they store 4.5TB off site

[bitrex]

What you've got to do is you've got to make punch-card copies of all your data. Then you're gonna take those punch cards, and you're going to put them on a wooden table. Then you're going to take digital photographs of them, email them to your backup site in the Netherlands, where they'll get the photographs printed and stack them in a vacuum sealed airtight length of 24" PVC sewer pipe stored on the top floor of a windmill to avoid flood damage. That is what we call mission critical backup procedure, my friends.

Re:Where do they store 4.5TB off site

[notthepainter]

Sadly, even modern day archives get wrecked. See http://www.spiegel.de/international/germany/0,1518,611311,00.html

Re:Where do they store 4.5TB off site

[Samah]

It's simple, the backups are compressed -- they simply remove all those useless zeroes from the binary data.

Compressed with XML! Because XML makes everything better... right?
Right?

Re:Where do they store 4.5TB off site

[zach297]

I'd suggest also using stone slabs. Water can do serious damage to paper, and don't get me started on fire hazards. Good old Stone Slabs resist both of those really well. I'm not sure what the write speed is, however, so you'll probably need to hire many stonecutters to work in parallel.

A math problem. My favorite. I don't know much about stone cutters but lets assume they can write one bit every 2 seconds. Thats 1 byte in 16 seconds. The internet archive is (4.5 x 1,125,899,906,842,624) 5,066,549,580,791,808 (5 quadrillion) bytes. That works out to 81,064,793,292,668,928 (81 quadrillion) seconds or about 2,570,547,732 (2.5 billion) years. That is far to long for their stringent 2 month backup cycle. They would need 15,423,286,395 (15.4 billion) stone cutters to keep schedule assuming they had unlimited stone. Last time I checked there were only between 6 and 7 billion people with only a small fraction of them being stone cutters. That leaves but one solution. Force the web developers to become stone cutters. This would not only increase the work force but also reduce the amount needed to backup because fewer people will be making more web pages to backup.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

Are you sure it wasn't middle earth? Frodo is pretty good at taking care of things.

Re:Where do they store 4.5TB off site

[fortapocalypse]

But then when the stone slabs deteriorate you have to build an Ark to carry them.

Re:Where do they store 4.5TB off site

[Omniscient Lurker]

Instead of writing in binary you could write the data in a base-36 format and then convert back to binary. The stone cutters could then store more data per glyph increasing their write rate considerably (and decreasing read rate) by amounts I am unwilling to calculate.

Re:Where do they store 4.5TB off site

[MaggieL]

Silly. There's no requirement that it be willful. Look at the education system.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

Can't they just use the Wayback Machine to recover the data?

I mean, I assume the Internet Archive at least archives itsself.

Re:Where do they store 4.5TB off site

[Tubal-Cain]

Did you know that if you stick the same disk back in, you can fit the entire backup on a single floppy?

Re:Where do they store 4.5TB off site

[Tubal-Cain]

any hard-coded data on silicon is just data on a stone slab.

Ceramic and glass are both stone, right?

Re:Where do they store 4.5TB off site

[k3vlar]

It's simple, the backups are compressed -- they simply remove all those useless zeroes from the binary data.

We should back up the digits 1 and 0 too, just to be safe. In the event of a fire, we want to be able to reconstruct the data, and what will we do if all the 1's get damaged? We'll also have a handy comparison for the 0, so we can insert it back without too much effort when we un-compress the data.

Re:Where do they store 4.5TB off site

[Rural]

Their aim is to preserve the content found on the Web. They need the hardware for that. I assume they don't need much for the "serving users" part.

Re:Where do they store 4.5TB off site

[garry_g]

It's simple, the backups are compressed -- they simply remove all those useless zeroes from the binary data.

... and once the useless zeroes are out, a RLE of the remaining data takes care of the rest ... then, once a day, some monkey comes in and does a backup on an old 1541 ... the disk is then xeroxed and the paper copy safely stored in between the stocks and derivatives of AIG, ensuring that no-one will ever find it in between that worthless pile of paper ...

Re:Where do they store 4.5TB off site

[Anonymous Coward]

They could use the c64 web-server c64web @ http://www.c64web.com a 1541 drive holds a lot of byte's.

Kind of David meets Golif.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

If you complain about the school system, you're living in a district that doesn't have
the resources to provide a quality education.
I have worked with kids that attended the best schools in Portland ( Lincoln and Lake Oswego High )
and have found that the students attending have a broad and varied education.
I was told by one student that a weak education was due to the failings
of the parents to achieve wealth and prestige and move to districts that provided a quality education.
This does not bode well for everyone that's excluded from these intense, focused and expensive
"publicly" founded institutions.

The children of the new poor will rise up and butcher the bastards that stole their
dream of a place of their own and food for their kids. We will have a Mao style cleansing.
And the it will be over.

Re:Where do they store 4.5TB off site

[PReDiToR]

Actually, all those "zeros" aren't totally zero. There are micro traces of energy used to make them up that get detected out of the equation and rounded down to zero.

I have written a program to transfer all those tiny bits of electricity into a battery I own in another country, and sooner or later with all those round downs, I'll have enough energy to sell to the ex-USSR countries so they won't need gas any more.

Profit!

Re:Where do they store 4.5TB off site

[aetherworld]

You could also GZIP it first!

Re:Where do they store 4.5TB off site

[Anonymous Coward]

Nice TDWTF reference.

Re:Where do they store 4.5TB off site

[knutkracker]

TFA indicates that they have a mirror at the library of Alexandria.

You mean the site of possibly the most catastrophic series of data-loss incidents in recorded history?

Who's idea was that?

Re:Where do they store 4.5TB off site

[fuzzyfuzzyfungus]

Don't worry, the Romans are gone, and they purchased some lions to keep Christians away from the backups...

Re:Where do they store 4.5TB off site

[ATMD]

> Anyone know what happened with the Netherlands site?

Umm... it flooded?

Re:Where do they store 4.5TB off site

[morie]

This is also the reason real loud amps go up to 11...

Re:Where do they store 4.5TB off site

[the cleaner]

could someone with experiance in such matters shed a little insight into the logistics of backing up such a vast system

The secrets of backups in a datacenter that big is: You don't. Backup, that is.

A simple calulation shows, that a full backup would basically never end, and incremental backups only help you if you if you had something to increment from. And a restore would never end as well.

Instead you make damn sure, you have redundancy in everything. DRBD is your friend. RAID controllers are, too.

If you have the money, integrate with the big ones (EMC, HP, SUN). If you don't, use a clever solution (Caringo).

Re:Where do they store 4.5TB off site

[commodore64_love]

>>>I can't take anyone seriously who puts "truth" and a link to Fox news in the same signature.

Neither can I take seriously anyone who believes MSNBC or CNN are unbiased and/or better alternatives. Or is prejudiced (prejudges a report without ever watching it). For example I may think Rachel Maddow is a joke, but at least I listen to what she has to say before I laugh. And sometimes, she says something worthy of hearing... it's good to keep an open mind and listen to the opposition.

Re:Where do they store 4.5TB off site

[commodore64_love]

>>>Cracks had likewise been discovered in the archive recently, but had been discounted.
>>>The collapse may be connected with the construction of a new subway line under the street

Brilliant. You know there's a subway being built underground, and you notice there's cracks appearing, but you pretend the two are unconnected and do nothing. Way to live in denial museum curators.

Re:Where do they store 4.5TB off site

[Anonymous Coward]

That's the whole point! The Archive has a partnership with the new Library of Alexandria to duplicate files; the Archive preserves its content in multiple locations for this very reason.

Re:Where do they store 4.5TB off site

[CopaceticOpus]

"XML is like violence. If it doesn't solve your problem, you're not using enough of it."

Re:Where do they store 4.5TB off site

[jd]

Ceramic certainly is, glass is one of those weird cases where the state of matter is not definitively in either the category of solid or liquid but if you were to use conventional classifications, glass could be considered an igneous rock.

Re:Where do they store 4.5TB off site

[AMuse]

One or two forklifts and one 18-wheeler. The data is already pre-packed in a shipping container.

Re:Where do they store 4.5TB off site

[bandmassa]

The stone cutters just have to work faster or they'll get fired.

Re:Where do they store 4.5TB off site

[DiLLeMaN]

In fact, if you have ever tried to retrieve an old version of a website, you would've found out that "serving users" is done using a single pentium 1 machine in the basement.

Re:Where do they store 4.5TB off site

[DiLLeMaN]

they're in /dev/null, of course.

Re:Where do they store 4.5TB off site

[Samah]

I've heard that quote somewhere before, and I was actually thinking of it when I posted. :)

Re:Where do they store 4.5TB off site

[Anonymous Coward]

Who's idea

"Whose".

Story is meaningless without LOC measurement

[Dr_Banzai]

I have no idea how much 4.5 PB is until it's given in units of Libraries of Congress.

Re:Story is meaningless without LOC measurement

[Anonymous Coward]

1 library of congress = 10 terabytes
4.5 petabytes = 4608 terabytes
So, that's 460.8 LOCs.

Re:Story is meaningless without LOC measurement

[Wingman 5]

from http://www.lesk.com/mlesk/ksg97/ksg.html The 20-terabyte size of the Library of Congress is widely quoted and as far as I know is derived by assuming that LC has 20 million books and each requires 1 MB. Of course, LC has much other stuff besides printed text, and this other stuff would take much more space.

1. Thirteen million photographs, even if compressed to a 1 MB JPG each, would be 13 terabytes.
2. The 4 million maps in the Geography Division might scan to 200 TB.
3. LC has over five hundred thousand movies; at 1 GB each they would be 500 terabytes (most are not full-length color features).
4. Bulkiest might be the 3.5 million sound recordings, which at one audio CD each, would be almost 2,000 TB.

This makes the total size of the Library perhaps about 3 petabytes (3,000 terabytes).

so 230 libraries by the old standard or 1.5 by the new standard

Re:Story is meaningless without LOC measurement

[commodore64_love]

83 terabyte in the LOC, so 4.5 petabytes == 54 Libraries of Congress

4.5 petabytes == 4500 terabyte hard drives, times $75 each == ~$340,000 == how much taxpayers spend, each hour, to maintain the LOC

Re:Story is meaningless without LOC measurement

[commodore64_love]

P.S. My "83 terabyte" quote comes directly from the Library of Congress statistics, mid-2008.

Re:Story is meaningless without LOC measurement

[clarkkent09]

Bah, LOC is outdated. 4.5PB = 1 Shipping Container

Re:Story is meaningless without LOC measurement

[v1]

a new 4.5 petabyte data center

4.5 PB? Is that the best you can do? sheesh, amateurs....

Though it also did surprise me they only get 200,000 hits/day. I expected the WayBack Machine to get a lot more traffic than that.

Re:Story is meaningless without LOC measurement

[clarkkent09]

I think that's 200K unique visitors. According to alexa, archive.org is the 386th most visited site on the internet last week which is not to be sneezed at

Re:Story is meaningless without LOC measurement

[dln385]

from http://www.lesk.com/mlesk/ksg97/ksg.html The 20-terabyte size of the Library of Congress is widely quoted and as far as I know is derived by assuming that LC has 20 million books and each requires 1 MB. Of course, LC has much other stuff besides printed text, and this other stuff would take much more space.

1. Thirteen million photographs, even if compressed to a 1 MB JPG each, would be 13 terabytes. 2. The 4 million maps in the Geography Division might scan to 200 TB. 3. LC has over five hundred thousand movies; at 1 GB each they would be 500 terabytes (most are not full-length color features). 4. Bulkiest might be the 3.5 million sound recordings, which at one audio CD each, would be almost 2,000 TB.

This makes the total size of the Library perhaps about 3 petabytes (3,000 terabytes).

so 230 libraries by the old standard or 1.5 by the new standard

Compress each audio file to a 5 MB MP3. That's 17.5 TB. Total size would be 750 terabytes.

So the data would be 6 LOC.

Re:Story is meaningless without LOC measurement

[merreborn]

Bulkiest might be the 3.5 million sound recordings, which at one audio CD each, would be almost 2,000 TB.

You compressed the video, and the photographs, but not the audio? And why do you need a full CD for every sound recording? Surely many of them are far shorter than a full CD?

Re:Story is meaningless without LOC measurement

[Xtravar]

The CDs are already in digital format, so compressing them is a cardinal sin.

The photos, movies, and maps are in analog format to start with, so we don't feel so bad using lossy compression. Image files are really big. I think the 1GB estimate per movie is pretty good, considering shorts, black and white, and the standard (or lower) definition of most of them. That would allow for a very high detail scan of the movie in something like MPEG4.

And, since they started in analog formats, there's no fair way to determine what resolution to scan them. I mean, even a million by a million pixels could not be a 'lossless' interpretation of a 1x1cm image, so you have to accept that any digital conversion will be lossy regardless of encoding.

At least that would be my rationale. Not that this question needed to be answered...

Re:Story is meaningless without LOC measurement

[turbidostato]

"According to alexa, archive.org is the 386th most visited site on the internet"

386? I would think that with all those Sun boxes they already were 64 bits at least!

Re:Story is meaningless without LOC measurement

[martin-boundary]

1 Library Of Congress = 1/200 inch wide cube approx.

Feynman estimated the LOC at about 1 petabit, which would make the Internet Archive containing roughly 36 petabits a cube on the order of 1/50 inch wide.

So it should fit in your pocket.

Re:Story is meaningless without LOC measurement

[Thaelon]

This should probably be added to units.

Re:Story is meaningless without LOC measurement

[Anonymous Coward]

460.8 Lines of Code? What's that supposed to be? Hello World in COBOL?

Re:Story is meaningless without LOC measurement

[Anonymous Coward]

How many shipping containers in a football field?

Re:Story is meaningless without LOC measurement

[robinesque]

Google needs to add this to their unit conversion table so that I can convert "number of horns on a unicorn petabytes in libraries of congress".

Re:Story is meaningless without LOC measurement

[Anonymous Coward]

And I guess the new meme will be "how many Internet Archive data centers is that?"

Re:Story is meaningless without LOC measurement

[emj]

In my experience black and white movies compress at about the same bit rate as color movies. Especially if they are bad quality.

Re:Story is meaningless without LOC measurement

[Anonymous Coward]

You really think it costs $75 an hour per drive per hour? Get real.

Also putting FoxNews in your sig makes you a wingnut.

Re:Story is meaningless without LOC measurement

[Lunzo]

You Yanks and your old-fashioned measurements. The rest of the world has moved to the metric system's "Libraries of Alexandria" unit.

Storage Envy

[jacksinn]

Does lusting after all their space make me a peta-phile?

Re:Storage Envy

[fm6]

Yes.

Re:Storage Envy

[Rude Turnip]

Why don't you have a seat over there...

Re:Storage Envy

[Anonymous Coward]

I do believe you have won.

Re:Storage Envy

[Anonymous Coward]

Not to be confused with a PETA-phile. Those animal lover lovers.

Re:Storage Envy

[Zapotek]

*Knock* *knock* Open up, we know you're inside!

Re:Storage Envy

[Anonymous Coward]

ohai, were from the partyvan club! you won a free ride to jailbait island! come with us nao!

Own the internet!

[Anonymous Coward]

so all one need to do to "own the internet" is to drive a big rig and ... lift the container off their parking lot?

Re:Own the internet!

[peragrin]

well if you plug in a laser printer you can print off a hard copy for your boss.

Re:Own the internet!

[Anonymous Coward]

Jack Bauer did it last week.

Re:Own the internet!

[PReDiToR]

I hope you're using company paper and ink.

Never pay for at home that which you can use for free at work.

Re:Own the internet!

[rrohbeck]

No joke. Where I work we had a big UPS with diesels for the data center outside on a trailer because the building didn't have enough space.

It disappeared one weekend.

Re:Own the internet!

[peragrin]

Honestly when i had two ink jet printers have the ink go bad and solidify in between the infrequent uses of printers at home i stopped keeping on.

If it is important I print t at work, or if it is too large, I send it to a friend who can print it on a large commercial laser jet.

Slight problem?

[girlintraining]

I can now theoretically steal "the internet" with a flatbed truck and a lift. There's something to be said for conventional data centers: They're rather hard to load onto a truck and drive off with.

Re:Slight problem?

[rackserverdeals]

Here's a video tour of one if you need it for reference.

Don't forget to turn off the water and unplug the ethernet cables. Just be very careful with the power cords.

Re:Slight problem?

[fightinfilipino]

so the Internet really is a big truck, hauling all of our lulz and our memes across the globe.

take THAT, Ted Stevens!

Re:Slight problem?

[diablovision]

Nothing is stopping you from putting it inside a building, cementing it into its foundation, or surrounding it with appropriately weaponized sharks.

Re:Slight problem?

[bigsteve@dstc]

But don't forget to wrap the container in cling-film before you drop it into the shark tank.

Re:Slight problem?

[caffeinemessiah]

There's something to be said for conventional data centers: They're rather hard to load onto a truck and drive off with.

Yes, but imagine the bandwidth!

Re:Slight problem?

[caerwyn]

Never underestimate the bandwidth of a... stationwagon filled with 4.5 petabyte shipping containers?

Re:Slight problem?

[taucross]

The idea of winning "one internets" is suddenly not quite so appealing.

Re:Slight problem?

[couchslug]

"or surrounding it with appropriately weaponized sharks."

or defending it with other ISO containerized systems:

Super Sangar

http://www.mod.uk/DefenceInternet/Templates/NewsArticle.aspx?NRMODE=Published&NRNODEGUID={8F432B04-D3C9-419E-8365-5204663E648C}&NRORIGINALURL=%2FDefenceInternet%2FDefenceNews%2FEquipmentAndLogistics%2FSecuritySurveillanceAndsuperSangars.htm&NRCACHEHINT=Guest

Re:Slight problem?

[rackserverdeals]

Speed limit is 65mph, 3024GB going 65mph = 1965560GB/h or 3276GB/s That's pretty fast.

Re:Slight problem?

[JambisJubilee]

I can now theoretically steal "the internet" with a flatbed truck and a lift.

Carmen Sandiego?

Re:Slight problem?

[Voyager529]

This gives the term "Rick Rolling" a whole new meaning.

Housed in a metal shipping container....

[MichaelSmith]

Well I hope it is bolted down.

Re:Housed in a metal shipping container....

[stonedcat]

And lets hope those bolts go into something solid, like not dirt.

Re:Housed in a metal shipping container....

[keithjr]

I'm surprised the summary doesn't mention it, but the shipping container is a Sun "Project Blackbox", or rather "Modular Data Center" as it's officially called.

Re:Housed in a metal shipping container....

[Anonymous Coward]

A story about 63 Sun servers replacing 800 linux servers won't fly here.

Only one new datacenter?

[Anonymous Coward]

Just imagine what you could do with a beowulf cluster of 4.5 PB datacenters. You could create regular archives of the internet archives!

(As a webserver administrator, I can't stress how important it is to keep backups.)

Nice use for a bunny-rabbit

[davecb]

Yes, "thumper" refers to the rabbit. I have a Sun Managed Storage slide somewhere about how data tends to, er, multiply...

--dave

What about 1996 and earlier?

[commodore64_love]

Are there any resources the let us see websites from 1996, 95, 94, or 93? I would love to revisit the web as it appeared when I first discovered it (1994 at psu.edu).

Re:What about 1996 and earlier?

[Tumbleweed]

I would love to revisit the web as it appeared when I first discovered it (1994 at psu.edu).

No, you wouldn't.

Re:What about 1996 and earlier?

[Matheus]

The entire internet prior to 1996 is archived on an old PC that I'm currently trying to get the 5GB disk restored on.. why I've kept all that old porn for so long completely escapes me tho. :)

Re:What about 1996 and earlier?

[Profane MuthaFucka]

Because after 1996 women shaved all their hair off due to a mistaken belief that men prefer their women to look like little girls. We don't, we like the big bushes, and that is why you must save that porn for the good of mankind.

Re:What about 1996 and earlier?

[scottrocket]

Yes, "The Wayback Machine", at archive.org. Coincidentally, I was there just last night, looking at a January '98 Slashdot.

Re:What about 1996 and earlier?

[scottrocket]

Oops. '93-'96 - apparently in my universe, '98 is before '93.

Re:What about 1996 and earlier?

[Hurricane78]

Speak for yourself. I like my pussy shaven. But as someone who never licks his girlfriend's pussy, you would not know why that is, would you?

Re:What about 1996 and earlier?

[Profane MuthaFucka]

Are you trying to insult me?

Re:What about 1996 and earlier?

[TuaAmin13]

Failed Connection. We're sorry. Your request failed to connect to our servers. This may be due to temporary problems in our data center, or difficulty serving a higher-than-usual volume of traffic.

I think Slashdot just slashdotted the wayback archive of Slashdot. I just tried looking it up and got the above error.

So it IS possible to slashdot slashdot.

Re:What about 1996 and earlier?

[scottrocket]

"So it IS possible to slashdot slashdot."

Yes but only in the past, in which case measures would be taken to block future people-but then we wouldn't do it, in which case no measures would be taken to block future people and then we would do it, and measures would be taken to block future people, and then...and then

^error

Re:What about 1996 and earlier?

[cffrost]

Next, we just need to thieve Hollywood's instantaneous "Enhance That!" app that converts 4-bpp GIF thumbnails into 6144x4096 floating-point TIFFs.

Re:What about 1996 and earlier?

[Anonymous Coward]

If I could attach an animated GIF of a construction worker and some yellow and black tape looking images and put UNDER CONSTRUCTION inside of a H1> tag, that would account for >90% of the webpages back then.

Re:What about 1996 and earlier?

[Matheus]

OR maybe the data center just drove away..

Re:What about 1996 and earlier?

[skeeto]

Surfing Freenet today is like surfing the web around '94 or '95, but with no blink tags.

Re:What about 1996 and earlier?

[Hurricane78]

Nah. If i would try to insult you, I would try to come up with something that is not true.

KAZBAAAM! XD

Re:What about 1996 and earlier?

[Profane MuthaFucka]

If you want to learn how to insult someone, you should take lessons from the woman who caught me masturbating outside her window last night. You're a fucking newb compared to her. Don't even bother to try.

That is a shit ton of space

[webheaded]

Unfortunately the Wayback Machine will still be slower than hell. :p

Re:That is a shit ton of space

[jd]

Fortunately, Hell has now been upgraded to 2 mb/s, thanks to British Telecom.

Re:That is a shit ton of space

[kelnos]

Two millibits per second? And that's an upgrade? Ouch.

Re:That is a shit ton of space

[jd]

One method of suspend-to-disk is to do a freeze/thaw. It has taken Hell over 16 billion years to do just the freeze. Two millibits per second should be able to do both in less than half the time.

They had it on South Park

[chadimus]

It sometimes takes the form of a giant blue linksys router. So that we may better worship it.

In reality...

[tacarat]

The internet is only about 2TB once you've removed all the redundant copies of 2g1c and goatse.cx.

Re:In reality...

[EdZ]

Then you lose any data that may be stored in the arrangement of those many redundant, redistributed and reencoded copies. Distributed steganography, if you will

They store 4.5PB in Egypt!

[CannonballHead]

The Internet Archive also works with about 100 physical libraries around the world whose curators help guide deep Internet crawls. The Internet Archive's massive database is mirrored to the Bibliotheca Alexandrina, the new Library of Alexandria in Egypt, for disaster recovery purposes.

Re:They store 4.5PB in Egypt!

[Anonymous Coward]

Egypt could be a good choice. The area is fairly famous for reliable persistent storage. From papyrus scrolls to stone engravings, things tend to keep there better than most places. There really aren't many other geographical areas on earth that can claim the same kind of data retention rates over the time periods they've dealt with. Though despite their impeccable track record with avoiding hardware failures, they've done significantly worse when it comes to data loss due to theft and/or hackers/pirates.

The one curious part about that choice is that the library at Alexandria is the one notable case where mass amounts of data were irreparably lost. So it's odd that they'd choose to entrust their data to that specific institution. Perhaps they felt that since it's under new management, the previous problems will have been resolved.

However, had the choice been mine, I would have chosen to store my offsite data in Luxor. It's data retention was quite good, and included one data store that was preserved in its entirety for over 3000 years. As an added benefit, it seems that they've opened a second location that's significantly more convenient for the IA since there's no overseas transmission to worry about.

Re:They store 4.5PB in Egypt!

[Darkk]

I oughta start a new company called:

Off-Planet backups where I'd use the moon to store your precious data!

Only three things I'd have to worry about would be:

1) Aliens (if they are out there)
2) Meteors
3) Solar Flairs

Other than that pretty solid plan to me!

Re:They store 4.5PB in Egypt!

[Hadlock]

3) Solar Flairs

Don't forget:
 
4) Spelling Nazis

Re:They store 4.5PB in Egypt!

[Tubal-Cain]

Just broadcast the (encrypted) data out into the void. When you need to retrieve it, simply invent a FTL drive and jump as far as the desired data will arrive shortly after.

Re:They store 4.5PB in Egypt!

[Anonymous Coward]

I don't want to talk about my solar flair.

Re:They store 4.5PB in Egypt!

[timbck2]

How many pieces of solar flair do you have?

In Other News

[Erik Fish]

Incidentally: FileFront is closing in five days, taking with it any files that aren't hosted elsewhere.

I am told that many of the Half-Life mods hosted there are not available anywhere else, so get while the getting is good...

Re:In Other News

[bluesatin]

It's a sad sad day when FileFront shuts it's doors.

Please don't make us start downloading things from FilePlanet again, it makes me cry a little inside.

)':

Re:In Other News

[skeeto]

I was going to say "good riddance", but I was mixing them up with places like FilePlanet. I pictured that annoying situation where you want to download a 200kB file from them, but first you have to sign up for an account. So, you fill out a big form, check your e-mail, log in, click your way through several pages to the actual download, then get in a 40 minute queue. When you make it out of the queue you have 1 minute to start your download or else you have to get back in line and wait again. Awful.

FileFront looks like it didn't have that bullshit. Sad news then.

Never underestimate the bandwidth ...

[Ungrounded Lightning]

... of a 4.5 petabyte datacenter in a shipping container in transit.

Re:Never underestimate the bandwidth ...

[Wingman 5]

(4.5 petabyte) / (1 year) = 153.114984 MBps
  now that would be some bad lag.

Re:Never underestimate the bandwidth ...

[Anonymous Coward]

I'd reply in a year and get modded +5 Funny, except /. closes discussions after a short time.

Re:Never underestimate the bandwidth ...

[giblfiz]

Government economic stimulus: Treating a patient for anemia with an iron supplement made from his own extracted blood.

I can't resist replying to your Sig...
It's like treating a patient for anemia with iron supplements made from his own extracted blood from the future. We are taking on debt, not trying to push through a one year ballenced budget. I'm not sure it's a good idea, but it's a much better one than what your describing.

Re:Never underestimate the bandwidth ...

[Ungrounded Lightning]

It's like treating a patient for anemia with iron supplements made from his own extracted blood from the future.

Unfortunately, when you finance with debt on an economy-wide basis you pay double - or more. There's the return payment. (Plus the interest - which is the "more".) But there's also the cost to the economy of whatever WOULD have been done with the "borrowed" resources but now is not done because the resources were diverted.

When they talk of how many jobs were created by the stimulus, ask how many jobs were destroyed by it: Destroyed because money was stolen by taxes or the value was stolen from the existing money by inflation. Normally the answer will be "more" - "a lot more" - because the funded programs are less productive than what was defunded and government-managed funds transfer is far less than 100% efficient.

Look up the "fallacy of the broken window".

30 comments...

[Anonymous Coward]

and not a single "finally a place big enough to store all of my porn" reference? Y'all are slacking tonight.

on a very slightly serious note, how much content would be referenced by, say, TPB? Sure the trackers are small, but that's got to be huge.

library of Alexandria, Egypt

[Anonymous Coward]

Didnt that burn down a few thousand years ago?

63 x 48 = 3024Tb

[eotwawki]

So wehre does the 4.5PB come in to this?

Re:63 x 48 = 3024Tb

[glitch23]

The article doesn't make it clear so I can only guess that the missing storage capacity is part of some SAN. Maybe the 48 1TB hard drives are only local storage (obviously) but are in addition to some existing SAN that they didn't mention in this particular article. Either that or the article is just wrong about the 4.5PB database.

Re:63 x 48 = 3024Tb

[SirLoadALot]

Good point. My best guess would be that they are actually 1.5 TB drives. That would get the numbers about right.

Re:63 x 48 = 3024Tb

[NickW1234]

That was my assumption as well. Of course, that's not accounting for any redundancy.

Re:63 x 48 = 3024Tb

[spinkham]

TFA says "...eight racks filled with 63 Sun Fire x4500 servers with dual- or quad-core x86 processors running Solaris 10 with ZFS. Each Sun server is combined with an array of 48 1TB hard drives." (emphasis mine)

I would guess this means there's a x4500 with 24TB in local disks, and 48TB in attached storage per machine. (24+48)*63 does give us the quoted number

Re:63 x 48 = 3024Tb

[rackserverdeals]

I don't think that's right. Sun's site has a video tour of it. Haven't finished it yet but it's here.

Re:63 x 48 = 3024Tb

[rackserverdeals]

The new datacenter is only 3PB. I guess the total storage, with the old data centers is 4.5 PB.

So 48x63 gives you 3PB of raw storage. I'm guessing there using less because I can't imagine them running it in raid 0.

Re:63 x 48 = 3024Tb

[pwnies]

Actually it's a bit less than that even. The Sunfire servers they're using, or "thumpers" as they're nicknamed generally use zfs to store their data. However, the default configuration of these systems is to use a Raidz config for the drives (think raid 5). Essentially, the configuration uses 8 6-disk raidz configs, all aggregated together into one giant pool. The reason why it's less than what they state here, is that one disk from each of those eight raidzs are parity disks. That drops the theoretical storage to only 40TB per thumper. That puts us at 2520TB.

Again though, even that's a high number, because once formatted, each thumper only delivers about 36TB of storage. So in actuality, they only have about 2268TB of storage (half of what they claim).

My only guess is that the 4.5PB number comes from Sun's advertising dept, who are running the numbers on the theoretical max that the container could hold. If you use the highest capacity drives readily available on the market right now (1.5TB drives, as the 2GB drives are a bit hard to get ahold of), no parity, and no loss in formatting, the numbers are correct. 63 x 48 x 1.5 = ~4500

Re:63 x 48 = 3024Tb

[rackserverdeals]

Sun has more information and an Interactive tour of the Internet Archive modular data center on their site.

The total raw capacity of the container is 3 peta bytes. In reality it's going to be less than that. First, 2 disks are likely to be setup in a mirrored pool for the system disks. I believe the root pool only supports mirrors, not raidz. Not sure if this has changed.

That leaves you with 46 disks for data. Maybe they partitioned part of the root pool to include in the data pools, not sure, but zfs works better with whole disks.

In the interactive tour, they weren't clear on how they set up the pools.

Side note. Maybe I'm cynical, but if this was the other way around, with linux servers replacing sun/solaris servers that probably would have been the headline.

Pretty neat to find out that the internet archive is powered by Java too. The wayback machine is java as well as the crawlers.

Re:63 x 48 = 3024Tb

[Anonymous Coward]

DO you understand how RAiD work? 4.5 petaphiles total, 3 peteaphiles usable.

Re:63 x 48 = 3024Tb

[Anonymous Coward]

If the headline was "800 x86 Linux servers replaced by 63 Sun Servers running Solaris" people would have thought April first came early this year.

Re:63 x 48 = 3024Tb

[Anonymous Coward]

They might actually be using 1.5TB drives.

Re:63 x 48 = 3024Tb

[isorox]

The total raw capacity of the container is 3 peta bytes. In reality it's going to be less than that. First, 2 disks are likely to be setup in a mirrored pool for the system disks. I believe the root pool only supports mirrors, not raidz. Not sure if this has changed.

Our Thumpers are like that, but the new ones (4550) have a CF card for booting the OS. ZFS booting is supported since u5 or u6, but only booting from a mirrored pool.

I was all excited by the storage denstity until recently. We had an issue with one of the controllers. A later patch fixed it. We installed that patch, which hung the machine (our backup machine). After powercycling (the alom didn't want to depower the box), we had a corrupt ufs boot partition that fsck couldn't fix. In the end I had to install u6 onto the boot drives.

Combine the ridiclous not-quite-x86 not-quite-sparc remote access (the VGA port doesn't work with our raritan systems, the alom network grabbing the console isn't reliable, depending on whether vga is plugged in), and the difficulty for our org of maintaining solaris (compared with linux), as well as the cost, means we're not recommending them for our field offices.

We have our machines set up in a z2, 44TB storage, with 2TB of parity. That's 40TiB, or 10TiB per rack unit.

Re:63 x 48 = 3024Tb

[rackserverdeals]

You had all the drives set up as one raidz2 array? From what I understand, that's not the best way of doing it. I can't find the link but there was an entry on blogs.sun.com about how it's better to use a number of raidz's in a pool than have the pool consist of a single raidz array. Here's an example of that zfs configuration. And here's one that discusses performance and MTTDL.

I was under the impression that the boards use the same type of daughter card that the TYAN server boards use. Raritan makes an IP KVM card for those boards. Have you tried contacint Raritan to see if you could use that instead?

Re:63 x 48 = 3024Tb

[isorox]

No,it's not the best way of doing it for a given value of best (performance). Our value was more storage density and disk failure. We offset the chance of getting random corruption by weekly scrubs and nightly rsyncs.

Whoopsie!

[Profane MuthaFucka]

That wasn't the ribbon, it was the powercord! Someone's going to be embarassed!

3PB or 4.5PB

[Anonymous Coward]

I guess /.'s readers can no longer multiply, but 63 servers * 48TB/server = 3024TB =~ 3PB.

I'm guessing they had 1.5PB already?

Andy

  P.S. yes, I'm looking for a class 8 truck and a set of hydraulic jacks... but before I steal the Internet Archive, as a consumer, I DEMAND that the entire thing fit in my shirt pocket, and have an Apple logo on it!!!

63 x 48 =3024

[eotwawki]

So where does the 4.5PB come into this?

You can ship it over OC-192...

[Ungrounded Lightning]

... one would assume that something like this does regular off-site back-ups, which must add up to a hell of a-lot,..

As I recall from one of Brewster's talks: Part of the idea was that you can install redundant copies of this data center around the world and keep 'em synced.

You can ship 4.5 petabytes over a single OC-192 link in about 71 days.

Re:You can ship it over OC-192...

[TheGratefulNet]

You can ship 4.5 petabytes over a single OC-192 link in about 71 days.

yeah, but just at the 70th day, someone will pick up the phone and the whole thing will have to be resent.

Re:You can ship it over OC-192...

[Anonymous Coward]

You can ship 4.5 petabytes over a single OC-192 link in about 71 days.

Assuming you have OC-192s to every location over trans-oceanic distances dedicated only to this.

The initial sync would probably be done locally with a trunked 10 GigE, and then ship the duplicate container to the back up location. Then send deltas over your trans-oceanic links.

Syncing snapshots (and only sending the deltas) is very easy with ZFS (which is presumably what they're using if they're on Solaris 10).

Re:You can ship it over OC-192...

[Anonymous Coward]

whoosh! four mods didn't get the joke??

Re:You can ship it over OC-192...

[aaarrrgggh]

Or, you can ship the 40' containers in just under two weeks!

Re:You can ship it over OC-192...

[Plaid Phantom]

But I don't think they would fit in a station wagon...

Re:You can ship it over OC-192...

[Anonymous Coward]

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
-- Andrew S. Tanenbaum

s/station wagon/shipping container/
s/tapes/Sun Fire servers/
s/highway/ocean currents/

Re:You can ship it over OC-192...

[evilviper]

Or, you can ship the 40' containers in just under two weeks!

Yeah, but RSync over tractor-trailer is still in ALPHA.

Re:You can ship it over OC-192...

[jamiethehutt]

Never underestimate the bandwidth of a cargo ship full of datacenters?

Math

[PowerKe]

63 servers * 48 disk of 1 TB = 3024 TB. According to the announcement on the archive.org 3 Petabytes would be right.

Re:Math

[Anonymous Coward]

And so, 1 library of congress.

Mental images of libraries

[macraig]

Riiiight... because you happen to have a really really good mental image of exactly how many rooms/shelves/books/pages are stored in the Library of Congress!

(Which incidentally doesn't happen to be static, BTW; yo momma's LoC ain't the same size as my LoC.)

Slashdotted

[thefolkmetal]

I don't know if I'm the only one who read it this way, but the summary makes it seem like these servers have a bit of a job on their hands as it is, what with hosting the site and doing their web-crawling/archiving...and we slashdotted this thing? We're going to blow that little metal building up.

"Sun Fire"

[fm6]

The new data center houses 63 Sun Fire servers

That's not very specific. "Sun Fire" is a brand that for a while got applied to all of Sun's rack-mount servers (except for NEBS-compliant servers, which were and are called "Sun Netra"). A little confusing, of course, which is why they've started calling new SPARC boxes "Sun SPARC Enterprise" to differentiate them from those mangy x64 "Sun Fire" systems. Except that there are still SPARC systems called "Sun Fire", so I guess the confusion factor didn't get any better...

Anyway, the specific server being used here is the Sun Firex X4500, a system with no less than 48 1 TB disks in a 4U space. Notice that this model is EOLed; presumably iarchive got a deal on some remaindered machines.

The shipping container is something we've seen before.

Re:"Sun Fire"

[ximenes]

Since they're using one of Sun's modular datacenters that is actually on the Sun campus, I would imagine that they got some financial incentives / support from Sun for all of this.

The X4500 is EOL as you mention, although it was still sold a few months back. It lives on as the X4540, which really isn't that different; the main thing is it's moved to a newer Opteron processor type and is a fair bit cheaper. So they didn't really miss out on anything.

It's kind of interesting to me that they went this route, as opposed to a bunch of servers talking to a bunch of storage separately. This seems to be an exact use case for the X4500-type system, which as far as I'm aware is pretty unique.

Re:"Sun Fire"

[fm6]

This seems to be an exact use case for the X4500-type system, which as far as I'm aware is pretty unique.

Indeed. Sun is on a density kick. Check out the X4600, which does for processing power what the X4500 did for storage.

In both cases, there actually are competing products that are sort of the same. The most conspicuous difference is that the Sun versions cram the whole caboodle into 4 rack units per system, about half the space required by their competitors.

More absurdly-dense Sun products:

http://www.sun.com/servers/x64/x4240/
http://www.sun.com/servers/x64/x4140/

The point of these systems is that they take up less expensive rack space than equivalent competitors. They're also "greener": if you broke all that storage and computing power down into less dense systems, you'd need a lot more electricity to run them and keep them cool. That not only saves money, it gives the owner the ability to claim they're working on the carbon footprint.

Re:"Sun Fire"

[Anonymous Coward]

Anyway, the specific server being used here is the Sun Firex X4500 [sun.com], a system with no less than 48 1 TB disks in a 4U space. Notice that this model is EOLed; presumably iarchive got a deal on some remaindered machines.

There are newer X4540s which are mostly the same, but have newer CPUs, and can hold more memory (16 -> 64 GB).

Re:"Sun Fire"

[Anonymous Coward]

Anyway, the specific server being used here is the Sun Firex X4500, a system with no less than 48 1 TB disks in a 4U space.

Speaking of density, these guys put 896 TB of raw disk (624 TB usable after RAID, overhead, and spares) in one cabinet.

Re:"Sun Fire"

[mgblst]

So not only are the archive old content, but old hardware as well!

Re:"Sun Fire"

[MrPerfekt]

Because ZFS is a gross memory hog. :) These servers work out pretty good for cold storage, but putting lots of random I/O read stress on it and you'll be left wanting some 15k SCSI disks. Oh well, price vs. performance.

they cut the ribbon?

[unfunk]

They cut the ribbon? How are they supposed to access that much data unless they buy a new one?

Re:they cut the ribbon?

[jd]

Easy. Ribbon's only good for short-distance parallel links. If they've got backups in Egypt, they must be using serial cables.

Not "THE" but "A" internet...

[denzacar]

You would be stealing A backup copy of THE Internet. An incomplete one at that, but still quite extensive.

Now... If you were somehow able to steal that copy AND break the internet... your stolen internet may be considered THE internet.

Load distributions

[Just Some Guy]

From TFA (yeah, I know):

a Web site that gets about 200,000 visitors a day or about 500 hits per second on the 4.5 petabyte database.

So they get all 200,000 hits in a 7-minute window? I picture a sysadmin going insane for a few moments then napping in a hammock for the rest of the day.

Re:Load distributions

[diablovision]

hit != visitor

Re:Load distributions

[JesseMcDonald]

I imagine that each visitor generates more than one database hit. This /. article page is made up of over thirty files, for example, each of which would probably count as at least one hit. By my simplistic calculations each visitor would need to generate about 212 database hits to maintain an average of 500 hits/s, which, while high, is not entirely unreasonable.

Re:Where do they store 4.5PB off site

[Chosen Reject]

[subject correction]
PB, not TB... hehe.

Minor problem with weaponized sharks

[Anonymous Coward]

Nothing is stopping you from putting it inside a building, cementing it into its foundation, or surrounding it with appropriately weaponized sharks.

The presence of weaponized sharks implies the need for a moat. Somehow I doubt the city and county governments would appreciate its construction on the premise, as the presence of the said sharks would preclude passing it off as a swimming pool.

Fool

[Anonymous Coward]

The internets isn't like a truck! It's a series of tubes!

Re:Fool

[palegray.net]

Is that kinda like these tubes?

4.5 PB... yumm

[Anonymous Coward]

Just imagine what you could do with a beowulf cluster of 4.5 PB datacenters. You could create regular archives of the internet archives!

Actually, I was thinking the largest collection of pr0n the world has ever seen (to date.)

Ok, how do you backup this thing ?

[slashdotlurker]

Inquiring minds want to know.

Re:Ok, how do you backup this thing ?

[Zapotek]

Compressed and incrementally?
Or is it incrementally compressed?
I'm no expert anyways. :)

The off-site backup IS the Internet.

[billstewart]

They're keeping the offsite backup distributed around the Internet, using the World-Wide Web to store it in real time.

Part of it may even be on *your* machine! We've really got to stop Brewster from leaching all your storage and make him store his backup himself - this business of using the originals to back up the backup just isn't sustainable!

Re:The off-site backup IS the Internet.

[PopeGumby]

They're keeping the offsite backup distributed around the Internet, using the World-Wide Web to store it in real time.

I heard that sometimes the backup even gets updated BEFORE the proper archive? As much as two months before!

Whats up with that?

oblig.

[indi0144]

Yo dawg we hear you like to back up your internets so we put a backup container inside your backup container so you can back up your back up while you back up your back up.

RAID Array/File system?

[Anonymous Coward]

So there are 48 1TB discs running in paralell? What does that mean? RAID1? Well, we know it can't mean that, so RAID5? Seems like it couldn't mean that either. Maybe multiple raid5 arrays with 6 discs per array?

Maybe some sort of nifty new filesystem I don't know about?

So slashdot, How do you think the discs are arranged and what filesystem do you think they use?

Whats the biggest tape backup one can buy nowadays?

4500 TB...

[Anonymous Coward]

Letsee...4.5 PB = 4500 TB

A Linksys NAS200 can support two 1 TB hard drives, which is 1 TB of storage configured as RAID 1.

Thus, 4500 NAS200 boxes could hold that much data. Dang. My house only has a 200 A main; I don't even have enough electrical service to run all of them. And my wife would have a fit when our electric bill showed up.

Donate

[Anonymous Coward]

http://www.archive.org/donate/

I knew it!

[thatskinnyguy]

... ribbon on a new 4.5 petabyte data center [CC] housed in a metal shipping container that sits outside.

I knew it! The internet is a big truck you can throw stuff in! What's this series of tubes business?

Awesome . . .

[petegas]

Now we'll never have to worry about losing the old cached version of goatse

Internet Archive= Censored webpages.Donors =STUPID

[zymano]

Why use it if webpages are being deleted from it.

I have tried to use it before on some websites and the information was ALL DELETED.

If a person puts a PUBLIC website up then it can be archived.

The internet archive just shows that it has no backbone and it ISN'T interested in being a legitimate archive.

way back machine dates are wrong

[societyofrobots]

I noticed that the dates on many webpages are entirely incorrect. For example, it says my webpage existed in 2001, when I started it in 2005 . . .

Re:way back machine dates are wrong

[Anonymous Coward]

Is it possible someone else owned the domain name before you?

Is this really fucking neccessary?

[Anonymous Coward]

How many copies of "website under construction" GIFs do we really need to archive from 1996?

A healthy brain forgets that which is not important enough to remember - maybe we as a planet should do the same?

This is a total waste of energy to maintain just so a couple of nerds can exclaim, "ZOMG remember frames!?!"

Re:Internet Archive= Censored webpages.Donors =STU

[u38cg]

Or possibly, there's just making the content unavailable until copyright expires? Seriously, they don't have any law behind what they do, so they have to tread relatively carefully in order to not cause themselves bigger problems than not being able to archive a small number of websites.

Bandwidth

[AliasMarlowe]

Actually, 100km/h (62.19mph) is 27.78m/s (91.13fps). So a 20-foot container on a truck will pass any given point in about 0.219sec. That's a burst bandwidth of 20.5PB/s or 164Pb/s.

My "fast" internet connection is more than 9 orders of magnitude slower, at a mere 100Mb/s. Now I'm really annoyed with my ISP.

Re:Bandwidth

[rackserverdeals]

Thats pretty fast! I figured I had the calculation wrong.

I've seen someone calculate truck bandwidth before when trying to decide whether to transfer backups over the wire or by wheels.

We should get rid of wires all together and have the internet run on trucks. :)

When "Skynet" becomes self-aware...

[Peet42]

...it'll have the whole canon of 1930s Film Noir to use as reference material.

Obligatory

[nmg196]

4.5 petabytes should be enough for anyone.

Re:Obligatory

[hesaigo999ca]

Not me, I have a desire to get ALL the movies off of pirate bay

What about all that heat?

[Ukab the Great]

63 servers with 3024 hard disks in total jammed into the confines of a metal shipping container that sits out in the sun.

That's sounds like either a recipe for disaster or a great Mythbusters episode.

ZFS?

[otis wildflower]

I wonder if they're running OpenSolaris/ZFS on these hosts...

Re:ZFS?

[rackserverdeals]

Solaris 10 and ZFS according to the article.

Confused

[imajinarie]

isn't the Internet Archive the same thing as Google Cached Pages?

You beat me to it

[denis-The-menace]

I don't know how many times it go to the "way back machine" to find NOTHING!

Ever since it's been used in courts, the archive has been deliberately censored BEFORE backups even gets inside! I bet most of it now are Spam sites and startups-before-lawyers-get-involved sites.

If MS takes something down, don't bother looking at archive.org. It was never copied there to begin with.

math

[misterjava66]

63 * 48 * 1TB = 3024TB ~= 3PB
For where does the number 4.5PB originate?

What about google....

[hesaigo999ca]

OK, so google is indexing the whole web, as well as these guys.....that's great, we have a sort of redundancy should anything go wrong.