Slashdot Mirror
The Past and Future of the Hard Drive
Snags writes "Brian Hayes of American Scientist has written a nice little historical review of hard drive technology, from the first hard drive (nice pic) made by IBM in 1956 to what may be available in 10-15 years. He muses on how to fill up a 120 TB hard drive with text, photos, audio, and video (60,000 hours of DVD's). Kind of ironic that this came in my mailbox today considering IBM's announcement."
I don't care how much space you give me.. its never enough. I'm running around 300 gigs right now and it mostly full. Why? because I do video work and motion graphics design, and I'm not very organized. I've got tons of source files from archive.org and I really don't want to go through the trouble of burning CD's or backing up to tape - I love to have it all accesible quickly via harddrive.
So of course I'm not going to fill all that space with typed notes, but even if I dont do as the article suggest and "document every moment of our lives and create a second-by-second digital diary". I still want that space for massive amounts of easily accesible data.. There's no reason I should ever have to delete anything ever again..
Uh. Except that I cant find anything I'm looking for anymore.. Can't this search function go faster?
air and light and time and space
120TB seems like an enormous capacity, but multi-terabyte storage mediums will be a neccessary in the future with REAL streaming media. What I'm worried about is the network connection - that has been my current bottleneck (even at 1.5Mbps). When we eventually combine HDTV PVR's, MP3 players, DVD archives, and pictures into a giant media database, the numbers don't look as staggering. But transferring that much data from one machine to another may prove to be the hardest part of all. 120Tbps network connection -now THAT'S impressive!
He muses on how to fill up a 120 TB hard drive
Let's see - if the history of the internet serves as an apt model - 120TB drives probably won't meet consumer demands for long.
First harddrives will start to fill up with fully-imersive holo-pr0n, followed quickly, due to adaptive marketing trends by fully-imersive unsolicitted holo-spam.
There... that solves that ol' capacity problem quite nicely then.
:)
Sure it can holds 60,000 hours of DVD. That's why when the time comes we'll come up will a more precise format. I don't believe we will had enoguht space until it can store a lifetime's worth of information in a format indistinguistable from reality to human senses. Until that point we will always be able to make it a little more lifelike, a little longer, there will always be somrething else to eat up the space, clockcycles and bandwidth. My 1 GB filled up just as fast as my 256 MB and I'm sure that my 60 GB will fill up as well. We still have a long way to go before we have "enough" space.
I stole this Sig
There are already a number of Terra satellites downlinking data at about 4GB/hr, circling from pole to pole in orbits lasting under 2hrs.
.
There are multitudes of airborne surveys churning out digital snapshots at 400MB a frame.
Mosaiced together at 1m resolution with R,G,B and mean height above sea level, how much storage will a single global snapshot of the earth take ?
Then consider for historical and environmental reasons, most urban/semi rural areas deserve a mosaiced snap at least once a year.
120 TB is just the start . .
Adding electronics to drives to gain more space is not a particular good idea in this day and age. You are not going to get 1000 bytes out of a 100 byte read very often, probably only if you are reading text. As you note Video and Graphics are usually stored in formats that are already compressed, so storing them on a drive with hardware compression wont win you much. And as the article says, text doesn't take up any noticable amount of space anyway. So even if you mostly care about your source code etc, you probably doesn't have anywhere near a GB, which is 1% of a modern drive.
Furthermore, compression and decompression takes time, so it would lower the performance of the drive somewhat (no idea how much, but some).
Wasn't Stacker those guys who made a piece of hardware to place between your drive and the IDE controller, to do the compression?
15 years ago, I was speaking with a freind of my fathers about these new exotic 'hard drives' you could get. He was a big time computer guru, and I was a little kid.
I said to him that I thought a 10 megabyte hdd would be perfect, and that someday we would be able to buy one for less then $1,000. He scoffed.
I still remember this very clearly. "Why in heavens name would you ever need that much space in a hard drive"
I worked it out, and every concievable program I'd use, including saved files, all told would only need 2 megs. It was just impossible to think up enough applications at the time that a home PC really needed.
I'd love to run into him again and ask him if he remembers that conversation. In 10 years, 100 Terrabyte drives will seem 'quaint'.
The Internet is generally stupid
"Thus the 120-terabyte disk will hold some 60,000 hours worth of movies; if you want to watch them all day and all night without a break for popcorn, they will last somewhat less than seven years." Most people can't even last seven minutes with high-res pr0n playing, much less seven YEARS.
Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
I used to think that my files would expand to fill all available space, but not anymore. Different tasks, different tools, and different personalities mean different thresholds, but I think everyone has a threshold above which they won't keep their disks full. For me, my disks stopped being full around 10 gigs. My wife's antique PC (running msdos 2.0 and used strictly as a text editor) had a 15 meg drive and never went above 10% full. Obviously your threshold is higher, but I'm sure it exists.
Even after they hit their thresholds, most people's use will grow over time, but slowly. We'll also start writing things in ways that don't try to conserve disk space. Compression will be used almost exclusively for data transmission. Future filesystems will probably keep every version of every file ever saved. (Hopefully with an option to delete the occasional residue of an indiscretion and accidental copy of /dev/hda). But even these things won't increase our use by us more than a factor of 10 or so. If we really do get 120TB drives, we won't talk about buying new ones very often.
Mmmm.... future pr0n.
sic transit gloria mundi
Whenever I see an announcement of the newest harddisc with record breaking capacity I think "How do you backup that beast?".
Whenever harddisc manufactureres manage to double the number of bits that can be written on an inch of a track they get an four fold increase of capacity. But unless you increase the rotational speed of the plattern the time to read the whole content of the harddisc will double as well since the recording/writing speed is proportional to the linear density.
And the rotational speed only increases very slowly - we recently saw the small jump from 5400 to 7200 RPM for the "standard" consumer (IDE) harddisc, the first for several years (I personally stick with 5400 for the cheapo IDE drives for the next few years. Reliability, you know --- see IBM)).
Given that the lower limit for the time to make a full backup of harddiscs will increase roughly with the square root of the growths of their sizes over the time.
The other problem is that backup devices and media affordable for the home user can't keep pace with the harrdiscs, so in my eyes the traditional full backup get's more and more inpractical.
One of the most cost effective backup devices for a harddisc today is another harddisc, but it still needs hours to mirror the content of one disc to another. RAID or something that keeps two discs in sync automagically in the background is no solution - it saves you from data loss by harddisc failures (good if you use IBM GXPs and the like) but it won't help you if you or your software have/has destroyed some important files you have created over the past few weeks/months.
Well, I don't know what other people do, but I stopped doing full backups of the whole disc: Thank God a large amount of files on my harddiscs is not backup worthy since in case of loss I won't miss it (think swap space or contents of a web cache), can easily recreate it (like mp3s made from my own CDs or object files - if you track netbsd-current you keep them around to save some time on future inkremental builds and don't delete them after "make install") or get it back from another of my machines, CDROM or the net.
So I only have to backup a fraction of my discs and the good news is that in absolute numbers the amount of data to backup doesn't grow nearly as fast as harddisc capacities. In my case with compression it easily fits on an older harddisc for complete backups and for weekly incremental backups I can still use and old 1GB DAT (DDS) tape I've got for free. It's not the best solution since recovering from desaster needs some time and a lot of manual work but I can sleep better than those who don't do backups at all...
1920 x 1080 pixels
16 bits/pixel x RGB = 6 bytes
60 frames/second
Yields 746,496,000 bytes/second. (Or about 8 parallel gigabit ethernet cards)
Do this at full bore, and you get 160,751 seconds of video, less than 2 days worth!
Sure, I know you could compress the video, but I've seen 1080p up close and personal, I noticed the artifacts in the video on the monitor of the broadcast quality HDTV demo, and the sales guy finally confessed that they just had to compress it to make it feasable to record it on tape.
If I noticed it right off the bat, someone will pay to have this quality level.
So... when do we get the petabyte storage?
--Mike--
I'm surprised no one else here has ripped a DVD to their HD before. 2 gig per hour seems like a minimum to me.
And even if it isn't, by the time the 120 TB disk come out, you think we're still going to be using the worst DVD format? 720p is supposed to be coming out in a few years. I can't find the page with the details now, but 720p will require a new type of DVD disk, one that can store up to 24 GB.
That's right, your DVD collection is going to become outdated and worthless as companies republish into the new "Hi-def" DVD format.
Let us say the average DVD uses 70% of the space, 16.8 GB. That's about 7,100 hours. Compression might be less effective on these disks, since the entire point of having Hi-Definition DVDs is the extra detail.
And I've come up with another use for a 120 TB drive. The biggest, most kick-ass TIVO in the world. Imagine having any TV show that was shown in your lifetime available to view.
Democrats or Republicans. They are both taking us to the same place and they are not afraid of us anymore.
A lot of people are mentioning 3d data but...
There is no way of capturing a fully 3d image
perhaps i'm just low on creativity, but I can't imagine any way of capturing vide from real life with a system that isn't functionally equivalent to some finite number of video cameras.
now lets say that in the future telivision is filmed at 10x DVD resolution and from 10 different angles
that only adds 100x to current video storage needs. nothing to sneeze at, but also not so spectacular that you'd need a petabyte drive either.
once we have the ability to record any sort information at fidelity approaching the maximum for human perception, storage growth will rapidly outpace our need.
people seem to be upset about this though...
once storage reaches the maximum that any user would ever need, it has no place to go but cheaper.
also, file system organization will need to be massively overhauled to make a petabyte drive remotely useful.
In Capitalist America, bank robs you!