Slashdot Mirror
Mass Storage Leaves Microchips in the Dust
Roland Piquepaille writes "This article from Wired Magazine looks at storage with a new angle. 'Right now I am sitting in front of a whirring 60-gigabyte hard disk that cost less than $100. Do the math: If back then 10 megabytes cost $1,000, then 60 gigabytes would have cost x, where x = $6,000,000 and "back then" = 18 years ago. I'm sitting in front of $6,000,000 worth of mass storage, measured at mid-1980s prices. We have Moore's law for microprocessors. But who's coined a law for hard disks? In mass storage we have seen a 60,000-fold fall in price -- more than a dozen times the force of Moore's law.' DeLong also looks at a non-distant future when a $100 mass storage device will hold a full terabyte. He also thinks that with disk space becoming cheaper and cheaper, we'll be tempted to archive everything about ourselves, including pictures and videos. This is in fact the goal of the Gordon's Bell project, MyLifeBits. You can learn more about the MyLifeBits project by reading this NewsFactor Network article. Check this column for more details."
No only the price, the size of the drives. 18 years ago a 40 Mb HD has the size of a toaster...
The package said "Windows XP or better. Pentium Class Processor or better"... So I got a Mac with OS X
In general the problem is that while capacities have lept up, the rate at which we can read/write to those drives has not kept pace. It's not so bad for the iPod in particular, but at some point it's going to be a real problem for desktops and laptops, assuming our appetite for capacity grows as the capacity does.
Can't you see that everyone is buying station wagons?
Bloat will kill the increase in storage available - one way or the other. It'll be a 3gig version of word, or windows movie maker that will only save in raw, non-compressed video. Anything to drive the market. We've seen it with processor speeds, if HD prices keep dropping I'm sure well see it with storage as well.
Come on, is XP is SO far ahead of NT 4 that it requires 4x the ram? Of course not. But what MS reccomends, PC manufacturers will have to yield to.
A lot of the developments that have made disks so high capacity came from spintronics research. Here is a link to an article on Scientific American about how it works: http://www.sciam.com/article.cfm?articleID=0007A73 5-759A-1CDD-B4A8809EC588EEDF
He's predicting only a tearbyte for a $100 in 2012. Right now desktop drives are about a dollar a GB. So, he's predicting about a 10 fold increase in the next 9 YEARS!!! What have we seen in the last 9 years, about 100 fold increase?
If that rate continues, some day hard drives will become so large that processesors will not have the power to process it all....
I will know that day has arrived when the length of my winamp playlist rolls over into negative integers. :)
Muerte
I have to admit that the notion that it is now techincally possible to mpeg-1 every moment of one's existence is a staggering one.
If you accept that Blogs satisfy some previously underestimated human desire for self-expression, think of what might happen if one could clip a web cam to one's collar, wear a storage device on one's waist and synch that with an online VidLog every night like a Palm Pilot?
I am going out back to sit among the dandelions.
The best way to do is to be.
An interesting feature of OpenNap is that it tells you exactly how many MB of files are out there for your downloading pleasure. I used to be blasted away at the large number. Sometimes I could get it up to 1 or 2 pedabytes.
A terabyte is 1000 gigs. You can get a terabyte of storage today for $1000 dollars. One dollar per gig. It's insane. Soon it will be a dollar a terabyte. We wont need things like divx anymore. We'll be looking for ways to increase the quality of our recording devices so that the video, image and audio files will take up more space. Nothing else really requires a large amount of storage.
The one limited is network speed. Sure, if I've got enough room for a collection of 2 gigabyte raw avi movies, that's great. But if I can't get enough speed to download them quickly it will suck.
Storage aint worth crap if you dont' got enough stuff to fill it.
Remember the days when DOS games would ask questions like this
minimum install (if you're low on space)- 50MB
standard install (reccommended)- 100MB
big install (runs faster)- 250MB
CRAZY INSTALL (no cd required!) - 500MB!!!
those were the days...
The GeekNights podcast is going strong. Listen!
Actually, I started a project to do just this for myself about 7 years ago. I'm 28 now and it's taken me until NOW to catch up (and I'm only about 80-85% caught up.... still have plenty of baby pictures to scan). I just spent last weekend sifting through old documents about yours truly that my parents has stored away. I now have a ton of interesting data about myself on tap, including every report card I had until college. :)
The original motivation for this project for me was the realization that my generation may never have to face death and I was terrified at the prospect of being 1000 years old and having no recollection of my life as of today. So I decided to digitally augment my memory. And it's VERY effective. Seeing scans of ticket stubs of concerts I had completely forgotten brought back all those memories in a flash. I'd hate to image what would happen if I just forgot permamently and never had a 'key' to unlock it.
Since starting the project the original motivation has been eclipsed by an even more compelling one: Going through all these documents made me realize that I've lived an amazingly full and rich life and I'm only 28. I was overwhelemed with awe and gratitude at how huge and wonderous my relatively normal life really is and how tragic it is that I've forgotten that. In other words, it was a great way to refresh the brightness of all the colors in my memory, which left me feeling uplifted and more optimistic about the future than ever.
(Still, every once in a while I imagine myself a few centuries from now, bored out of my mind on an multi-year interstellar trip to somewhere and enjoying a good browse through 'The Story Of Me')
In a few years time, when storage is cheap enough, I'm going to have a camera permanently strapped to my head (think better minaturisation + wireless tech) recording my life full-time. At moments of boredom I'll be able to relive any part of my life.
How much storage? Say, 500Mb/1hour (better compression as well, hopefully) * 24 * 365 ~= 4.4Tb/year. Doesn't seem that far away...
The challenge is no longer whether you can store everything, it is whether you'll be able to find it later when you need it.
This sig has been temporarily disconnected or is no longer in service
Well, they have in some ways.. Increases in capacity have come due mainly to vast increases in the areal density of the media. This, in turn, yielded massive increases in the rate the data moved under the heads..
The problem with hard disks isn't the data transfer rates they are capable of - it's their latency we need to worry about.
We need better defragmentation algorithms - I suspect that files are usually accessed in list order.. When running a program, for example, it's always going to want to read the same files in the same order. If we can arrange files that are usually accessed to be contiguous on the disk surface, and also make the filing system read the whole list of files that are situated contiguously into the disk cache when the first file in that list is read then the relatively high burst transfer rates will take more precedence over the access times, and things will seem a lot quicker.
Moore's law is largely due to manufacturing improvements in which the feature size of transistors keeps becoming smaller, such that you can get (approximately) twice as many transistors in the same amount of space. (yes, yes, I know, die sizes keep growing, but not nearly at the pace at which transistors shrink.) The tricky part here is that this shrinking has generally been coupled with ramping up frequency. Increasing the capacity of a disk has no such benefit due to the fact that mechanical parts (disk heads, spinning platters) are the overwhelming determining factors for performance. Hence, the gap between processor performance and disk performance is being exacerbated - we can only make a disk spin & heads move so fast.
It's an interesting comparitive trend to notice (between processor performance growth & disk capacity to see the effect on the overall system), but you can't really compare the way disks have improved with the way microprocessors have.
30 years ago Century Data/Diablo 5Mb "pizza oven" drives cost about $5000 each. This equates to approximately $1K per megabyte.
40 years ago IBM top-loader 10Mb drives were in the $30000 range (IFIRC). This equates to approximately $3K per megabyte.
As an exercise, the student can compute doubling rate and/and or the cost of current hard drives at those price points.
Except that processors don't just give up the ship randomly(well, except in VERY rare circumstanecs)- drives do it all the time; it's almost expected. I don't give a crap about another 20GB or $20 off, I want a hard drive that won't turn itself into a paperweight after a year or two. If I'm going to own the drive for 5 years, what's another $20?
SMART was an improvement, but most OS's(linux included) don't even recognize SMART info out of the box. Even if you've got the SMART utilities installed and the kernel modules etc, /var/log/messages is so noisy, I mostly ignore it- same for Win2k boxes, Event Manager is full of TONS of crap(thank god it has filtering, but still...) If SMART were to be useful, the HD would beep at you, or blink its LED, or the OS would annoy you with popup messages so you knew, "oh shit, I gotta back up my stuff to somewhere else, NOW!"
I had an ancient 4GB Digital drive I got second-hand, in the early 90's; it was already several years old when I got my hands on it, so it was probably pre-90's. It weighed a ton, took up the full space of a 3.5" drive bay, and even had its own little suspension system. I abused that thing to hell and back, carrying it in bookbags, cooking it when the fan on the external case died...the whole nine yards. I think I low-level formatted it a dozen times(something you're not supposed to do often on SCSI drives, supposedly). It only finally gave up the ship around '99, when it spent a couple months cooking itself to death hooked up "temporarily" to a machine I forgot about.
Meanwhile, I've lost two quantum drives(one laptop, one Ultra2 3.5") and my athlon's Maxtor drive is making funny noises every once in a while. None of them were more than 2, 3 years old TOPS. WTF? The excuse seems to be that consumers don't need the reliability corporate users 'demand'.
Home users users have, at the very least, equal needs as business users, because while businesses need to keep going 24x7, they often have backups, clusters, RAID units, etc. Most home users don't have any of their data backed up, RAID is practically unheard of among the jane-and-bob computer users, and of course no clustering.
Please help metamoderate.
"The point was that they haven't gotten faster in proportion to the increase in size. "
Ah I see, that's true.
Not that surprising, though. The mechanical arm inside of the drive has its limitations. I wonder what it'd take to replace it with a magnetic field sorta like what TV's use to fire energy at the phosphors on the tube. I wonder if a disc (maybe optical disc?) could be read that way. Seems like you could dramatically increase it's read speed that way.
Blah I'm sure there's a serious issue that I'm not thinking about. Oh well.
"Derp de derp."
There are physical limits sure, and yes as I'm sure someone is going to point out, they have gotten faster, but nowhere near the same rates.
While processors and storage space have all grown at significant rates, storage transfer has increased incredibly slow. I also remember reading an article not so long ago that the limits on spinning a hard drive any faster than it currently is (in research not in your desktop) is that spinning faster will cause itty bitty sonic booms which of course won't treat the hardware so nicely.
And you raise the issue of optical drives... there is some nice research (an old prof of mine is working on) where they store databases in crystals, 1 cm^3, such that a pulse of light is sent through it and all the information (terabytes) are read instantly. However, the optical->electronic transfer still takes significant time.
Course he also mentioned if someone bumps the desk holding the crystal it takes days to realign the laser.
Drives today have 10.000 rpm or 15.000 rpm. Eight years ago the high end was 7200 rpm, 5400 before that...
;)
That's approximately a 2X performance increase per EIGHT YEARS. This is very very far from being impressive.
Disk seek time is dominated (today) by rotational latency. The fastest disks have seek times around 4ms, and that is pretty much the rotational latency on a 15000 rpm disk.
In order to improve disk performance (the seek time, not the throughput), disks need to spin faster. This does pose some interesting problems though...
A normal 3.5" drive has a platter with approximately 48mm radius, giving roughly 0.3 meter circumference. At 15000rpm the speed of the circumference is 75.4m/s.
Doing the math, this gives us a centripetal acceleration of v^2/r = 118435 m/s^2, or roughly 12085G. Sure as hell beats most drag racers out there (by more than a factor of 12000
The fun part is, that a simple doubling of the rotational speed, will do really interesting things to the acceleration (note the v^2 thing above).
A 30000rpm disk will have a centripetal acceleration of the circumference of approximately 48000G.
A mass-element at the circumference weighing one gram, will have a "pull" corresponding to (F=m*a) 118kg - which again will be approximately half a tonne on the 30000rpm disk.
You need to find a material that will weigh little, not deform under the given stress, and still have the necessary properties for use as a hard drive platter...
Moores is the # of transistors/processing power every 18 mos... you're looking at price per byte.
lets see what $100 gets you
$100/meg = 1985 10 meg
$50/meg = 1986.5 20 meg
$25/meg = 1988 40 meg
$12.5/meg = 1989.5 80 meg
$6.25/meg = 1991 160 meg
$3.13/meg = 1992.5 320 meg
$1.56/meg = 1994 640 meg
$0.78/meg = 1995.5 1.2 g
$0.39/meg = 1997 2.4 g
$0.19/meg = 1998.5 4.8 g
$0.09/meg = 2000 9.6 g
$0.04/meg = 2001.5 18.6g
$0.02/meg = 2003 37.2g
$0.01/meg = 2003.5 74.4g
Looks like the curve is a bit faster than every 18 mo... I think 12 months might be a better approximation of storage/cost.
meh
"Basically if programmers still gave a damn like they did when writting code for C64's we wouldn't have alot of these issues. Nowdays they would rather churn out crap so long as it's better than some of the other crap they've seen."
If you're trying to tell me we should go back to the days of non-portable assembly, I think I'm going to cry. Yes, people should write tighter code, but trying to make believe that we should write code just like in the "good old days" is ignoring years upon years of advancement in the field of computer science.
And, also, look at what they were doing back in the days of the C64, and look at what they're doing now. You really do need more code to do more. Trying to tell me that they had 6kb executables with the C64 and then telling me our 6mb ones are bloated is ludicrous.
-Erwos
Plausible conjecture should not be misrepresented as proof positive.
I remember the 486 with a 255 MB hard drive we had when I was in middle school. My Dad was pissed because DOS and Windows 3.1 took up nearly 70 MB of precious storage space when the version of DOS that came with our old Tandy 1000 EX fit on a single 5.25 inch floppy diskette. Dad was even more pissed when I filled the drive with games, WAV files, and pictures. Back then, I was excited whenever I was able to free up another 750 kilobytes of disk space. Then there were the hard decisions...is removing Rise of the Triad, my all-time favorite game, worth freeing over 20 megabytes?
When I started college I bought a Pentium with a 4 gigabyte hard drive. Unlimited storage space! Well, until a friend showed me this awesome new program called "Winamp."
To this day, I'm very frugal with disk space. My home directory resides on a 60 gigabyte drive split into 3 20 gigabyte partitions, and I'm only using 17% of one partition right now.