Slashdot Mirror
The End Of The Road For Magnetic Hard Drives?
Phase Shifter wrote to us about the limits of conventional hard drives, which Scientific American is discussing. The article talks about the history of hard drives, and why sometime soon, due to the limitations of the superparamagnetic effect, we will need to find a new storage type. It's a cool background read on hard drives and what goes into them.
OK, we are going to have to find a new technology because they're going to hit a limit on increasing drive density... Ummm, no.
To double storage capacity it is not essential to double density - you could double the physical size of disk (or add another).
Just because they cannot get smaller doesn't mean they get ditched - Want double the storage, get double the number of disks...
I did a report in college on bubble memory. Wave of the future. Got a B. Bubble memory got the boot.
If you aren't part of the solution, there is good money to be made prolonging the problem
Without the common "thin film" type data storage we've been with for so long (MFM started major production around 83?) what will we be switching to next. There have been a few /. articles about such stuff, I'm too lazy to go and get some links, but they're there. Either way, optical storage is the future. Fof instance, there was a /. article talking about being able to have a CD that held 140 GB of data, and a roll of scoth tape being able to hold 10 GB of data. Another possibility is RAID will become common, allowing the easy use of multiple drives. I imagine this will be the temporary solution, as the above two optical solutions are a long ways away from being seen at your local PC store.
Don't call my crazy, that's what they called me back in the home!
A Brief History of Hard Drives
(a la Book-A-Minute).
Scientists: OH NO! Hard drives can't get any better!
Engineers: Wait! Your science is WRONG! (Writes some new equations).
Computer industry: You have SAVED us!
Geeks: YAY!
Unlike other computer technologies, the hard disk market consistently finds some revolutionary way to make their products faster, bigger, and cheaper, while staying in business. With that kind of competition, I don't think the hard drive is going away...
--
--
E2 IN2 IE?
Xerox is sponsoring a research of 3D storage devices that can manage tens of thousands of gigabits in a volume of sugar cube at the University of Toronto
You can't handle the truth.
What to do? If I needed a RAID system that would have access time of RAM or even faster. In fact it should be possible to do even with the current technology, the rotation speed of a hard drive should be incremented by about 100 times and the number of tracks should be decreased by 10 times (making the bits on the platter much larger, let's say by 10 times) the distance between the tracks should stay almost the same (less radial travel time for the head) maybe increased by 2-5 times. I want this sucker to have up to 50 reader heads on each platter, let's make the platters larger in size and all. This should be able to bring the speed up to match the performance of today's RAM speeds, of-course the capacity on one such hard drive will go down by about 10 times (so instead of 9GB it would be about 900MB, then put something like 50 of those together into a RAID system and we have about 40GB of data with access time of RAM. The cost of one drive would go up due to the better mechanical part of the drive (liquid bearings instead of ball bearings, higher rotational speeds and the technology to combat air turbulence) but it also would go down due to the lower resolution of a platter and larger bit sizes.
This is EVIL EVIL EVIL idea HAHAHAHAHHAAHAHAAAAA!
You can't handle the truth.
Hard drive technology has progressed at least as fast as other computer technologies. Let's compare the present day to the day of the original IBM PC XT, some 17 years ago.
Processor, 4.77 MHz -> 600 MHz: 126 times
(let's say 1000 times, because the P III does a lot more with each MHz than the 8088)
RAM, 64 KB -> 64 MB: 1024 times
Modems, 9600 baud -> 56K: 6 times
(even 1.5M for cable modem is only 156 times)
Hard disk drives, 10 MB -> 20 GB: 2000 times
Hmmmm, seems like the much-poo-poo'ed electro-mechanical technology has easily kept pace with the straight electronic technologies, including the breathtaking advances in chip density.
Now, when it looks like CPU speed and RAM density really ARE about to reach a plateau for a while, or at least lower its slope of advance, hard disk technology is poised to really rocket ahead. Look at the news from IBM research, foretelling VAST advances in the fairly near future.
I don't have a link to this info handy, but my recollection is that bubble memory was _way_ too slow - and hard drives just kept getting better ...
... but at least some of that will be due to the massive research that has gone into making the current technology work well.
It seems to be a bit of a trend in this industry that whatever works early on gets a lot of resources put into incrementally improving it and making it cheap, such that competing technologies have to be _hugely_ better to have any chance of taking over.
That is (IMO) partly why:
- we still use hard drives,
- CPU's still use CMOS rather than one of the faster switching methods,
- the x86 architecture is still dominant,
- the UNIX model is the base of nearly all operating systems.
There may be potentially 'better' technologies than these out there, but there has been so much engineering and optimisation gone into these technologies that it is really hard for anything to compete.
The case of the Exponential PowerPC is an example of that - it used ECL rather than CMOS to get substantially higher clock speeds, but before it had really got up to speed, the incremental improvements in CMOS had passed made it look less attractive, and Exponential was dead.
I expect someone to reply to this and say how much better CMOS (or whatever) is better than anything else
Since everyone will be replacing their hard drive with rolls of scotch tape, I'll corner the market!
Fight Spammers!
Then, one day, someone realised that - hey! If you throw away the assumption that baud == bps, you can actually drive up speeds to 56Kb/s!
Then, as modems went up in speed, the same engineers moaned and groaned. The 56Kb/s limit was near, and without a total rewiring of the phone network, an act of Congress in the US (an act of God elsewhere in the world), and more money than anyone had, the 56K barrier would never be breached! Calamity!
Then, one day, another bright spark realised that if you had modems at the junctions, you could shove REALLY high-speeds down the wires without either Congress -or- God having to do anything. (Much to the relief of both.)
The Doomsday Crowd, defeated once more, lurked on the fringes. Until, one day, redemption! Hard Disks can't pass a certain density!
This, of course, is as bogus as all the other claims. If it's possible to read the past ten writes on a given sector, then you can you can increase the density of the disk by AT LEAST an order of magnitude. You just have to remember to read/write all ten layers at one time, and you're fine.
Then, of course, there's no rule which says you have to use 2-state logic. It's easy, but it's not mandatory. Magnetic fields can have any orientation and any strength. So long as the maximum strength isn't so high that you get bleeding, you're fine. Recognise 256 possible states (using any combination you like of orientation and strength), and you've "encoded" a single byte into a bit - a x8 gain in disk capacity.
Combine the two, and you've increased the capacity by over 80 times! This can be increased still further, by increasing your ability to scan over-written layers, and by increasing your ability to distinguish magnitudes and orientations. You have two degrees of freedom for rotating the magnetic field, which means that by doubling the ability to distinguish, you quadrouple the number of possibilities available.
The scientists may be correct about the density, but the density is NOT the only variable open to hard drive manufacturers. In the future, it may become one of the least significant, as others are explored.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Yet again. Densities may not increase forever, but when you can cram 40+ GB on a single platter, just add a couple of platters and make the drive a little thicker. Anyone remember full-height 3.5" drives? Maybe they'll make a brief comeback once density plateaus.
I'm much more concerned about two other relevant factors:
1: The I/O bottleneck inherent to IDE and SCSI interfaces. All this horsepower, and all this storage, and we can't transfer it fast enough.
2: In case nobody's noticed, tape drive technology has gotten faster, but it has not kept up anywhere near hard drives from a capacity standpoint. In a network server setting, this can be a real problem! The data sizes and drive sizes are growing, tape speeds have increased somewhat, but the network speeds are still mostly at 100 Mbps or slower, and the backup window times are shrinking quickly. That's a bigger problem. We need faster interfaces and bigger tapes - or cheaper jukeboxes.
- -Josh Turiel
-- Josh Turiel
"2. Do not eat iPod Shuffle."
If cheap+fast permanent storage arrives quickly, then Oracle, for one, will be in deep trouble.
Durable storage without moving parts could easily be three orders of magnitude faster than magnetic disk tech.
With permanent storage that fast, PostgreSQL 7.0 would perform on a par with, if not faster than, Oracle 8i. All the work Oracle has done to optimise around magnetic disks would be rendered worthless or worse-- imagine how annoying it could be for a newly hired developer to slog through all of that newly-obsolete disk "wizardry" just to fix a bug...
Size is the only dead end in site for hard drives.
- Speed. The average hard drive is spinning at 7200 RPM nowadays. At this speed, there is an average latency of 4.6ms just to spin the track under the head. You can't do much about this except spin the disk faster. At 10000 and 15000 (thanks Seagate), you still have 3ms and 2ms, respectively. This is on top of any time needed to move the head itself. With most access times <8ms in the low end and <5ms in the high end, this ceiling isn't too far off. Sure you can spin the disk faster, but this gets expensive (money, energy, and heat).
- Size. I think the article addresses this quite nicely. If we hit the ceiling here, we can increase the surface area. But this again gets expensive in all ways and precludes a 100 GB laptop drive in a 1.5 in width and 1 watt power consumption. You know you want it.
- Reliability. To me, this is the biggest problem. Hard drives are the most relied upon moving part in a computer and yet are the first thing to go in most systems (followed closely by the power supply, who's death is usually hastened by a power hungry hard drive spinning up and down). RAID (or similar) can tackle this, but is expensive in all ways and requires the user's attention.
Finally, I won't argue that hard drives will meet their doom in 5 years, hell we don't even have a suitable replacement yet (only stuff in research). I just figure they will be a story that I will tell to my grand children.The thing which would be valuable to consumers would be a sharp increase in data throughput. It's true that disk drive capacity has grown faster than CPU speed over the past few decades -- but data transfer rate has not. The result? The CPU is data-starved, both by the bus and the swap speed.
According to a friend of mine who works in the industry, the leading limitation on density is seeking from track to track and remaining locked to the track. There is some new head technology coming down the pipe which should vastly improve hard drive densities. One of the most difficult things to do is servo the heads. This new technology should eliminate this limitation (sorry, I won't go into details).
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
Mmm hmm. And do you think anyone would have gotten around to that realization had someone not observed that the "baud == bps" approach would not work forever?
Right, but would anyone have bothered to do this had someone not pointed out that you couldn't get higher speeds using the conventional approach?
The moral of the story is that there is value to pointing out the limitations of current technology because that is what allows us to avoid wasting effort by developing new technologies to replace existing technologies that don't need replacing. Conversely, it helps to anticipate problems in existing technology before they start to limit progress, so that new technologies will be ready by the time those limits are reached. This is not "doomsaying", it is simply having a good understanding of current technology. You have to have a thorough understanding of existing technologies, including their limitations, before you can hope to improve on them.
-rpl
Wow, it really is Scientific American. Down with metric!
73GB, Ultra-160 SCSI (160Mbps), 10K RPM. About $1650 available almost anywhere (except in Seagate's online store. Go figure.). Quantum's got essentially the same drives now tho I didn't notice them for sale.
Do the math: Put, say, 7 of these drives in a $300 external enclosure and you've got over 400GB usable RAID-5 for < $12000! That's $0.03US / MB.
"When an elderly but distinguished scientist says something is impossible, he is almost surely wrong" - Arthur C. Clarke
I collect Scientific American, and one of the most fascinating aspects of my collection is the series of articles on why this or that technology won't work or has reached it's limits. The authors that SciAm gets to write it's articles usually fit the definition in Clarke's Law above, and they have invariably been wrong, usually quickly.
Two examples:
SciAm published an article in 1947 on why long range ballistic missiles wouldn't work, mostly based on the inability to make the guidance systems accuarate enough. About 5 years later we were deploying them.
They also published an article in the 1980s on why space-based lasers for strategic defense wouldn't work. I was working in that area at the time, and the problems they raised had already been solved, we just couldn't talk about it because it was classified.
Here's an approach for increasing magnetic storage capacity I haven't seen elsewhere: Current tape drives are high capacity but slow. They work just like ancient scrolls, unrolling and rolling up on a spool. Think instead like a codex (i.e. a modern book with pages). Have a stack of magnetic sheets arranged like the mess of catalogs at an auto parts place (spines down, pages held to +- 45 degrees of vertical by end holders). Use a static charge to fan out the leaves at the place you want to read, then slip in the read head from above. This gives you 3-D magnetic storage with fast (at least compared to tape) access time.
Daniel
Magnetic Drives have been predicted to fade out long ago. Ever since the late 80's with the optical drives that held ~2gigs, magnetic hd companies struggled against the beliefs of the physicist. Engineers developed newer ways of making magnetic media more efficient, while the optical theories were more effective in the long run, the drive for magnetic media was more prominent, as were the funds. Lets face it....how many people own magnetic media as opposed to optical media. In the hardware industry, if there is funding and money, there is progress. MORAL: Support the optical media foundations around you. They will give you terabytes to loose files instead of gigabytes :-)
By the time we've met with the capacity of magneto-resistive drives, we'll be moving on to something else. As the article said, thin-film didn't last forever, who/what is saying that MR has to last forever?
There will be no storage shortage in the future. Who cares about the death of MR... bring on the next generation.
PS: Imagine how long a surface scan is going to take on one of these babies. Pack a three course meal, and a good book.
-- kwashiorkor --
Pure speculation gets you nowhere.
-- kwashiorkor --
Leaps in Logic
should not be confused with
Jumping to Conclusions.
FYI, here are the general RAID levels:
Raid 0 - disk mirroring
Raid 1 - disk striping/no parity
Raid 4 - hair striping?...no wait, wrong list *hehe*
Raid 5 - disk striping with parity
Do you ever feel like you're diagonally parked in a parallel universe?
Worrying works!! 99% of all the stuff I worry about never happens
Reasons I'd love supermassive, small capacity drives. Imagine buying books, and having them all available on your palm pilot (every o'reilly, right there at your fingertips, with of course every RFC, the entire acm digital library and a few others). Imagine having every movie you own in your pocket, also on your palm pilot, with an adapter so you could watch any movie you already own while you're travelling. Every movie, every cd, uncompressed, full quality. No need for the distortion of mp3 when you have a terabyte in compactflash format :-) The reasons why there isn't any current applications like this, is because it isn't currently feasible. At home I have around 150gigs of storage on my network, most of which is in one RAID cluster. Useless? it seemed so but I keep on finding handy ways to use it. My latest project is storing all my live concert recordings (dat and cd) (yes they're legal), to .shn files so that I can easily spin off copies of them on demand. If you use your imagination, you'll realie that the number of people who keep legal video on their machines is small because it's not generally practical. Consumer-ized special-purpose computers such as the TiVo are changing this. A super-high capacity TiVo like device combined with broadband access and you could start selling movies legally, on-line.
----------------------------
Of course, we know what happened after that -- they quit using visual light, and started using shorter-wavelength beams.
A friend of mine says "Is tape storage on the way out? It's not keeping up with disk storage!".
Seagate and HP just introduced a tape drive with 100gb (UNCOMPRESSED) capacity, and they say that they can take that same technology to 250gb native. These LTO drives do this by having oodles of tracks on a tape so that a stretch of tape may have hundreds of tracks in parallel, and by using new tape materials that allow them to make the tape thinner so they can pack more tape into a cartridge. People said linear tape was dead, that helical scan would always be faster and higher capacity, but it appears that conventional wisdom is foiled again...
I don't know what hard drives are going to be like five years from now, but I do know they're not going to stall, capacity-wise, due to some "inherent" limit. Too many smart people are looking for ways to bypass those limits, either by using some other technology altogether (hmm... photo-sensitive materials??) or by figuring some way around the "limit" using clever application of the underlying physics.
-E
Send mail here if you want to reach me.
I've been looking at the data sheets on some of the big enterprise-class storage systems. We're talking about boxes that have 5 to 15 drives, and attach via fibre channel loop to multiple servers that need to be backed up, and that have hundreds of tapes that they manage via robotics. Yes, I'm working on enhancing the Linux 'mtx' tape library control software to drive these things, though I'll never be able to personally see or test one :-}. There are some interesting challenges to handle with fibre-attached storage, specifically, the one of "who has the robotic arm now?!", but none that are unsolvable. I am confident that no matter how big hard drives get, we'll be able to back them up -- albeit for a price!
-E
Send mail here if you want to reach me.
Tape drive manufacturers are raising capacities via a variety of methods. They are coming up with thinner tape materials so that they can cram more tape into a cartridge (I understand there will be a DDS5 that crams over 200 meters of tape into a tiny 4mm tape cartridge!). They are coming up with new heads that either store data more densely linearly, or that store data more densely vertically (i.e. put more tracks on a tape). They've also been playing with the speed at which data is recorded, and perhaps varying that to adjust to tape quality etc. There are also experiments ongoing with multiple heads and serpentine tapes, though I haven't heard that this is buying anything (easier to have a smaller cartridge and multiple simple drives rather than big complex cartridges and one complex drive). Having seen these guys do so many "impossible" things (they said that DDS4 was impossible!), I've given up on figuring out where it's all going to end, but I do know that traditional tape drives are nowhere near their limits as far as speed and capacity go.
-E
Send mail here if you want to reach me.
I am working on a FibreChannel-RAID adapter at my job. FibreChannel-RAID answers these questions pretty well..for now.
1. I/O Bottelneck - Our card does 190 MB per second with 23,000 I/Os per second in non-RAID mode (direct connect). Another good things to do is offload as much as you can to storage processors. This saves the main CPU. Relying on System DMA is a big part of what kills IDE performance. Both SCSI and FibreChannel adapters are DMA Busmasters meaning they can read/write to host memory on their own, without using host processor. Always use hardware RAID (adapter or external/cabinet based) instead of software based. Software based RAID kills processor.
2. Backup - Various forms of RAID can help here. You can configure things so that there are always at least two copies of your data. This doesn't help for real backup where people need things that were overwritten, like tax records from five years ago. Using RAID arrays of FibreChannel tapes speeds things up quite a bit.
As for network speeds...you are right about 100 Mbs being too slow. Heck 1 Gbps (or 128 MBps) are still too slow. That is why you use FC arrays that support multi-initiator. Here, multiple hosts are connected to a set of storage. In this model there is no server front-end to the storage share. No network latencies.
SAN technology is really just starting. Target mode systems (like EMC's storage cabinets) have great possibilities. Simple FC-Adapters can run in this mode as well. In a raw format, they can avoid the OS almost entirely, using it only for initialization and configuration. Backup can be done without any OS interaction.
-- soldack
These things are becoming more common each day. Data warehousing and SANs will create even more demand. Then you have video-on-demand servers that have full screen digital movies on tap. I am talking about full control video with all your VCR functions of stop, pause, fast forward, rewind, etc. stored remotely and sent to simple cable box. This kind of applications take up lots of space.
As for a data-starved CPU...not with IDE. IDE controllers use the host CPU for DMA so your CPU is quite busy. SCSI and FibreChannel adapters are busmasters but they are also faster. It is true that today's CPUs can push much harder than today's storage. It is also true that even a 66 Mhz, 64-bit bus is too slow. Interrupt sharing doesn't help either. That is why PCI is on its last legs. PCI-X will not last too long either with InfiniBand on the way.
-- soldack