Slashdot Mirror
The End Of The Road For Magnetic Hard Drives?
Phase Shifter wrote to us about the limits of conventional hard drives, which Scientific American is discussing. The article talks about the history of hard drives, and why sometime soon, due to the limitations of the superparamagnetic effect, we will need to find a new storage type. It's a cool background read on hard drives and what goes into them.
But then again people have been predicting the demise of the hard drive for nearly twenty years.
Its luck has to run out sometime, but I am willing to bet that it will still give the best value for money for some years to come.
ungodly huge doesn't mean ungodly fast. can you say bottleneck...
OK, we are going to have to find a new technology because they're going to hit a limit on increasing drive density... Ummm, no.
To double storage capacity it is not essential to double density - you could double the physical size of disk (or add another).
Just because they cannot get smaller doesn't mean they get ditched - Want double the storage, get double the number of disks...
I did a report in college on bubble memory. Wave of the future. Got a B. Bubble memory got the boot.
If you aren't part of the solution, there is good money to be made prolonging the problem
1) What is the largest capacity hard drive money can buy today?
2) Do non-logarithmic graphs that don't start at zero suck, or what?
3) "kilometers per square inch" is a sweet unit.
Without the common "thin film" type data storage we've been with for so long (MFM started major production around 83?) what will we be switching to next. There have been a few /. articles about such stuff, I'm too lazy to go and get some links, but they're there. Either way, optical storage is the future. Fof instance, there was a /. article talking about being able to have a CD that held 140 GB of data, and a roll of scoth tape being able to hold 10 GB of data. Another possibility is RAID will become common, allowing the easy use of multiple drives. I imagine this will be the temporary solution, as the above two optical solutions are a long ways away from being seen at your local PC store.
Don't call my crazy, that's what they called me back in the home!
A Brief History of Hard Drives
(a la Book-A-Minute).
Scientists: OH NO! Hard drives can't get any better!
Engineers: Wait! Your science is WRONG! (Writes some new equations).
Computer industry: You have SAVED us!
Geeks: YAY!
Unlike other computer technologies, the hard disk market consistently finds some revolutionary way to make their products faster, bigger, and cheaper, while staying in business. With that kind of competition, I don't think the hard drive is going away...
--
--
E2 IN2 IE?
til magnetic hard drives are moved out of the market. There is still much research to be done before new types can even start to be available to consumers. Then, the first 2 or more years they will be buggy and extremely expensive. After enough people start buying them and they drop in price, there will most likely be a huge surge in development for a better way to make these hard drives and the ones that all the people bought will be obsolete. Then after about 7-10 years or more they will become common place in the market. The thing is that this isnt new news at all, computer hardware and software have and will continually replace itself with newer and "better" versions of itself. Chances are that the system you run now wont even have the opportunity to try to run one of the new types of hard drives that will be made available and even if it could, it would be too slow to handle them. So in conclusion, you have to roll with the market and upgrade as you see fit. Unless you have an interest in design, don't worry yourself about products that wont be available for many years.
Bigger hard drives = more bloated code. :-)?
Either that or...
Bigger hard drives = more pr0n and mp3s.
Man, I miss my C64...who needs hard drives when I've got my trusty cassette drive
A genius writes code an idiot can understand, while an idiot writes code the compiler can't understand.
Xerox is sponsoring a research of 3D storage devices that can manage tens of thousands of gigabits in a volume of sugar cube at the University of Toronto
You can't handle the truth.
It's like tying ribbins round your fingers.
q PE
And we're rapidly running out of limbs.
Next we'll have probabilistic memory based on quantum theory. (Such as the latest secure communication proposals)
And to think I used to send email out continuously onto the internet back to myself in the days when my university limited us to 200K network space...
I had dreams to write a virtual disk driver using mail servers across the world.
(I have better sense now though...)
zUE65db/j/nDUFJYb6i88bhwJz26I1SMdr78iB6VjqA+tp6
p7L98Z/QCg/7JD
-grin-
I mean, can you imagine attempting to eat a burger that size ? It is quite simply ridiculous.
thank you
dmg
bottleneck
If you aren't part of the solution, there is good money to be made prolonging the problem
Just bring one of these non-mechanical drives to the market and stop teasing us! :(
What to do? If I needed a RAID system that would have access time of RAM or even faster. In fact it should be possible to do even with the current technology, the rotation speed of a hard drive should be incremented by about 100 times and the number of tracks should be decreased by 10 times (making the bits on the platter much larger, let's say by 10 times) the distance between the tracks should stay almost the same (less radial travel time for the head) maybe increased by 2-5 times. I want this sucker to have up to 50 reader heads on each platter, let's make the platters larger in size and all. This should be able to bring the speed up to match the performance of today's RAM speeds, of-course the capacity on one such hard drive will go down by about 10 times (so instead of 9GB it would be about 900MB, then put something like 50 of those together into a RAID system and we have about 40GB of data with access time of RAM. The cost of one drive would go up due to the better mechanical part of the drive (liquid bearings instead of ball bearings, higher rotational speeds and the technology to combat air turbulence) but it also would go down due to the lower resolution of a platter and larger bit sizes.
This is EVIL EVIL EVIL idea HAHAHAHAHHAAHAHAAAAA!
You can't handle the truth.
I can get all my 16-64K Apple ][ programs on that 70 gig drive. And still have space for the C64 code.
Not to mention all my AppleScript letters. The biggest problem is getting UDMA/SCSI drives working on the Apple ][. The C64 is a whole different interface issue....
(Gonna run out of space? Bah, go on a data diet. Do you NEED to be storing HDTV DVD's on your hard drive?)
If it was said on slashdot, it MUST be true!
Hard drive technology has progressed at least as fast as other computer technologies. Let's compare the present day to the day of the original IBM PC XT, some 17 years ago.
Processor, 4.77 MHz -> 600 MHz: 126 times
(let's say 1000 times, because the P III does a lot more with each MHz than the 8088)
RAM, 64 KB -> 64 MB: 1024 times
Modems, 9600 baud -> 56K: 6 times
(even 1.5M for cable modem is only 156 times)
Hard disk drives, 10 MB -> 20 GB: 2000 times
Hmmmm, seems like the much-poo-poo'ed electro-mechanical technology has easily kept pace with the straight electronic technologies, including the breathtaking advances in chip density.
Now, when it looks like CPU speed and RAM density really ARE about to reach a plateau for a while, or at least lower its slope of advance, hard disk technology is poised to really rocket ahead. Look at the news from IBM research, foretelling VAST advances in the fairly near future.
I don't have a link to this info handy, but my recollection is that bubble memory was _way_ too slow - and hard drives just kept getting better ...
... but at least some of that will be due to the massive research that has gone into making the current technology work well.
It seems to be a bit of a trend in this industry that whatever works early on gets a lot of resources put into incrementally improving it and making it cheap, such that competing technologies have to be _hugely_ better to have any chance of taking over.
That is (IMO) partly why:
- we still use hard drives,
- CPU's still use CMOS rather than one of the faster switching methods,
- the x86 architecture is still dominant,
- the UNIX model is the base of nearly all operating systems.
There may be potentially 'better' technologies than these out there, but there has been so much engineering and optimisation gone into these technologies that it is really hard for anything to compete.
The case of the Exponential PowerPC is an example of that - it used ECL rather than CMOS to get substantially higher clock speeds, but before it had really got up to speed, the incremental improvements in CMOS had passed made it look less attractive, and Exponential was dead.
I expect someone to reply to this and say how much better CMOS (or whatever) is better than anything else
Since everyone will be replacing their hard drive with rolls of scotch tape, I'll corner the market!
Fight Spammers!
Then, one day, someone realised that - hey! If you throw away the assumption that baud == bps, you can actually drive up speeds to 56Kb/s!
Then, as modems went up in speed, the same engineers moaned and groaned. The 56Kb/s limit was near, and without a total rewiring of the phone network, an act of Congress in the US (an act of God elsewhere in the world), and more money than anyone had, the 56K barrier would never be breached! Calamity!
Then, one day, another bright spark realised that if you had modems at the junctions, you could shove REALLY high-speeds down the wires without either Congress -or- God having to do anything. (Much to the relief of both.)
The Doomsday Crowd, defeated once more, lurked on the fringes. Until, one day, redemption! Hard Disks can't pass a certain density!
This, of course, is as bogus as all the other claims. If it's possible to read the past ten writes on a given sector, then you can you can increase the density of the disk by AT LEAST an order of magnitude. You just have to remember to read/write all ten layers at one time, and you're fine.
Then, of course, there's no rule which says you have to use 2-state logic. It's easy, but it's not mandatory. Magnetic fields can have any orientation and any strength. So long as the maximum strength isn't so high that you get bleeding, you're fine. Recognise 256 possible states (using any combination you like of orientation and strength), and you've "encoded" a single byte into a bit - a x8 gain in disk capacity.
Combine the two, and you've increased the capacity by over 80 times! This can be increased still further, by increasing your ability to scan over-written layers, and by increasing your ability to distinguish magnitudes and orientations. You have two degrees of freedom for rotating the magnetic field, which means that by doubling the ability to distinguish, you quadrouple the number of possibilities available.
The scientists may be correct about the density, but the density is NOT the only variable open to hard drive manufacturers. In the future, it may become one of the least significant, as others are explored.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Yet again. Densities may not increase forever, but when you can cram 40+ GB on a single platter, just add a couple of platters and make the drive a little thicker. Anyone remember full-height 3.5" drives? Maybe they'll make a brief comeback once density plateaus.
I'm much more concerned about two other relevant factors:
1: The I/O bottleneck inherent to IDE and SCSI interfaces. All this horsepower, and all this storage, and we can't transfer it fast enough.
2: In case nobody's noticed, tape drive technology has gotten faster, but it has not kept up anywhere near hard drives from a capacity standpoint. In a network server setting, this can be a real problem! The data sizes and drive sizes are growing, tape speeds have increased somewhat, but the network speeds are still mostly at 100 Mbps or slower, and the backup window times are shrinking quickly. That's a bigger problem. We need faster interfaces and bigger tapes - or cheaper jukeboxes.
- -Josh Turiel
-- Josh Turiel
"2. Do not eat iPod Shuffle."
If cheap+fast permanent storage arrives quickly, then Oracle, for one, will be in deep trouble.
Durable storage without moving parts could easily be three orders of magnitude faster than magnetic disk tech.
With permanent storage that fast, PostgreSQL 7.0 would perform on a par with, if not faster than, Oracle 8i. All the work Oracle has done to optimise around magnetic disks would be rendered worthless or worse-- imagine how annoying it could be for a newly hired developer to slog through all of that newly-obsolete disk "wizardry" just to fix a bug...
And more permanant than semiconductor memory.
I see that commodity retail core memory is running
about $0.75 a megabyte today, and commodity disk
about $8 a gigabyte. That is a factor of a 100.
The "limits" of both have been decried for decades
without much effect.
This is right in line with "processors will never break 100mhz" as they would require lead shielding to keep the RF interference for Ionizing all the air in you house.
When someone yells "Stop" or goes limp, or taps out, the fight is over.
Size is the only dead end in site for hard drives.
- Speed. The average hard drive is spinning at 7200 RPM nowadays. At this speed, there is an average latency of 4.6ms just to spin the track under the head. You can't do much about this except spin the disk faster. At 10000 and 15000 (thanks Seagate), you still have 3ms and 2ms, respectively. This is on top of any time needed to move the head itself. With most access times <8ms in the low end and <5ms in the high end, this ceiling isn't too far off. Sure you can spin the disk faster, but this gets expensive (money, energy, and heat).
- Size. I think the article addresses this quite nicely. If we hit the ceiling here, we can increase the surface area. But this again gets expensive in all ways and precludes a 100 GB laptop drive in a 1.5 in width and 1 watt power consumption. You know you want it.
- Reliability. To me, this is the biggest problem. Hard drives are the most relied upon moving part in a computer and yet are the first thing to go in most systems (followed closely by the power supply, who's death is usually hastened by a power hungry hard drive spinning up and down). RAID (or similar) can tackle this, but is expensive in all ways and requires the user's attention.
Finally, I won't argue that hard drives will meet their doom in 5 years, hell we don't even have a suitable replacement yet (only stuff in research). I just figure they will be a story that I will tell to my grand children.The thing which would be valuable to consumers would be a sharp increase in data throughput. It's true that disk drive capacity has grown faster than CPU speed over the past few decades -- but data transfer rate has not. The result? The CPU is data-starved, both by the bus and the swap speed.
According to a friend of mine who works in the industry, the leading limitation on density is seeking from track to track and remaining locked to the track. There is some new head technology coming down the pipe which should vastly improve hard drive densities. One of the most difficult things to do is servo the heads. This new technology should eliminate this limitation (sorry, I won't go into details).
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
Mmm hmm. And do you think anyone would have gotten around to that realization had someone not observed that the "baud == bps" approach would not work forever?
Right, but would anyone have bothered to do this had someone not pointed out that you couldn't get higher speeds using the conventional approach?
The moral of the story is that there is value to pointing out the limitations of current technology because that is what allows us to avoid wasting effort by developing new technologies to replace existing technologies that don't need replacing. Conversely, it helps to anticipate problems in existing technology before they start to limit progress, so that new technologies will be ready by the time those limits are reached. This is not "doomsaying", it is simply having a good understanding of current technology. You have to have a thorough understanding of existing technologies, including their limitations, before you can hope to improve on them.
-rpl
Wow, it really is Scientific American. Down with metric!
73GB, Ultra-160 SCSI (160Mbps), 10K RPM. About $1650 available almost anywhere (except in Seagate's online store. Go figure.). Quantum's got essentially the same drives now tho I didn't notice them for sale.
Do the math: Put, say, 7 of these drives in a $300 external enclosure and you've got over 400GB usable RAID-5 for < $12000! That's $0.03US / MB.
"When an elderly but distinguished scientist says something is impossible, he is almost surely wrong" - Arthur C. Clarke
I collect Scientific American, and one of the most fascinating aspects of my collection is the series of articles on why this or that technology won't work or has reached it's limits. The authors that SciAm gets to write it's articles usually fit the definition in Clarke's Law above, and they have invariably been wrong, usually quickly.
Two examples:
SciAm published an article in 1947 on why long range ballistic missiles wouldn't work, mostly based on the inability to make the guidance systems accuarate enough. About 5 years later we were deploying them.
They also published an article in the 1980s on why space-based lasers for strategic defense wouldn't work. I was working in that area at the time, and the problems they raised had already been solved, we just couldn't talk about it because it was classified.
Here's an approach for increasing magnetic storage capacity I haven't seen elsewhere: Current tape drives are high capacity but slow. They work just like ancient scrolls, unrolling and rolling up on a spool. Think instead like a codex (i.e. a modern book with pages). Have a stack of magnetic sheets arranged like the mess of catalogs at an auto parts place (spines down, pages held to +- 45 degrees of vertical by end holders). Use a static charge to fan out the leaves at the place you want to read, then slip in the read head from above. This gives you 3-D magnetic storage with fast (at least compared to tape) access time.
Daniel
Magnetic Drives have been predicted to fade out long ago. Ever since the late 80's with the optical drives that held ~2gigs, magnetic hd companies struggled against the beliefs of the physicist. Engineers developed newer ways of making magnetic media more efficient, while the optical theories were more effective in the long run, the drive for magnetic media was more prominent, as were the funds. Lets face it....how many people own magnetic media as opposed to optical media. In the hardware industry, if there is funding and money, there is progress. MORAL: Support the optical media foundations around you. They will give you terabytes to loose files instead of gigabytes :-)
Great post...I'd only add one thing to it; Rambus memory.
With all the $$$ Intel is dumping into it, they seem to have forgotten what you've outlined. So, what do we see? DDR memory that can match and pass Rambus throughput for about 1/2 or less of the cost.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
Science schmince! Quickly...I need more space to store MP3s!! NOW!
---
Sig Return: 204 No Content
I know a guy who has been working on nanotech for real products. Apparently the first realistic uses of the stuff is as thin films.. the soonest apparently may be a carbon tubule based lubricant for hard drives, which lets the head skate over the medium. Problems include toxicity of the stuff plus difficulty in quality control but it ain't impossible. Covered in the article? dunno, slashdotted.
1 lightyear=5.88 trillion miles
speed of light=700 million mph
1 fortnight=14 days (336 hours)
If you travel 1 lightyear in a fortnight, you will travel 5.88 trillion/336 = 17.5 billion mph: 25 times the speed of light.
Does anyone know of a site which graphically shows (pie graphs or whatnot) the dropping price of magnetic (or any other) storage?
/that/ much cheaper.
Think about how much the first CD-ROM drives cost. The first writers. Now, $300 gets you a rewriter from Best Buy:)
To show the price drop, you have to keep dropping the appropriate scale, from "thousands of dollars per byte" to "hundreds of dollars per megabyte" to "dollars per gigabyte." That, or deal with figures so far in the decimals that you have to count on your fingers to make sure it's really
I think there used to be one at hatless.com, but it seems to have slipped into 404dom.
Anyone know of a good replacement?
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
Here's your non-mechanical drive, jrs.
Will I retire or break 10K?
By the time we've met with the capacity of magneto-resistive drives, we'll be moving on to something else. As the article said, thin-film didn't last forever, who/what is saying that MR has to last forever?
There will be no storage shortage in the future. Who cares about the death of MR... bring on the next generation.
PS: Imagine how long a surface scan is going to take on one of these babies. Pack a three course meal, and a good book.
-- kwashiorkor --
Pure speculation gets you nowhere.
-- kwashiorkor --
Leaps in Logic
should not be confused with
Jumping to Conclusions.
It's so fscking lame of you Americans to keep flaming eachother. Just because you had a war for 186 years ago you don't have to be enemies.
/Rovfrukt (peace@america.now)
Sweden and Norway had a war about the same time, yet we don't hate eachother (that bad), we just make really bad not-so-fun histories about the stupidity of eachother.
>Do you NEED to be storing HDTV DVD's on your hard drive?
No, but I would like to record, oh, say the next Woodstock 20XX Weekend Marathon of 48 or so hours on my Tivo++ while I'm out of town at some work function.
After that, we'll be sure to think up other uses for 70+GB drives.
Hmmm, how about scaling those matchbox 340MB PCMCIA drives up to a few GB so that I can record a decent length (home or otherwise) video on one? Would that be nice or what? Forget DVDs, carry a couple of the videos(packaging and all) in your pocket. How about being able to backup, copy, and carry your whole MP3 collection processed at 256Kbps over to your friend's house for a party? Those matchbox drives are just barely on the threshold of usefulness today. Put 4-8-12-24 GB on them and suddenly they become very handy indeed.
Good judgement comes from experience, and experience comes from bad judgement.
- W. Wriston, former Citibank CEO
They don't, but then that wasn't a design requirement. Shooting down ballistic missiles in flight was. The way you stop back-seat nukes is by keeping really close tabs on who has the critical technologies (like isotope separation and making nice, symmetrical trigger implosions), and where the inventory is (fissionable materials, live bombs). Your other defense is that old standby, MAD. Any organization that uses nukes on the US knows that there won't be anything left of them after we are through.
For additional nightmares, consider that a liquefied natural gas tanker carries a nuclear bomb worth of energy. If you can figure out a way to make it go boom in a city harbor, it would be as bad as using a nuke. Or consider an truck bomb attack directed against a nuclear power plant. The containment buildings are pretty tough, but the control rooms aren't.
Daniel
Ok, don't mean to beat a dead horse, but WHY DO YOU NEED ALL THAT DAMN SPACE!!! Exactly how many mp3's can you listen to??? Do you NEED to have The Matrix in some insanely high res. ON your harddrive?!?!? Hell, if you love it that much go out and *gasp* BUY it!!! Then play it from the damn disc. And how many games do you need to have on your harddrive AT ONE TIME. Why not work more on improving access times for the drives, rather than size. "Yeah, but back when i had my 5 meg hard card, i didn't think i could fill that either" One does need a certain amount of space for some things, such as word processing/entertainment and whatnot. But really, let's think about this. Do you NEED or WANT for that matter a bigger word processing program??? Can't 70 gigs ON ONE DRIVE satisfy you?!?!? *and the rant begins* Hell, for the price (of Office) one would be better off going out and buying a SEPERATE word processor complete with monitor and mouse and ALL THE SAME F***ING FUNCTIONS as Office, except that LITTLE FUCKING OFFICE ASSISTANT NO YOU MOTHER F***ER, I AM NOT WRITING A F***ING LETTER JUST BECAUSE I WROTE THE F***ING WORD "DEAR" THEN HIT "ENTER" Plus, one can transform their info from a WP to a pc using utilities that come with the WP. HA! Anyhow, with the current permanent storage available (and on the horizon (i.e. DVD burners)) the need for insanely large hard discs is just dumb. It'll die one day and take lots of data with it. I think i am going to go lie down now.
Doesn't work. Huge discs require huge heads, which, together with larger tracks, increases seek times. And of course, there is simply no way anymore to get a consumer to buy anything that doesn't fit into his 3.5"-normed box.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
Now, take that thinking and apply it to hard drives. Instead of just buying a faster and bigger disk every two years or so, you'll start getting disks with new features that make using them better. Features like plush leather seats, rear data connector defroster, and a tiny little winshield wiper on the activlity LED. Those are features that would vastly improve the life of the hard drive user, but have been ignored in the past in the mindless quest for bigger and faster.
FYI, here are the general RAID levels:
Raid 0 - disk mirroring
Raid 1 - disk striping/no parity
Raid 4 - hair striping?...no wait, wrong list *hehe*
Raid 5 - disk striping with parity
Do you ever feel like you're diagonally parked in a parallel universe?
Worrying works!! 99% of all the stuff I worry about never happens
Reasons I'd love supermassive, small capacity drives. Imagine buying books, and having them all available on your palm pilot (every o'reilly, right there at your fingertips, with of course every RFC, the entire acm digital library and a few others). Imagine having every movie you own in your pocket, also on your palm pilot, with an adapter so you could watch any movie you already own while you're travelling. Every movie, every cd, uncompressed, full quality. No need for the distortion of mp3 when you have a terabyte in compactflash format :-) The reasons why there isn't any current applications like this, is because it isn't currently feasible. At home I have around 150gigs of storage on my network, most of which is in one RAID cluster. Useless? it seemed so but I keep on finding handy ways to use it. My latest project is storing all my live concert recordings (dat and cd) (yes they're legal), to .shn files so that I can easily spin off copies of them on demand. If you use your imagination, you'll realie that the number of people who keep legal video on their machines is small because it's not generally practical. Consumer-ized special-purpose computers such as the TiVo are changing this. A super-high capacity TiVo like device combined with broadband access and you could start selling movies legally, on-line.
----------------------------
Does anyone really believe this? This is what they've been saying about silicon based processors for years too... touting that we'd have to find an alternative real soon, be it copper or bio or whatever.
That said, harddrives, be they IDE or SCSI are still unbearably slow for the size of modern OSes (Be and QNX excluded). So, a replacement that has some speed behind it would be more than welcome!
"How much truth can advertising buy?" - iNsuRge - AK47
Of course, we know what happened after that -- they quit using visual light, and started using shorter-wavelength beams.
A friend of mine says "Is tape storage on the way out? It's not keeping up with disk storage!".
Seagate and HP just introduced a tape drive with 100gb (UNCOMPRESSED) capacity, and they say that they can take that same technology to 250gb native. These LTO drives do this by having oodles of tracks on a tape so that a stretch of tape may have hundreds of tracks in parallel, and by using new tape materials that allow them to make the tape thinner so they can pack more tape into a cartridge. People said linear tape was dead, that helical scan would always be faster and higher capacity, but it appears that conventional wisdom is foiled again...
I don't know what hard drives are going to be like five years from now, but I do know they're not going to stall, capacity-wise, due to some "inherent" limit. Too many smart people are looking for ways to bypass those limits, either by using some other technology altogether (hmm... photo-sensitive materials??) or by figuring some way around the "limit" using clever application of the underlying physics.
-E
Send mail here if you want to reach me.
I've been looking at the data sheets on some of the big enterprise-class storage systems. We're talking about boxes that have 5 to 15 drives, and attach via fibre channel loop to multiple servers that need to be backed up, and that have hundreds of tapes that they manage via robotics. Yes, I'm working on enhancing the Linux 'mtx' tape library control software to drive these things, though I'll never be able to personally see or test one :-}. There are some interesting challenges to handle with fibre-attached storage, specifically, the one of "who has the robotic arm now?!", but none that are unsolvable. I am confident that no matter how big hard drives get, we'll be able to back them up -- albeit for a price!
-E
Send mail here if you want to reach me.
The one that used to be up at hatless.com (I think that's where it was) also showed more information about how that info was obtained. Like what sizes of hard drive were available, what they cost for base model and for higher capacities. The SA chart is interesting in its speculation, but it's short on the details that actual end users would have to know.
Better than nothing, but I'm looking for a better one.
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
Tape drive manufacturers are raising capacities via a variety of methods. They are coming up with thinner tape materials so that they can cram more tape into a cartridge (I understand there will be a DDS5 that crams over 200 meters of tape into a tiny 4mm tape cartridge!). They are coming up with new heads that either store data more densely linearly, or that store data more densely vertically (i.e. put more tracks on a tape). They've also been playing with the speed at which data is recorded, and perhaps varying that to adjust to tape quality etc. There are also experiments ongoing with multiple heads and serpentine tapes, though I haven't heard that this is buying anything (easier to have a smaller cartridge and multiple simple drives rather than big complex cartridges and one complex drive). Having seen these guys do so many "impossible" things (they said that DDS4 was impossible!), I've given up on figuring out where it's all going to end, but I do know that traditional tape drives are nowhere near their limits as far as speed and capacity go.
-E
Send mail here if you want to reach me.
I am working on a FibreChannel-RAID adapter at my job. FibreChannel-RAID answers these questions pretty well..for now.
1. I/O Bottelneck - Our card does 190 MB per second with 23,000 I/Os per second in non-RAID mode (direct connect). Another good things to do is offload as much as you can to storage processors. This saves the main CPU. Relying on System DMA is a big part of what kills IDE performance. Both SCSI and FibreChannel adapters are DMA Busmasters meaning they can read/write to host memory on their own, without using host processor. Always use hardware RAID (adapter or external/cabinet based) instead of software based. Software based RAID kills processor.
2. Backup - Various forms of RAID can help here. You can configure things so that there are always at least two copies of your data. This doesn't help for real backup where people need things that were overwritten, like tax records from five years ago. Using RAID arrays of FibreChannel tapes speeds things up quite a bit.
As for network speeds...you are right about 100 Mbs being too slow. Heck 1 Gbps (or 128 MBps) are still too slow. That is why you use FC arrays that support multi-initiator. Here, multiple hosts are connected to a set of storage. In this model there is no server front-end to the storage share. No network latencies.
SAN technology is really just starting. Target mode systems (like EMC's storage cabinets) have great possibilities. Simple FC-Adapters can run in this mode as well. In a raw format, they can avoid the OS almost entirely, using it only for initialization and configuration. Backup can be done without any OS interaction.
-- soldack
These things are becoming more common each day. Data warehousing and SANs will create even more demand. Then you have video-on-demand servers that have full screen digital movies on tap. I am talking about full control video with all your VCR functions of stop, pause, fast forward, rewind, etc. stored remotely and sent to simple cable box. This kind of applications take up lots of space.
As for a data-starved CPU...not with IDE. IDE controllers use the host CPU for DMA so your CPU is quite busy. SCSI and FibreChannel adapters are busmasters but they are also faster. It is true that today's CPUs can push much harder than today's storage. It is also true that even a 66 Mhz, 64-bit bus is too slow. Interrupt sharing doesn't help either. That is why PCI is on its last legs. PCI-X will not last too long either with InfiniBand on the way.
-- soldack
Shit, John's average post is probably a +5 but here he's posting at +2 by default. I'm surprised it hasn't raised itself. Everyone enjoys hearing the man speak, his posts ought to be gold plated or something :-)