Slashdot Mirror
Ask Slashdot: Breaking the Computing Bottleneck?
MidKnight asks: :
"With CPU speeds venturing into the GHz, typical DRAM
access times falling towards 6ns, and network speeds trying
out the 100Mb waters, the major computing bottleneck
continues to be the hard drive. While other hardware
speeds grow in orders of magnitude, hard drives aren't
much faster than they were in the 70's. So, the question
is, when will mass storage speeds start catching up, and
what will the cause the revolution?"
Good point. I don't know why ppl don't start considering SCSI. Clearly, its a much better alternative. The only drawback is its price. Not so much the hard drives themselves, but the controllers. Adaptec in particular. They make good controllers, but their situation is very similar to Intel's: nice hardware, insane prices.
SCSI helps with the interface, but the real limit is how fast that ole platter spins multiplied by the data storage density. The only real answer is something electronic, like cheap nonvolatile RAM. That one may be a way off. Mechanical devices just aren't going to keep up.
The major bottleneck is rather the Mhz wall, and the memory wall: 286 didn't need caches, PII need them badly (and 2 levels ; remember the first Celerons?).
Today you could easily put all the data you'd use in RAM, so faster hard drives won't be of tremendous help for workstations or home machines. For server or for video they might be of some help, but not revolutionnary.
There was a point in time when a "workstation" was defined by the "Three M's".
- A Meg of RAM
- A Million IPS
- A Million pixels
'tis a shame that display resolutions haven't followed Moore's law.
Ditto disk seek times.
i heard about this ages ago
,green, etc...)
:)
the idea behind it was that a computer would use a crystal (diamand?) and lasers.
there would be multilasers that would actual write on the reflected light that would come from the crystal.
in theory there would be close to an infinit amount of space and you could write on each seperate color (red, blue
the only problem that i can remember was that it was the same idea as a cd worm (write once read many) not the best for your typical pc user
great for mass storage tho
ill see if i can find a link about it
or if someone else finds one please post
Well, i dont know if i could be considered an average computer user ( after all i read slashdot), but i myself have got 3 harddisks in my
pc, and that would have been 4 if my linux HD hadn't died on me...and i know other ppl who have got several scsi HD's
No, they're not fast already - and they are the problem. The problem, as you point out, is not bus bandwidth - it is latency. A 500Mhz CPU has a latency of 2ns, main memory is like 10ns, a 10K cheetah drive - 9ms. That's a MILLION times slower than main memory. From the point of view of the processor, drives don't spin.
0 62201 and see how many drives it takes to keep an 8-Way Xeon server busy - 350 10k drives! Oh and btw, that's doing RAID 0 - if you want some fault tolerance, and go to the next highest performing RAID level, RAID 1, plan on doubling that.
Using multiple drives and RAID can help alleviate the problem, if your CPU has a million other things to do while it's waiting for that data it just requested - but few people realize how MANY drives you need. Take a look at http://www.tpc.org/new_result/c-result1.idc?id=99
Silicon and other IC devices are getting exponentially faster, but unfortunately mechanical devices don't follow Moore's law.
This is a major performance issue and one that gets worse with time, because drive density does increase exponentially and people buy less of them, when they really need more.
-Jay Urbanski
strops@well.com
Crap * 20 is still crap. :P
As long as we have mechanical storage, will run into a performance.. the future lies in solutions like solid state drives where its simply gigs worth of DRAM.. how's that for bus filling performance? :)
.05ms =]
BTW, seek times on those kinds of storage systems is around
The answer is buy more RAM.
My recollection about the sound-driven fridges was that they used absurd amounts of power.... something on the order of hundreds of KW/hours. A little freon is a lot more environmentally safe than a bunch of those things draining the power grid.
The next "big thing" in large scale storage is MEMORY. With prices falling (generally) and sizes increasing, I can see a not-too-distant future where "primary store" implies main memory. All that is needed is a reliable and inexpensive means of keeping the memory non-volatile.
Nonetheless backups would be still be very important, and hard media backup systems would have to evolve to meet the needs of this kind of system. Perhaps something like a removeable hard drive which is constantly "on" doing incremental backups of the main store as the user changes files in main store...
EXACTLY! The problem isn't that disks are evolving slower, but that we aren't bothering to use the exiting design strategies on hard disks. What did we do when we wanted faster processor access? One thing we did was widen the data path. Same on the buss. Same for memory. And that's exactly what RAID striping is.
As an analogy, think of the current PC model as the same as a 1-bit machine becaused you only have one drive to send/recieve from at once. So you add 7 more Hard drives in a RAID 5 configuration and you've got an 8-bit machine because you can send a full 8-bit "word" at once by sending in paralell. Now your limitation is the buss again.
So my question - when do we start seeing single hard drives designed this way - with 8 seperate read/write mechanisms and RAID striping internally so you get the advantage of RAID without the hassle of setting it up? Since the technology exists to do that by simply building the RAID controller into the disk I'd say it'll happen as soon as people really care about hard disk performance. Right now it's just not as important to people as you say except in extreme situations (where you see RAID striping already).
hitchhiker
I have an Ultra/66 hard drive and controller.. Certainly it is still a bottleneck, but not really as bad as it used to be.
Linux boots in 15 seconds, X loads in 5 to 10, windows boots in 30, even Half-Life loads in just a couple seconds. With my old hard drive it took almost 20.
You don't have to get SCSI or RAID to get a noticeable performance difference.
Jim
Most mb's today have pci 2.1 but 32bit. If all pci cards installed are 2.1 then the pci bus is run at 66MHz else it's drops back down to 33MHz.
equal to the speed of agp 1x.
Hi there :) ,the idea of nanotech has pioneered these designs and keeping a close watch on these intelligent and creative advancements are the best means of knowing what's coming our way - i guess all human intelligence is geared towards the potential of sentience .
The advancement of biotech will soon bring about the design of new and creative means of storing data in a bioengineered device - e.g maybe plasma,crystals or devices which store energy and viceversa - these application will bring about a proliferation of smart technology
Do you know how they keep this "protein-drive" from denaturing? Besides issues with reliability, another thing that makes me skeptical is the yield they'll be able to hit
and the cost/performance ratio. With access
time in the order of ms I don't think this device will survive the hostile storage device market.
I believe researchers in the databases community have been studying algorithms for faster disk access for a few decades. A CS professor at Brown (his name slips my mind) once said that the entire history of the database industry could be seen an intensive study of the disk head.
In any case, algorithms alone simply cannot provide the answer. Even an optimal caching strategy would still leave a lot of idle CPU cycles while the processor is waiting for disk.
~k.lee
It depends on what you are doing. If you are building a server (especially a video server), this won't work.
If you're just using a computer at home, why not ? In 1993, I bought 8 MB RAM for $400. Today I could buy about 1 GB, and install a pretty much a full-featured Linux distribution in RAM. More and more space on hard disk is used less and less often ; it happened to me to discover after 6 month a cdrom image left in some directory, or a 200 MB trace file.
Honestly I don't see the hard disk speed as a problem 99.9% of the time on my computer. Starting netscape or xemacs, or compiling is slow the first time, but this is not a problem in practice.
There is an article in the march 1999 popular science about using a protein from bacteria in plastic cubes and using lasers to change it's "photocycle" states. It claims 10 gig + per 1cm x 1cm x 3cm cube. The research is being done by Robert Birge at Syracuse University's W.M. Keck Center for Molecular Electronics.
Hi!
According to me, the important point is not what the technology should be.
The ONLY important thing is to know what you want to do with technology.
Think of it and be happy.
If you want to throw ram at the problem, you should add to the computer's ram, instead of puting it on the hard drive. I am guessing that a cache in the computer's ram is more efficient than a cache all the way on the other side of the hard drive cable, not to mention that it could be used for other things.
To repeat it again: It's SLOW.
It's a lot easier to use static ram and a battery backup.
And NO, you would NOT be running nearly 100 times faster of flash. I've been working on an embedded system with Linux in flash for a while now, and it's certainly not that much faster. Actually I'm not sure if it's much faster at all.
And NO, flash can't accomodate the size of next generation machines today, unless you put a hell of a lot of flash chips in your machine. Intels largest flash units are 8MB at the time. They might get to 16MB soon.
What about optically assisted winchester technology? From the interview with IBM that I saw in maximumpc, the IBM guy said that it wasn't all that it was cracked up to be, but then again, it was a competitor's technology. A technology to pay attention to though.
Don't hold your breath, they've been working on this stuff for about 20 years now, and has kept claiming it's right around the corner....
In theory, solid state should always wind up cheaper than something with moving parts, such as a hard drive. I do hope this eventually proves the case.
I remember that "Floptical" drive; 20M, wasn't it? The optical sync was required because it was a floppy disk, with the head in contact with the medium, and relationships would shift. Hard drives at that time had exceeded that storage ratio; they used magnetic sync bits embedded in certain places, and since the platters were solid and the heads flying, they only needed to readjust occasionally. (Every 10 minutes you'd hear that idle drive do some seeking.)
The problem is that the DRAM guys have been optimising for area/density at the expense of performance (after all that's what their customers have wanted - more ram) so speeds remained relatively constant.
Packaging is another big problem - getting data on and off chip costs - in pins and power - it's hard to scale data transfer rates in the GB/sec range at the same rate as the silicon in the on-chip processor.
BTW we are starting to see changes that address some of these issues by using split transaction memory buses and multiple banks of dram with overlapped access times (RamBus for all it's faults has agressivly moved in this direction) - but to use these sorts of rams you need a CPU that is capable of issuing many concurrent memory transfers (multiple on-chip CPUs or multi-threaded CPUs start to look awefully interesting at about this point)
In order to use commodity drams we CPU designers have been forced to push more and more smarts into silicon - an emourmous amount of a CPU's die is spent trying to soften the 'memory wall' - in caches, speculative access hardware, OOO processing to keep stuff going after a cache miss etc etc.
Finally some numbers to think about - a 1GHz system wunning on a Pc100 bus with an 8 clock access time takes 8+1+1+1=11 clocks 110nS to read a cache line - that's the time it can execute 110 instructions in - if it misses in it's caches 1% of the time it's going to spend more than 1/2 it's time waiting for main memory - it is equivalent to a 99/(110+99)*1GHz = 470MHz CPU that never misses. A 50MHz cPU on the same bus that misses 1% of the time only wastes 55 clocks and is equivalent to a 99/(55+99)*500 = 320MHz missless CPU - you doubled the clock rate but the performance only went up 45% - purely due to memory system effects. Suppose you kept the clock rate the same and reduced the the miss rate to 0.1%? you get a 470MHz equivalent cpu.
But then which of you gamers would buy the 1GHz CPU and which would buy the 500MHz CPU with big caches? Marketting guys hate this sort of multivalued analysis - they can't say "we've got the best chip" - it's much easier to sell "we have the chip with the higher clock rate". Again this leaves us CPU arhitect types in a bind - we know that the 500MHz part will scale with process over time much better than the 1GHz one and getting the CPU to 500MHz is soooo much easier than 1GHz - but that's not what sells the chips with the big margins
Now I've been simplifying here - CPU design is has many many more sides to it than just the memory vs clock issue I've outlined here (for example a big issue is 'can we test a really complex design and still meet our tapeout goals') - but hopefully you get the point
read this for fun: http://reality.sgi.com/mccalpin/papers/bandwidth/b andwidth.html
Put a little thought into what the real problems are. Programs getting bigger -> Systems get more RAM -> Processor Cache become more worthless... RAM speeds are too slow.. With multitasking workstation (and single function servers) disk access is more predictable, and disk caching is effective there, but for RAM it isn't..
Thanks the real problem.
Score=??
-toppk
There is at least one shareware/freeware ramdrive program for Dos/Win9x. I think I've also seen one for NT ($40US or so, IIRC.) Check shareware.com.
Forcing electrons through oxide barriers creates mechanical stress on the device and will eventually cause it to fail. Most flash rom chips will wear out after a million or so rewrites. Magnetic spins can be changed an infinite number of times without affecting device reliability. For frequently modified data, magnetic storage is more reliable than EEPROM.
I agree that modern HD quality is pathetic. I have an 800MB and a 1GB Maxtor drive, neither have any bad sectors. My brother, however, has to buy a new drive every six months -- they keep dying for various reasons (bad sectors in important places, controller failures). Each drive was of a different brand, too! He and I have recently moved to SCSI for increased reliability and so far, this has been true.
In a German Unix magazine, tough luck 8) ... an interesting approach is to use a different drive architecture. More a logical view instead of pressing files into a heads / sector / cylinder scheme. Some simulations look pretty promising but, as usual, I'd not expect to see this outside of high end servers soon.
Gee, I use a 10baseT network at work (which I am so lucky to run programs off of along with the other fifty to a hundred people on my network segment :), and a 56K modem at home. I've got a fast HD and a fast CPU, but I still wait for network connections.
Sure, xDSL and cable modems are becoming reality, and many businesses use 100Mbps ethernet, but neither is "the standard". I would not be surprised if there are more people with PC100 RAM and 7200+RPM hard drives than with broadband internet access or fast local networks.
In terms of latency, disk drives are an order of magnitude worse than memory, but guess what? Networks are often an order of magnitude worse than disk. Especially modems (150ms is the best? That sucks)
Right now, one side of each HD platter is using only one head. For faster speed, use more heads.
Place another head on the other side of the platter. Restrict the motion of one head to outter half of the platter and the other head to the inner half of the platter.
If you had three heads per platter, you would chop the tracks into three sections. Head#1 access only tracks 1-33. Head#2 access tracks 34-66. Head#3 access tracks 67-100.
--
st2
I receantly mooved my computer from a single 3.2 Gig IDE disk to a 9.1 Gig SCSI. I couldent be more pleased with the performance increase.
With the IDE, every time that the disk was accessed, it took processor time. with SCSI this isn't the case (I play MP3s 24/7 and when the disk was accessed it would stop for a second) The real advantage with SCSI is the CPU usage in transfers. with my SCSI I get 12Meg/Sec sustained with 1-2% CPU usage. with my IDE, I got maby 4 Meg/Sec sustained with 20-40% CPU usage. now I only have one device on the SCSI chain, but I can still tell a big diffrence between SCSI and IDE
Modern operating systems have made ramdisks unnecessary. If block caching is implemented correctly odds are that you're reading most of the data from memory, not the disk platter. This is true of Linux with it's very aggressive disk buffering. Windows NT also caches disk blocks.
http://www.anandtech. com/html/review_display.cfm?document=913
CDC, or I should update that, Seagate. I've got a CDC that survived over a decade of use before it was retired when the system was retired.
Between the reliability I've seen with Seagate and IBM, I don't see a reason to chose any other drive manufacturers. Long term reliability in a constant on environment is what I look for.
Anand tried the card, used canned tests on it, and gave it a lukewarm review because it seemed to make little difference on his benchmarks.
Basically, he found that it provides around 20% bandwidth improvement when striping sequential data. As this is at variance with the results reported by readers of Usenet's rec.video.desktop newsgroup, I'm left thinking that he failed to configure the Promise card or the drives or the drivers.
The way to get maximum throughput out of this card is to use one IDE drive per controller, which guarantees that the controller and IDE bandwidth are always available for the drive. I'm left wondering whether Anand didn't put two drives on one controller.
I was under the impression that RAID 1 _did_ confer a performance benefit - on reading only, though. This would be from the instruction manual for my DPT controller, which I have yet to use after purchasing it 1+ yrs ago..
How about having just a chunk of memory as a harddisk? For the computer it would look just like any harddisk, but inside it would contain a few gigs of memory (cheepo :P).
When you turn on the power, it would load all the data from the harddisk into the memory. When power off, it'd write it back. At other times it would just use the memory. Boots would be really slow, and it would be still pretty expensive, but fast and compatible.
Unfortunately, they seem unable to comprehend that opening up their code base would allow for vast manpower additions (they currently have very few folks for the amount of work)...
The hardware's performance has been quite good. I haven't played with it since the 8-processor upgrade, but things to seem to scale linearly as you'd expect. It's quite nice.
Jason, ejr@cs.berkeley.edu (who has an account on the Tera at SDSC)
Ethernet might be 100Mbps, but I'm stuck with my (theoretical maximum) 64kpbs line. Unless I want to buy a T1 or I$DN, that is. Of course, since I don't have ~$500/month there's no end in sight for my bottleneck. I wonder if they'll let me string some fiber to my ISP...it's only a mile down the road! Hmmm....i'll have to check that out.
And multiple processors on one chip is another variation of multithreading. The Hydra project at Stanford is simulating exactly that situation. It's pretty much programmed as a multi-threaded architecture, as far as I could tell from the presentation.
BTW, it's quite possible simply to have one register file per thread... The Tera does this, so it doesn't have the context switching overhead you fear.
Cheap hardware support for threads (not POSIX, which really can't be implemented cheaply) and synchronization (full/empty bits on every word of memory are beautiful) will encourage massive use of threads. If CPUs support many suspended threads, DB servers can use them for tolerating disk latency, too. Yes, it's a really long latency, but CPU-level context switching can get rid of the OS-level context switching overhead for responses.
No, MTAs aren't currently built for tolerating disk-sized latencies, but there's no reason why cheap hardware thread support can't at least help with them.
Jason
wouldn't that decrease latency?
Hard drives are using more or less the same technology they have been for the last 25 years (IDE, EIDE, SCSI, MFM, yaddayaddayadda). It's all just heads moving around on a spinning magnetic disk. They are slow and clunky and create too much heat (some of the faster ones) and use up far too much energy. Other than fans and removable media (CDs, Zip disks) they are the only moving parts in our machines and we use them all the time. I'm not a storage expert, but I'm sure the industry can come up with something better. Whether it's crystals or something else, we desperately need faster mass storage. SCSI is FAR from affordable and my IDE drive uses up so much time and processor power loading things up. 80% of my boot up is just waiting for the hard drive. Moving parts are a no-no. Imagine how much longer laptop batteries would last too.
Internet access is also far behind in most areas. Even where there is good support for things like ASDL, Cable and whatnot, it is far too expensive. I think the next 3 years will see the elimination of modems and a complete change in the way we access the internet. The war is already being waged between companies and technologies. The modem i'm using now is just a slightly faster version of the 1200 baud POS that I started telecommunicating with 7 years ago. As the internet becomes universal, we will need cheaper and better access for everyone.
Are you really serious? Do you have *any* idea just how much of a performance problem uncached memory accesses are?
More spindles, more arms, more heads.
Which theory is that ?
I think a lot of it is luck. Sometimes you get
lucky and the cheap drive lasts over 10 years.
Sometimes the expensive one dies in less than
5 years. Whatever you have, back up data as
much as possible and plan on buying a new drive
every 5-10 years. After 5 years, your drive will
look pretty slow and dinky compared to new ones
anyway.
Cheap SCSI alternatives are flooding the secondary channels, but SCSI means a bigger pipe, not a faster drive. For that, motor speed comes into play. 5400 RPM drives are too slow for most of todays standards, at least in audio/video, but I've found that, say, two 9.1 GB, 7200 RPM SCSI drives from compgeeks.com on a Software RAID 0 array, can seriously knock down access times, especially in programs with huge page files like Photoshop, or SoftImage... without breaking your wallet.
The curve you've described has disk seek times halving about every 4 years, which, while it isn't as fast as Moore's Law, isn't _that_ bad.
:)
Which means, at that rate, disk seek times will reach that of today's RAM (say 5ns) in 79.72 years.
On the other hand, if a miracle occurs and moore's law holds for that entire time, the processors will be approx. 10^16 times more powerful.
Massive striping across different controllers coupled with applications that can grab a bunch of memory and utilize it are the way to go for now.
At work, I work on a Sun server running Slowaris/Sybase.. It has 12 390ish mhz processors and around 12 gigs of memory. (Sybase only sees 4G of it, so there are OS work-arounds to allow it all to be used.) For typical use, disk speed is not a problem for the beast.
Of course, this is for OLTP usage. Throwing memory/striping at a DSS box would probably be less effective (Although it ALWAYS helps!)
I am not that old (24) and I remember when hd's for :^)
home pc's were to expensive for most people (several
thousand dollars at least for 10M.) Most people
just used floppy drives -- two 360k 5.25" drives
if you were lucky. I was young and poor so I used
cassette tapes. This was only 10-15 years ago.
A long time ago in computer terms but not that long
in real terms. I am sure most people older than me
can remember when things were even worse (but we were
all thankfull
How about this for a solution until we can get all-solid-state memory hard drives: take a bunch (10-20?) of those really small hard drives that were on /. a few days ago, tie them up into a RAID (0,1,5,?,...) array (maybe even make it user-configurable) and package it as one smokin' HD? Oh, and add a couple of MB's of buffer too. How fast would a 10 drive RAID 0 setup be? That oughta be fast enough for a year or two...
Not a final solution, I know, but maybe a good start?
Mike
I agree that the hard disk access for virtual memory is pathetically slow, I really don't beleive that this is truly a computing bottleneck. System designers have accounted for this slow access and subsequently increased system RAM to offset the effects. With proper system tuning swaps to HD should be relatively rare(you may have to load up a system with 4GB+ main memory). I believe that the performance eater is main memory. With billions of accesses guaranteed(sp?) to hit main memory, any reduction in the Time to First Access would tremendously help performance.
Look into buying a Tekram scsi controller. They are about 1/2 the price of adaptec and based on the symbios logic chipset. Its supported in just about every OS. My DC390F is great.
Use a bigass ramdisk for the system files, like binrays, shared librarys, config files, temp files, and the harddrive to store ordinary data. :)
an ramdisk of 512 MB or less would be enough for
me to fit my entire debian system inside. (system files that is, not manual pages and email
i'm a dork, but seriously, who stalks the slashdot forum waiting for new articles to be posted?
:P
Adaptec? Nice hardware? Put your hands in the air and step away from the pipe ...
Jeeezus Keeerist on a pogo stick. Adaptec. I do not use Adaptec and neither should you. Here are some reasons:
1. The GPL/hairy anarchist reason: Adaptec has been one of the worst companies for releasing specs -- even worse than refusing to release, they release, then stop, then insist that they never did, then refuse to provide updated data when their bioses screw everything up, and so on. They will not release specs. The only reason that they have started again is that Dell pressured them into it so that their servers would be able to run Linux.
2. The RAID freak reason: Adaptecs choke under load. If it says 20MB/s with DPT or Mylex (hell, Tekram and Buslogic and NCR, really), it will do that, unless it is Adaptec, in which case it may or may not, depending on board and phase of the moon. Adaptec stuff has bad problems with noise. Look at different revisions of controllers -- major trace differences. And Adaptecs, more so than almost anyone out there, are incredibly sensitive to cable issues. RAID5 for a database runs poorly if the controller has notched the speed back gradually (without letting you know) to 10MB/s because it is getting stuff it cannot recognize. RAID also needs cache, and Adaptec stuff needs huge amounts of cache. That isn't because it is so studly kids -- it is because it uses very poor algos and handles the caching poorly. Bottom line, I have seen Buslogic cards with software RAID on slow disks push more data than expensive Adaptec cards with fast disks, and I have seen DPT and Mylex cards with minimal cache push 4+x the volume with half the RAM (or less).
3. Cheap bastard reason: Adaptec is expensive. You frequently get better performance from a Buslogic or Tekram card.
4. Bitter sysadmin reason (that's me): Adaptec changes their microcode all the goddamned time. And they do a piss-poor job of testing the code ahead of time. Notice how often Adaptec releases code updates. Now look at DPT and Mylex. Notice a difference? I will not spec out Adaptec stuff at this point -- I like going home at night, not applying downlevel code patches to try to figure out why the controllers are refusing to see partitions larger than 12GB (that would be ALL OF THEM!!!) on the mail server all of a sudden.
Look, I know that DPT and Mylex aren't cheap, but they really, really work nicely, the physical quality isn't crappy -- it is very nice, the microcode works and the caching is very neat. if you don't the speed, Buslogic is good, Tekram is fine. I have been using DPT for year and Buslogic occasionally and have recommended them to friends without one bad experience.
SCSI drives also have a lot of cache -- the newer IBMs have 8MB of cache. This really helps keep the pipes full.
I might agree with you on this point, except that solid state drives could start advancing along a pattern that follows Moore's Law. This would be a good thing.
SCSI hard drives are quite a bit more expensive,
they command anywhere from a 40-100% price increase for the same capacitiy and rotational speed.
Most people only have 1 hard drive and would never see the performance improvement from SCSI. Most of SCSI's benefits are when you have more than 1-2 drives and lots of simultaneous traffic which the average PC user simply doesn't do.
I work on mainframes, which have much greater data moving ability than desktop PC's. The drives they use are basically the same however as PC drives, but the BUS interface they use is much better. Until the PC's BUS is updated you won't get the performance you are looking for from PC's. Ideally the bus would run at the same speed as the CPU so you keep a 1:1 ratio for timing. Right now the CPU is twiddling it's thumbs waiting for the I/O to occur.
I saw it operating about 3 years ago on the Discovery channel - the system uses a galium arsenide (spelling?) cube about an inch square that contained gigabytes of storage capacity. The primary and reference lasers (a row of 100 beams each) did the writing, and the primary did the reading - and it was rewritable. They even played a segment of the host show from it. The entire setup took up a whole table-top - so I'm sure miniaturation is their primary concern now. This is assuming some asshole corporation isn't tying up the technology in court with copyright lawsuits (a common ploy to slow down competitors). In anycase - we have the technology. Today. Whether we see it on the market is up to the analysts and whether they think it's economically feasable. Chances are, it'll go the way of the sound-driven refrigeration unit (no pump, no freon - and it works). Our kids will probably see the stuff trickle to market when they're having kids unless someone is willing to take a chance and do something different now... It sickens me to see how much radically different technology is available but being held back by an industry that doesn't like change.
Only if the CPU has no way to tolerate the latency involved. Go read about multithreaded architectures. The principal is that the CPU should never wait for a high-latency event to complete before continuing. Instead, it just switches to a new thread. Normally these threads are much lighter-weight than the POSIX variety, but they're of similar flavor. I do not know a good, on-line summary of the various types of multithreaded CPUs. If you're interested, look around U. Wisc., Stanford, and the ACM Digital Library for phrases like `multithreaded', `SMT', etc. You could also go to a bookstore and flip through Culler and Singh's ``Parallel Computer Architecture'' book for some now-slightly-dated info.
One of these machines does exist. It's from the Tera Computer Company. It has no data cache, and it's pretty fast for scientific-style jobs. IIRC, we were getting the equivalent of an 8-way UltraSparc-I on only one or two processors (argh, I really can't remember) on a list ranking benchmark. They're only considering the latency to memory locations (around 170 cycles, which is between 9 and 1.5 insn word opportunaties for a stream), but the general idea should hold for any high-latency event. The engineering for extremely high latencies probably gets trickier, though.
Expect multithreaded architectures in the higher end workstations within the next 3-4 years (at the longest). A certain up-coming architecture has the ability to grow in this direction pretty easily (I believe, haven't thought about it too hard), but one of the folks involved in the design was quite down on multithreading a few months ago.
Jason
I have quite a significant chunk of RAM, but I am looking for something similar to Ramdrive.sys. However, this file gives you a maximum of 32MB, which is pretty pityful if u wanna run something like Quake, or quake2 at full speed, or capture a large amount of data over a long time without wearing your HD out. Any urls people? ????
Onei
I work on a project at Michigan State univeristy on Diamond Latice growth. In conjunction with the physics program, we have been attempting to grow carbon nano tubes. There has been success in the growth of the tubes, and the bucky balls, but moving the bucky ball from one end to the other of the tube has been a pain in the ass. There needs to be the creation of a EMF to "un-stick" the bucky ball. This would elimate volitle storage (RAM) and terabites of storage would be avaible on a single chip. It is a strong possiblity and there is going to be a conference in September/October at MSU that is the correlation of the research across the world.
Exciting stuff! More storage/less space.
You can think of all the storage space in your computer as being arranged in a heirarchy from fastest to slowest: CPU registers, L1 cache, L2 cache, RAM, disk, nonlocal media (network, tapes sitting on the shelf, etc.).
The CPU works directly on registers. When it needs something else, it sends a request to the next level, L1 cache. If it's not there, that level requests the data from the next level, and so on. The first four levels of the memory heirarchy are fairly close to each other in performance:
- Registers are zero-latency by definition.
- L1 cache is very nearly zero-latency.
- L2 cache requires the CPU to idle for a couple of clocks while waiting for the data.
- RAM makes the CPU wait for a few dozen cycles in the best case, on a fast bus.
And then there's the disk. Disk latencies are measured in milliseconds. With a modern 500-MHz CPU, a millisecond means 500,000 cycles. When the CPU needs data from a page of virtual memory that's on disk, it has to wait millions of cycles for that data to be loaded into RAM so it can then fetch the data.That's five orders of magnitude worse than RAM (by contrast, RAM is one order of magnitude worse than L2 cache). This is why virtual memory thrashing is so disastrous for performance.
RAM-to-disk is the biggest critical gap in the memory heirarchy. SCSI hardly does anything to close the gap. A 7-ms latency is not much better than a 9-ms latency-- it's still millions of cycles-- and SCSI's higher transfer rate doesn't mean a thing for virtual memory since pages are pretty small to begin with.
OTOH, applications that are limited by transfer rate rather than latency may benefit from SCSI, but transfer rate has never been the big problem with hard drives.
~k.lee
Researchers at the Hitachi Cambridge(UK) Lab at Cambridge University are developing a new memory device named 'PLEDM' - Phase-state Low Electron (hole)-number Drive Memory, which they believe to be 'a promising candidate for the multi-Gbit memory chip which is scheduled to become available early in the next century'. In principle, it is possible to make a fast non-volatile PLEDM cell by modifying the barrier-structure in the channel, and they are confident that many of the present day data storage devices (eg: computer hard disk drives) could be replaced by PLEDM in the future.
The Hitachi press release
A minor correction on your last point: you are correct in saying that the interface speed isn't that much different, but three factors that scsi is still far superior to IDE in are: expandability to more than 2 drives/channel, simulataneous data transfer to multiple drives on the same channel, and CPU usage.
I'm seeing a lot of comments to the effect of drives not being all that slow. Let me provide an example to demonstrate just how slow they are.
... go on vacation for a while... probably get a new job... new car...
Let's say you have a large forklift which is capable of carrying a few hundred boxes of paper from your archives. You're the forklift operator, and I'm the CPU.
Ok forklift, go get me box of paper #421769, which is in our on-site storage facility downstairs.
You drive off, and are back about two minutes later with the paper. That's the speed of memory.
Now, forklift, I'd like box of paper #090210, which is in our long-term storage facility.
You drive off. I go home. I have dinner, come back the next day, do stuff, go home
FOUR YEARS later, you come back. That's the speed of hard disk.
No matter how big that forklift (bus bandwidth) is, the latency difference between two minutes and four years is a pretty huge one.
--
Well either that or place a few thousand heads per platter in a drive. For every head you place on a drive per platter, you increase access times, transfer rates, etc. At least logically this makes sense,
:)
:
I can imagine one really big head the size of the platter itself with 1 mini reader/writter per track, hell you wouldn't have the spin the thing
Of course, that might cost too much
--
The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
Virtually any hard drive on the market today can fill a 10baseT ethernet continuously, even with noncontiguous reads/writes. As for 100BaseT, you can fill that with a RAID setup. For example, cdrom.com fills its 100BaseT ethernet constantly, so the network is the bottleneck there, not the hard drives.
As for latency, hard drives are behind memory and CPUs, sure, but they're still ahead of networks. Most large networks have much worse than 9ms latency.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Posted by Largo_3:
I disagree, compare the price of a decent tape drive to the average harddrive size, the price is almost on par with each other, hence Tape media is not a useful option for people anymore, unlike years ago when everyone wanted a 'colorado jumbo 250'
it only takes time...
Posted by Largo_3:
I agree. less moving parts also means === SMALLER
Your hard drives do that right now. Data doesn't move to your drive in serial, it uses a parallel style. That's why you have that wide ribbon cable that can only be so long. Hard drives can usually write more than one thing at a time also. Much of the technology they use is a 'trade secret' as I recall though. You may be able to find some more info on how it works at a college research site or something. Trust me, the hd makers are pushing as much as they can. Raid does help alot, but the problem is still latency. Cache helps that some... We'll have to see what happens.
Is faster yes, the problem is that as long as we use a spinning platter to store data with a physical head moving around it that imposes some limitations. The platter spins at a finite speed and the head moves again at a finite speed. We need some sort of solid state mass storage, the problem is that is *MUCH* more expensive.
--Zach
Erlang Developer and podcaster
Yeah, a new hardware architecture can help lots. At Cambridge Computer Labs (part of the University) they have a prototype computer built on the Desk Area Network. The premis here was that the main system bus was replaced with an ATM switching fabric, so any device could talk directly to any other device, and these conversations could go on in parallel.
A good example is drawing video to the screen from a camera. The data would come off the capture device, and talk straight to the video card. No going thru main memory. Or network packets could go straight into the processor cache.
I saw it running 6 odd real time video displays (some TV, some local camera) at a very good framerate, yet the processor was only an Arm 6.
So, to cope with modern loads (especially multimedia data types) a good architecture helps a lot. You can download the PhD thesis describing the DAN architecture from the cambridge FTP stie somewhere. Can't remember the URL off the top of my head though.
-- Michael
PS: Little factlet - the DAN prototype site right next to the (in)famous coffee pot that was the subject of one of the first web cams :-)
What an interesting work of fiction.
Stating on Slashdot that I like cheese since 1997.
It is the "tube" between our CPU's and out memory....
Yea Lisp, scheme, Turing, Backus....
I'm done
---------------------------------- I like fig newtons...they're tasty
Silly me, I always thought the bottle neck was the user...
One of the more insigtful comments is that we need to go back to Knuth's Fundamental Algorithms and see how he handles the memory hierarchy. In this case, tape drives. Tape drives in those days had reasonable bandwidth (for the time) and miserable latency. Sounds like hard drives now. So we need to reread the old algorithms and do a little translation:
Tape drives -> disk drives
Disk drives -> DRAM main memory
Core (RAM) memory -> 2nd level cache
Cache -> 1st level cache
This isn't simple, of course, because things have changed, amongst other things there is no detailled control over what is in which cache level, but the gist of it is that we need to drop the assumption that accessing random addresses in RAM is just as fast as accessing RAM in such a way that cacheing and readahead are possible. This is still a universal assumption in algorithms textbooks.
Similarly, we need to drop the idea that we can get to any part of the disk within a reasonable time. If we treat modern disks like a tape drive with really good mechanics for fast-forward and rewind, then we are closer to the reality, where you can transfer over a megabyte in the time it takes you to read a random bit from the other end of the disk, (about 10ms).
This implies our 'hard disks' should be RAM disks, and the current hard disks should be relegated to some sort of backup role.
The mainframe I used at Cambridge (Phoenix, a heavily hacked IBM 370 - this is only 10 years ago!) would move your files out to tape if you didn't access them for a few weeks. Next time you accessed them it took a few minutes for the tape robot to find your tape and put it back on the disks.
Hard disks are about 3 cents per megabyte.
RAM is about $1 per megabyte And according to rumour, IBM is about to speed up the capacity race in hard disks, while memory is about to hit the quantum effects wall.
RAM isn't a real alternative to disks and if you want to use RAM to cache disks, then you need to be very careful that your access patterns are such as to let the cache have an effect.
Research should be into how we make a large amount of RAM and a humongous amount of disk space work as fast as if the disk were RAM. Perhaos ReiserFS will help Linux with this.
Incidentally, DAT tapes aren't much cheaper than hard disk space these days, though they are more portable, which is important (if you back up by copying to different disks within the same LAN then you have a problem if lightning strikes your ethernet).
Actually, I think hard disk capacities are pulling away from memory capacities.
After all, it's almost trivial to have as much as 1GB in your home machine right now
Yes, if you buy a new PC, but then it will probably come with a 10GB hard disk.
Argument: how about power failures?
Reply: can you say UPS?
Can you say "irritating patronising phrase"
If a new fast version of ext2fs came along that always did a reformat instead of a file system check would you use it? Even if you had a UPS? You'd be crazy. Linux may be stable, but we are not at the point where I would be willing to bet the entire contents of my harddisk that none of the drivers have a lock-up bug in them.
The solution probably is something that uses RAM to hide the slow disk speed, but it has to be more complex than your suggestion if it is going to work.
I don't think that's the limitation just yet. Light can cross 1m in 3ns, that's less than the distance from my processor to it's RAM and back, yet the RAM has a latency around 100ns. And it's less than the distance to my hard disks and back, but they have a latency over a million times more than this.
Rubbish, it's because IDE isn't terminated and isn't differential, and for all I know it isn't asynchronous. You can make Ultra-2 SCSI much much longer, even extend it to a couple of miles with fibre.
I think we need CPU architecture to build upon single simple devices that communicate in a way that is not bound by the speed of light. Once the CPUs get there, the rest will follow, as is seen to fit. Read ``Communicating sequential process'' by Hoare et.al. and think about the communication happening way above c. That's sweet :)
Either you or Hoare or Einstein has misunderstood something. I can't say for sure which of you it is...
scsi may be faster than ide. the scsi bus is
quick. scsi hardware tends to be higher speed.
however, the gap between scsi and ide is never
more than an order of magnitude even when you
consider raid solutions. ram is 5 orders of
magnitude faster than disk - scsi or ide. it's
not like doubling disk speeds would help. we
need to multiply them by 100,000 to catch up.
Yoghurt
Actually, diamonds would be an excellent choice, were it not for the DeBeers family. You see, the hell of it is, diamonds aren't even all that rare. It's just that the DeBeers family owns all of the known mines. They exercise monopoly power in ways which would make Bill Gates look like RMS, and that's why diamonds are so expensive.
Diamonds aren't that tough to manufacture in labs, either, believe it or not (after all, diamonds are just carbon, which is plentiful; add a bit of boron and you get one hell of a semiconductor).
However, you still have one major problem: speed. Consider the following: CD/DVD has made creat strides in the past years speed-wise, but they're still much slower than a hard drive. And that's only on 2-dimensional media. Add a third dimension, and you get something which is absolutely wonderful for backups, but not for general use.
Though I'll be the first to admit, the day I can use Trek-like "isolinear chips" will be quite cool indeed...
Why not build hard disc heads the same way we build CPUs?
Instead of a single head sitting on an arm that has to move back and forth to reach a particular track, why not use chip manufacturing technologies to build "head chips" which have thousands of heads, at least one per track.
Then transfers will be purely related to rotation speed.
Cheaper too.
Deleted
Seek time doesn't scale, ok you got that right.
But it doesn't either on CPUs or RAM (well it's done some, but it won't go on).
The fact is, information travels thru copper with less than the speed of light, and certainly never with more than this speed. We can only increase transfer rates (by using wider busses), we can't decrease latency, using current techonology that's bound by the speed of light.
Until we start using tunneling (that some physicists seem to believe can transfer information above the speed of light) we will not scale well in latencies.
Disk latencies, being so much larger than that of RAM and CPUs, will scale for some time to come. RAM and CPUs are almost dead in the water, until we get this breakthrough.
Until then, we can only re-arrange our problems to be transfer-rate bound, more than latency-bound. That requires thinking ofcourse.
I don't know much about physics, but I try to keep up by asking the right people the right questions. And there seem to be some possibilities in either tunneling or super-conducters, both which in some cases might allow information to travel above this limit of c. Meanwhile I'll be doing what I can do, to make problems rate-bound rather than latency-bound.
Still, you have _miles_ of wire in your CPU alone.
:)
And light travels thru copper wiring at about 0.6c.
Why do you think your IDE cables can't be any longer than [insert spec number here. it's less than 1 meter]. It's because of 0.6c and the latency that follows. The speed of light is a real limitation even in low end devices _today_.
Disks can improve latency for quite some time to come, because they're bound by the rotational speed rather than the speed of light, so no worries there. But the rest of the machine is dead in the water in this comparison.
I think we need CPU architecture to build upon single simple devices that communicate in a way that is not bound by the speed of light. Once the CPUs get there, the rest will follow, as is seen to fit. Read ``Communicating sequential process'' by Hoare et.al. and think about the communication happening way above c. That's sweet
Sorry to follow up on my own posing.
But light of course doesn't trave thru cooper. What I mean was: Information travels thru copper at around 0.6 times the speed of light.
For an electric signal to travel thru less than 1 meter of copper, it still takes more than three nanoseconds (because we're travelling at at less than c).
Build your disks infinitely fast, with 1ps seek time, you'll still have more than several nanoseconds of latency.
Build the same disks, but use superconductors/tunneling for transport, and you'll be rockin'.
I believe neither I, Hoare nor Einstin misunderstood anyting in this subject. Because neigher of us really cared/cares much.
As long as information travels at a speed slower than some fixed boundary, the information latency is going to become a bottleneck at some point in time.
Still, to get back to the original point of this discussion, disks have a long way to go, and CPUs, RAM etc. doesn't.
Expect superconducting/tunneling to become big, or expect disks to scale much better than CPUs/RAM/etc. in the future.
Or, perhaps you have some theory that I'm not aware of ? I'd appreciate to know about it.
By turning copper into fibre, you only scale from the copper speed to somewhere nearer the light-speed (in vaccuum). You'll never _really_ scale.
As was pointed out in another post, another solution might be (temporarily) to decrease the length of the wiring. Still, in the long run, we need communication faster than the speed of light.
Drives is not a bottleneck. If you need performance, go with RAID.
I can transfer 34 MB/s from my disks (sustained), and around 100 MB/s from my main memory (RAM). As I see it, disks are not the problem here.
RAM today does some sort of parallel-like access to get the performance they get today. If you do the same with you disks, (and why wouldn't you, if performance is that important?) you get equal speedup. Also put more CPUs in your box, when the CPUs become the bottleneck.
Today in PCs, the PCI bus is the only real bottleneck, perhaps along with the memory subsystem. The memory subsystem has improved a lot over the last few years, but the majority of PCs still come with 32 bit 33MHz PCI busses.
When we start seeing motherboards with 64 bit 66 MHz PCI, that's when we'll again be looking at disks, cpus and memory as the bottleneck.
I think there's room for the Solid-state "drives". Which is actually memory, but in harddisk speed and size (several Gigabytes). But it should be cheaper than hard disks (sooner or later ;), and it is already faster.
Actually you can even buy Solid-state drives today, but it costs much, isn't big enough for your bucks...
Can anyone throw in a few URLs?
Promise Technologyy ( http://www.promise.com) makes a Raid controller for Ultra DMA drives. I've been looking at these for a while, but just haven't had a good enough reason to go out and buy one. Not yet, at least. (I was planning on getting one for my _next_ computer.) It seems like a great way to get a lot of performance out of cheap drives. I think it is all done "in the hardware" -- it looks like a single drive to the os and so It _should_ work with linux, but don't quote me on that one.
The price is good, about $125 (US), and four Ultra DMA drives to fill this up would be much, much less expensive than four equivalent SCSI drives. Granted, an uIDE Raid array will be different than SCSI-based array, but it should be close enough for a meaningful compairison, I think.
Has anyone used uDAM Raids? Heard Any reviews?
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
Dang! It looks like the current card, the uDMA one, is plug-and-pray. I think it _could_ work, but booting from a pnp device could be a real drag. I guess if you used a boot floppy. . .
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
The Promise cards doesn't sound that good, on further thought. I can put two different uDMA's on my current motherboard, each on a different chain, and set up one for all the data and the other for swap. I'm not sure how different that would be, performance wise, from an uDMA "Raid".
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
They recently presented an operative one-gigabyte ROM version of their product to the public, and from what I understand, they're currently working to commercialize this technology. They also claim to be able to produce a working RAM version of their product in less than a year. This article (in Norwegian, I'm afraid) sums up their product and marketing ambitions for next year.
I'm not qualified to judge neither the quality nor the feasability of their technology, but they've been highly debated in Norway. The debate has apparently subsided somewhat now when they have working prototypes that may indicate they're moving in the right direction. If somebody knowledgeable out there finds their theories worthy a comment, I'd really like to hear about it!
And despite what you said, you would have to spin head-per-track. Maybe you're thinking of the obvious extension of head-per-bit, but without motion there is no signal for a magnetic change detector.
More to the point, if you were going to make a head per bit, just use a memory chip: if you're already going to have O(one piece of circuitry) per bit, just make it a flip-flop or a capacitor instead of a magnetic reader/writer.
Actually, I have been thinking that having maybe two or three arm assemblies, instead of just one, would be a good idea: they could each be assigned to a smaller group of cylinders, reducing seeks, or all cover all cylinders but be arranged around the surface to reduce latency, or some combination of these. It would be expensive, and it would make the electronics in the controller a lot more expensive, but that's what these new faster ICs are for, right?
I remember the other day someone was complaining that the drive manufacturers keep concentrating on making the drives bigger instead of making them faster, which is the opposite of what is needed for big RAIDs. I guess the problem is that capacity looks more impressive, and the extra expense for a faster drive of the same size wouldn't impress most normal people, so it wouldn't pay off so well.
David Gould
David Gould
main(i){putchar(340056100>>(i-1)*5&31|!!(i<6)<< 6)&&main(++i);}
Heh. I suppose that means that the computer market might be really interesting in a decade or so, instead of just chosing between different variations of similar technology, we'll have to see if we can settle for existing optical and/or biological devices or if we want to upgrade to the new devices based on quantum effects. Much more interesting than RISC or CISC, SCSI or IDE, blah blah ..
I'd been wondering why I couldn't read stuff on screen as fast as stuff on dead trees, no matter what size or resolution monitor. Got any links to those lab tests?
I see even classic Slashdot is now pretty much unusable on dial up anymore.
Chevy had a fuel injected V-8 back in '57 or '58.
('course Diesels have always been fuel injection)
I see even classic Slashdot is now pretty much unusable on dial up anymore.
I remember performance-measuring on my first drive. About 100k per second. The PC probably contributed to that number: It did 3600 RPM, 17 sectors per track, 0.5k per sector, 60 seconds per minute: 510kbyte per second.
:-)
My new Quantum does 15Mb per second. That's a factor of 30. Moore's law predicts a factor of 30 every 10 years. (= A factor of 1000 every 20 years, = a factor of 5 every 5 years).
How long ago did I have my first hard drive? '87: 12 years go.
I'd say that harddisks follow Moore's law pretty closely.
Indeed, processors seem to have outperformed moore's law a bit the last decennium, so indeed the gap has increased a bit. (Oh well a factor of 20...
Does anybody remember those PC indexes that measured how much your PC outperformed a 4.77Mhz IBM PC? What would a Celeron/400 score now?
Is that realistic?
Roger.
with the assistance of "Tata Industrial Semiconductors" (name changed for security purposes), a Taiwan based (location changed for security purposes) silicon foundry.
"new Electron Trap design..."
"We may just NOT sell it to the top 10 PC Companies who monopolize our industry."
and my favorite:
"We have no idea where the drawings from which we derived our TCAP came from."
Umm.. aliens, maybe? And put it in a PII shell? Come on.... The whole thing just sounds ludicrous. :-)
Heh, did some more digging, and they *do* claim that the technology came from aliens! :-)
Here's a guy who has some info on their claims, mostly debunking them. http://www.uni-weimar.de/~kuehnel/TrueorNot.htm
Actually, IBM did this with their (3350?) series of disks for mainframes. It was a conventional disk, with the addition of a separate set of heads fixed over the (I believe) outermost two tracks of the disk. Very handy for swap space and system libraries.
Just junk food for thought...
I don't think that massive buffering on the disk is the answer, but I imagine there is a lot that could be done with more memory on disk controllers. If a controller could cache several hundred pages, it could more easily order writes according to locality. This could also extend to reads, to a lesser extent. We now have video cards with 16 and 32 MB of memory, why not a disk controller with a similar amount? Of course, there's a problem with system crashes causing "written" blocks to be lost, how long to copy 32M of memory to flash ram?
Just junk food for thought...
If the people working on this live up to their promises, we'll have terrabytes of data stored in a cube one inch on a side, at access speeds similar to those of RAM.
It's in the labs, but who knows when it'll be available....
--
-- Out of cheese error! Redo from start.
What kind of crack do you smoke. All the Firewire stats I've seen report throughput in bits per second. SCSI stats are reported in Bytes per second. Ultra2 SCSI does 80 MegaBytes per second, Ultra3 SCSI does 160 MegaBytes per second. Firewire claims to support 800 Megabits per second, this is 100 MegaBytes per second, which is between Ultra2 and Ultra3. Also, I don't know if you can build a hardware raid array on the Firewire bus; at least I've never seen it done.
...Linux!
Andrew N.
(I probably own Apple money for saying Firewire)
--
"You never know when some crazed rodent with cold feet might be running loose in your pants."
-Calvin
Could you give me a link to a RAIDED Firewire array? I've never seen one and don't think they exist.
...Linux!
--
"You never know when some crazed rodent with cold feet might be running loose in your pants."
-Calvin
Argument: how about power failures?
Reply: can you say UPS?
Static RAM. NVRAM.
-fb Everything not expressly forbidden is now mandatory.
In all seriousness it's not the buss speeds that are really killing us for the most part (true a faster bus helps, but a pinto can only go so fast, even on the Utah Salt Flats.) Still the big slow down is that damned platter and head moving all over the place, which actually does a pretty good job considering the age of the base technology. One thing that really hurts us is fragmentation, people need to defrag, people don't realize how much this really helps. And yes, 5% fragmentation is real bad, think about it, you really only use a small percentage of your drive (depending on your size and habits) and that small percentage that is fragmented is in the most used part of the drive, so that head searching all over the place for stuff you use constatly really hurts. One thing that's been a good move is more chache on the drives. Then there's the cool move by quantum, the solid hard drives, -real- expensive, but damned neat, last I remember it was some tens of thousands per gig, but oh so fast. According to their site they seem to be up to 3.2 gig now (no price listed of course) so you could actually use these for something cool, no raid needed I'm sure, but wonder what it would do on a raid.....
matguy
Net. Admin.
matguy(.com)
ok, so I should read further down before I post, doe!
matguy
Net. Admin.
matguy(.com)
Far more detailed info at the subject URL.
Research has been underway for a few years on the "protein drive." The operative principle is a photosensitive protein which occurs naturally in swamps. I believe that it darkens upon exposure to light to cause the (light/dark)-(1/0) memory mechanism. Throughput is proportional to the time required for the protein to respond to the light condition, and storage density is proportional to molecular "resolution" and the energy of the impinging light waves. The storage capacity of the funny-looking-prototypes in the labs in '97 was on the order of 1GB/cm^2. The access time was on the order of 10^-3 s. Improvement should come from tweaking the molecularity of the protein, possibly by mutating the life-forms that produce it (speculation.)
This is of course not much help for small files, but for larger...
Even if the seek time is not much improved, the total read/write time should be at least a half magnitude faster. Think of bigger programs, graphical files a s o.
Mundus Vult Decipi
- large
- cheap
- reliable
You can see this has been teremendously successful, and hard disk technology has been progressing extremely quickly. I don't think it's outside of "the revolution" at all. Speed has merely taken a back seat. This is why you see things like RAID, to bring higher speed to storage.John Mashey (SGI, was MIPS, Bell Labs) has been lecturing recently on "Infrastress" which discusses the problem of computer architecture not keeping up with CPU speed. Not just a problem of slow disks, which can be raided to get parallel speed increases, but bus and net bandwidths, etc. He says: "It's as if cars suddenly became 10 times cheaper, but the roads didn't change." See articles: here and here.
RAID 1 is mirroring and therefore gives no performance advantage. In fact, there is a slight performance hit for the overhead of writing the same data twice, once for each disk.
RAID 5 is a good compromise between data integrity and speed. You should really go for at least 4 disks and more if you can afford it as the % of space lost to parity information goes down as you increase the number of drives.
Free Mac Mini. Yes, I'm
RAID 0/1 requires 4 disks and it is configured with 1 pair of disks (the RAID 1 part) mirroring a striped pair of disks (the RAID 0 part).
- RAID 5 will do 50% worse again than RAID 1 for the same number of drives.
That is just plain wrong. RAID 1 is mirroring. Because the controller has to write the same data to two disks, there is a performance decrease.RAID 0 is just a striped pair. The data is spread out over two disks so that reading and writing data can be distributed over both disks.
RAID 5 is the addition of parity information to a striped set. The data and parity info is spread out over all the drives with the parity info never residing on the same drive as any other part of the file.
Look at the following diagram (fig 1.1)
drive #1 | drive #2 | drive #3
-------------------------------
file 1.1 | file 1.2 | file1.p (parity information)
file 2.p | file 2.1 | file 2.2
file 3.2 | file 3.p | file 3.1
If you go to read file 1, the RAID controller will read part 1 from drive #1 and part 2 from drive #2 simultaneously. Read performance from RAID 5 is essentially the same as RAID 0.
If you want to write file 1, the RAID controller will write part 1 to drive #1, part 2 to drive #2 and the parity information to drive #3. This is slower than RAID 0 because of the parity information but still faster that RAID 1 because you are not writing the entire file to two places. You are splitting up the write duties to 2 of the drives and taking some overhead time to calculate and write the parity information.
- Only if you have a server where performance isn't much of an issue, e.g. a file server, should you consider RAID 5.
RAID 5 is excellent for applications where you need performance and data integrity. Add a hot spare to the array and you will have to have 3 disks fail simultaneously to take you down!Read a little bit more about RAID. It can be complicated but it'll be nice to know. Good luck.
Free Mac Mini. Yes, I'm
The only performance benefit you might get from RAID 1 is that it is SCSI. The file exists on each drive and must be read from the primary drive. In order to appreciate any performance gain on reads in a RAID you must be reading from multiple sources simultaneously.
Free Mac Mini. Yes, I'm
You are correct. I wasn't trying to write a complete discourse on RAID, just point out details relevant to the original post. My mistake for intimating that it was only striping.
You will have to re-read my post as none of what you said contradicts me. I made the point that, with a hot spare, not until the 3rd disk fails will you be unable to use the system. Just a matter of how you phrase it, but the point is, you can lose 2 disks and still function. Granted, the performance is degraded by using the parity information.
RAID 5, according to everything I have been able do read on the subject, does give you increased performance over RAID 1, especially when reading data. Essentially, it is like reading from a RAID 0 striped set. Writing data does take some more time due to the parity bits as you mentioned. I believe it was pointed out elsewhere that file servers are perfect for RAID 5 due the the increase in performance coupled with the data redundancy but Database servers would not function very well with RAID 5. I was merely contradicting the previous statement that a file server should use RAID 5 because of the lack of performance. To repeat another poster, if money wasn't an issue, we'd all have RAID 0/1.
Free Mac Mini. Yes, I'm
I'm shocked that no hard drive manufacturer has thought of this before, but I've had two simple solutions for years that will solve the bottleneck quickly and easily.
:)
Instead of one carrier arm with read/write heads, why not just two, three, or four with a simple multiplexing controller?
The positioning commands would be synchronized, while data bits would be fed in alternate, optimally sized data packets to the heads.
And if they're not already, the heads themselves, on alternate sides of the platters should be running with a split data stream, too. I think that they're not now, otherwise, multiple platter drives would have a much higher sustained data transfer rate than single platter drives.
The old way of thinking has the formatting tracks, as viewed from above with theoretical x-ray vision, laid directly above and below each other on opposite faces of the drive. This is probably thought of as a challenge to maintain, but this is completely unnecessary.
One set of tracks will be offset by a slight amount under my scheme, but that would be completely harmless, as the second head would always be fed and read by the controller with the exact same delay that was incurred by the drive while formatting. Thus the data remains in sync automatically, althought the formatting tracks are offset.
Simple, cheap, effective, and very quick to develop and implement. If the controller were 100% efficient and the data chunks even, dual multiplexed controller arms would double the performance (and half the access time), and multiplexed heads would then again double the data transfer rate. Ta-da! 3ms or less access, and a theoretical potential to quadruple the sustained data transfer rate. All without increasing the platter speed any more.
With further research, a bit of time, and some new logic circuitry, I'm sure that the drive head carrier arms could later be ran asynchronously, therefore reducing even further the random access time.
All rights reserved, manufacturers are invited to contact me at mturley@mindspring.com for reasonable licensing terms.
They physcial side of SCSI is exactly the same as with IDE: new stuff is put into higher end drives first, and higher end drives have SCSI interfases.
The reason why SCSI drivers are at 10k and IDE is at 7200 is because people who want/need/can afford 10K drives have SCSI, anyway.
Get away with you! There were never any hard drives fitting that description in 1970! In 1970 most computers still only had magnetic tape for mass storage, I don't think even drum storage was widespread yet. The biggest hard drives in the mid-1980's were about that size and capacity.
How old are you? I suppose you think everything before 1980 was in black and white, etc. etc.
Consciousness is not what it thinks it is
Thought exists only as an abstraction
I lack the hard (no pun intended) data to back this up, but I think that hard-drive raw transfer rates have been rising and access times have been falling. I will agree that they haven't gotten better anywhere near as quickly as other parts of a computer system though.
That being said, the only thing I can see as being an eventual competitor to hard-drives is the drawing board idea of holographic storage. Whether or not any products based on it ever make it to market is anyone's guess.
Need a Python, C++, Unix, Linux develop
The most problem I have with my computers or other computers is that the dang hard drives start dying!!! I still have a 386-40 and it works great execpt that it's had 5 different hard drives in it's life!! But then again I have a 400mb Maxtor that runs every day since it was the bigest drive on the market... It's all about quality, a lot of people hate Western Digitial and I know why, I see a lot of them die compared to other drives. All that is needed is companies to make good drives. Solid state storage is comming, but we'll have to hold on.
Couple of corrections:
1) RAID 0 can either be striping, doing writes a chunk at a time to each disk in the stripe set, or concatenation, which is slapping multiple disks together to make one big virtual disk and then writing so you fill up the disks in sequence.
2) RAID 5 can only tolerate losing 1 disk in its set. The volume can continue in "degraded" mode where for every read off of the volume, the system has to calculate the parity bit to make up for the lost disk. If you throw in one hot spare you could lose 2 disks total. (The hot spare takes over for the 1st disk to go..the volume needs a period of time to sync the data/parity to it, but then continues normally. You can then lose 1 more disk..this will put you in "degraded" mode.)
3) I'm not sure where you've gotten the impression that RAID 5 is excellent for performance. Calculating parity bits for *every* read is the antithesis of performance. RAID 5 (or "poor man's mirroring") is good for systems that need the data redundancy but cannot afford the cost of doubling their disk usage for mirroring. However, no compentent SysAdmin would use the terms "performance" and "RAID 5" in the same sentence.
Programmers have been getting lazy. Digital assistants have no hard drives and they have programs that are functional. They programs are not as bloated as the ones on the desptop.
A nice laptop with 64 magabytes of memory, and half a gig of solid state disk would be enough. Espsically, if the VM scheme just mapped programs to the memory. And need a game, pop in a 64 meg solid state floppy with the game on it.
Now all we need is for reasonalbly prices flashdisk memory.
Um...Apple didn't invent SCSI or FireWire...
I'm a loner Dottie, a Rebel.
RAID
I'm a loner Dottie, a Rebel.
The chief reason why hard disks are "slow" (ever used a tape as your ownly means of storage? or how about floppies?) is the RPM of the disk itself. 10,000 RPM drives are much faster than 5400 RPM drives ect.. If you really want to increase performance of your hard disk use a RAID system, either EIDE or SCSI. SCSI will be much faster, but DMA/33-66 makes EIDE relatively fast. When more FireWire drives come out we'll probably see nice and fast FireWire RAID setups too, which would be alot faster than SCSI because all the drives run peer-to-peer which means you dont have to bother with the SCSI controller spreading data around. Until we start seeing affordable RAM drives we're going to be stuck with out solid state magnetic media. BTW, my bottleneck has usually been my internet connection, even my cable needs a modem for upstream.
I'm a loner Dottie, a Rebel.
I can't believe how many people responded with completely irrelevent replies about "faster hard drives". Improvements in memory & cpu have made my p2-750 ~1000x faster than my 8088. But that hard drive with the same basic technology is ~10x faster. Even if you throw memory at the bottleneck, you're still 100x slower than on-chip access. That's why future discontinuous improvements will be made with increasing local memory...but wait, isn't that how the brain work? The processing and memory use the very same architecture! This isn't so far off really; the hypercomputer (http://www.starbridgesystems.com/Pages/about.html ) will be cpu-instruction-set + memory programmable on the fly. By the way, check out http://www.sunsite.auc.dk/FreakTech/Hard.html for coming technology.
tcboo
IBM a while ago issued a press release claiming
they've developed the "holy grail" of memory:
SRAM speed (10ns access)
DRAM density
nonvolatility
If they realize the promise of this type of memory, many computers may not need hard disks at all in 10 years, when 4Gb chips are being manufactured.
DRAM access is 50ns, or so, and that is just
to get stuff off of the chip once it's asked for
at the chip.
In reality, it takes more than 120ns to do this:
1) Miss the cache
2) Ask for the data on the bus
3) Get the data off of the chip
4) Ship the data back to the CPU
5) stick it in a register
6ns is just the clock period of the memory bus
on the very fastest SDRAM interfaces. 133MHz
memory does NOT imply a 6ns access time: it
means that you can put data across it every
6ns.
Gallium arsenide? I don't think so.. maybe something like indium phosphide.
There was an interesting piece recently on sciencedaily about some research being done on electro-optical effects in photo-reactive crystals.
If you like that link, Salamo has a (very brief) page describing some of his other work as well.
Look around, and choose your own ground. -PF
Nanochip, Inc. says they've got a 56-pin BGA-packaged device that holds 900MB (re-writable, 2.5G WORM) with max latency less than 500usec and a data transfer of 3.1Mbytes/sec.
They also say, "In addition to the Nanochips, we plan to provide a disk drive replacement which will have a capacity of up to 200 Gigabytes using BGA packaged Nanochips, and 1.4 Terabyte drives using flip chip packaged Nanochips in a standard 3.5" form factor (5"x3.25"x1.5")".
Not too shabby, especially if you get a few working in parallel. OTOH, 500usec is still a long way from 6ns, and look at all them moving parts! Wonder what the MTBF is?
( http://www.nanochip.com/)
Look around, and choose your own ground. -PF
RAID 1 gives slightly slower performance on writes since you have to write the same data twice, but if you have a high-end system, you'll have an intelligent controller and multiple SCSI buses, so this is pretty much invisible. It gives better performance on reads, because you can read from either disk.
RAID 5 is fair for sequential access, but not so good for random access, particularly random writes. For example, if you need to update a bit of data on disk 1 in a RAID 5 set, you have to also read the same location on every other disk, calculate the parity value, and write that to the parity disk. For random writes, RAID 5 is slower than a single disk. RAID 5 is good for file servers and web servers, and lousy for database servers for this reason. RAID 5 is a trade-off between speed and cost. If cost wasn't an issue (hah!) you'd go for RAID 1+0 every time.
SQL Error
Static RAM. NVRAM.
Static RAM isn't that static! It still loses its contents when you lose power. Besides, static RAM is much lower density and much more expensive than dynamic. The largest current SRAMS are 16Mb.
NVRAM is NV, but is even lower density and more expensive than regular static RAM. And Flash is slow and has a nasty habit of burning out if you re-write the same block too many times.
SQL Error
In 1982.
I remember that HP had a special on at the time: three of those drives for only A$250,000! But they didn't have many customers who needed 1.2GB.
Going back a bit further, we had 5MB 14" removable disk packs. I don't know what the state of the art was as far back as 1970. Try alt.folklore.computers.
SQL Error
Its Simple, Its 2 words, it would be incredably fast... Flash Rom, High Capacity Flash Rom cards, or whatever you would like to call them, it allmost has to be the future of storage (other than organic, which would probably be painfully slow, and might start stinking after awhile) If you had flash that was as fast as todays ram, you could remove the need for ram allmost entirly, and the more that is mad the less expensive it is, Imagine running linux off flashrom, you would probably be running nearly 100 times faster, and with no moving parts.. you should have no HDD Crashes.. flashrom can accomidate the size of next generation machines and when upgrading is just plugging in a new PCMCIA sized card... wow..
Forgive my grammer and spelling, this is rant mode..
"A learning experience is one of those things that says, 'You know that thing you just did? Don't do that.'" - DNA
I remember reading about hybrid magnetic & optic storage a few years ago. As I remember it, bits were stored as normal on the magnetic tracks, while a read-only optical track provided sync bits. This allowed the magnetic 1's and 0's to be much smaller and denser. I believe it's what eventually lead to products like SuperDisk.
Anyway, I was always wondering why they didn't apply this optical sync track to a standard harddrive? The paranoia side of my brain says "companies regulate technology advances so that older technology becomes obsolete at a steady rate." In other words, if Maxtor came out with a 1 terabyte drive tomarrow for $500, they'd be left with a warehouse full of worthless 18 gig drives... sad.
_______
2B1ASK1
Hard drives and HD controllers allready have a mediocre amount of cache. Why not Increase the amount of cache massively? A large amount of ram dedicated to the task of caching reads/writes for the HD might increase performance significantly.
Take your average 20G SCSI drive and slap on 2G of ram. 1G can mirror recently accessed files, possibly guessing what the computer will need next. The other gig can act as a write buffer. Perhaps dynamically decide how much should be read and how much should be write buffer based on recent performance. Things would slow down if there was too much data throughput, but you can get more ram to balance out your massive permanent storage.
With the execution path of a modern CPU increasing as it is and with a few tweaks of the linux scheduler it may be possible to more efficiently manage disk reads and writes. Those 500k cycles can be saved for something that isn't stuck on a missed page.
No, this doesn't solve the problem but it is a good transition to massive ram disks. As ram prices fall and HD sizes increase I think we can see a more intelligenly scheduled data pipelines and multiple RAID arrays that will both allow for terrebytes of storage and gigabytes of bandwidth. Now you just have to worry about making it cheap.
Actually, SCSI drives tend to be quite a bit
faster than IDE drives in their media transfer rate and average seek times. Also, note that the top-end scsi drives now spin at 10Krpm, while IDE is only now getting to 7200. With ATA/66, the interface speed isn't actually all that different.
There is also the point that SCSI can generally queue commands to multiple devices, while ATA is just getting into that.
This is an April Fool's joke from a while ago, don't bother trying to dig into it...
Did anyone else notice that this was probably one of the world's longest run on sentences?
"Let's not bicker about who killed who." Monty Python
800 mbps Firewire, i don't see usb 2.0 doing anything fast, apple did it right with SCSI and apple's doing it right with firewire, need i say more
Program size won't help you when you have a large amount of data to throw around. Raw uncompressed video is NOT going to work in a 64 MB laptop.
Sure got a bunch of responses to this question, which makes me feel all warm-n-fuzzy. But it seems like a lot of people are focusing on transfer speeds, which have gotten faster, as opposed to disk access latency, which has barely improved on a comparative scale with processor, memory, and network speeds. Sure, if you have a program that demands your hard drive spew out 40 MB/second, you can get disk arrays that offer excellent performance, but that's not the way an operating system typically uses the hard drive.
/. so much is that it re-affirms my belief that not *everyone* is an idiot!
If you've ever spent much time studying network traffic, it's a comparable situation. Like network traffic, hard drive traffic is very bursty in nature. And for each separate burst of requests, we have to wait for the hard drive arm to position itself, then wait for the platters to rotate half way around (on average). As some of the above posts pointed out, this is an eternity of time for the processor to wait. And this occurs for every burst of I/O requests.
Several strategies attempt to mask this access latency (process/disk caches, read-ahead requests, etc), but none of these can make up the difference to get us to the "next level" of disk I/O performance.
Still, overall a great response from everyone... one of the reasons I love
-- Mid
http://www.m-sys.com
SCSI, IDE, and ISA cards. Yay.
-Chris
AND less power, less heat. Plus they'll probably last longer, so total CoO will be cheaper...
Well, "for comparable speed" there's less heat. Yeah, billy's overclocked 650Mhz celeron generates some heat, but to match a 10K rpm drive, we won't need something that burns holes in steel.
-Chris
Partway there, but that's only 4MB, right? Or something thereabouts. Good for moving your pr0n from one computer to another, but you can't exactly install windows on it. (Well, anything since Windows v2, anyway)
-Chris
It's not that SCSI is expensive, it's that IDE is disgustingly cheap since Intel decided to stick it in their chipsets. If they had picked SCSI as the interface-of-choice, we'd all be packing UW storage now, and there'd be people evangelizing about some other format, say Firewire.
Of course, (as mentioned elsewhere on this thread) the point is the actual storage speed, not the interface speed.
-Chris
It's a shame that so many parts of PCs are still made with mechanical parts. And in my experience, those are the ones that fail. The hard drive, floppy drive, plethora of fans, and removable storage of all kinds. And these are all the technologies that "slow us down" on the path to Uber-computerdom. What we need is standardized, solid-state memory devices. Two kinds: a fast one that can be assembled into huge storage devices, and a cheap one that can be used to transfer data between machines. Forget floppies, CDs, Zip disks, and hard drives. We'll have the storage catch up with the memory, cpu, and system bus.
:)
The questions of course are: reliability, performance, and durability. Even using old 30-pin simms in huge arrays attached to a constant power source would probably be faster (overall, not bursting, unless there was a really nice cache on there) than =5400rpm drives, but how long is that going to last? And if you build a twenty-meg portable "disk" out of this, what happens when you drop it on the floor?
Oh well. I think I'll go drool about that atlas 10K now...
-Chris
"10 gig + per 1cm x 1cm x 3cm cube"
Hate to nit pick....but
Cube means all sides the same length.
You can use a RAID, but this doesn't mean that you can get the fastest rate. You have to configure it to to striping and all that so the data isn't just on one disk, but spread over others, so one block may be on one disk the other on another and etc, so basicall you pull so many blocks at one time increasing the file pulling. Also if you go this way you may want look into the 10K RPM scsi disks also for better higher tranfer rates on each disk.
I ate my tag line.
I ate my tag line.
-=Ellis (D)25=-
For those people willing to shell out the big money, fast I/O is certainly possible. However, I/O is not just the raw hardware but also the file system, data organisation and other important points like data integrity. RAID systems are available but figuring out an automatic mechanism of splitting up higher level files to optimise the access patterns is non-trivial. For the average consumer, there is no real need to stream multiple Gbytes off the disk. On the other hand, businesses in the document storage business don't blink when salespeople quote 6 or 7 figure prices for high-end enterprise storage systems (one reason why EMC and kin are earning 50%+ gross margins). Sure beats the 5% on PC sales.
You want, you pay for it.
LL
I remember reading in boot magizine (about a year ago) about this hard drive that was made up of EDO Ram chips I beleive. They were talking about how large banks use them for transactions, or email servers with lots of data going through, etc. While reading data from the hard drive proved significantly, the computer they tested it on ran at the same speed, just with faster disk operations (reading/writing, etc.) If somebody could dig up this article, I'd appreciate it.
- ----
-------------------------------------------------
Check out FreakTech I think your find a link to silicon needle
device you talk about + a some more.
I've seen this pattern alot.
If it's a ATAPI Seagate, it dies. But not WD.
If it's a SCSI WD, it dies. But not Seagate.
WD has the ATAPI market and Seagate has the SCSI market. IMHO.
-e
-- The intelligence on this planet is a constant, but the population is growing. --
Five years ago, the idea of a 120Mb RAM drive (that is, one
on par with current drive sizes) was enough
to make anyone drool. Unfortunately, they were just way too
expensive. Two years ago, after the precipitous RAM price
drop (from about $52/Mb to around $4/Mb) a 1Gb RAM drive
was still just way too expensive. And today, an 8Gb RAM
drive is _STILL_ way too expensive.
The price is always going to be too high for solid state drives.
Thats just the nature of if. While traditional magnetic hard
drives are around, they will always be significantly
cheaper than solid state drives. Always. Just as tape
drives are always going to be significantly cheaper than
hard drives.
Hmm.. perhaps you are correct with regards to tape drives, but I still maintain that since a solid state drive would be orders of magnitude better than a magnetic media drive, the price point will always remain orders of mangnitude higher. Its what the market will bear.
Barring a radical technology shift, that is.
I don't agree that harddrives aren't much faster then back in the seventies. Harddrives have something like 10 - 20 times faster (data tranfer rates). Even the access times have gone down a little. Ofcourse if you compare this to the way processors have evolved this doesn't seem like much, but when you look at the numbers they're still pretty impressive.
Still, I would like to see some faster drives too. Not really sure on how the hdmakers are going to accomplish this though.
Message on our company Intranet:
"You have a sticker in your private area"
beauty is only a light switch away
I think it's possible that with faster, more demanding standards, more will look into SCSI...The only drawback is it's quite expensive. (duh.) Most people in the mid-range of budgeting for computers/components don't even consider SCSI because of this. (I know I didn't. I'm just a kid.) Hey, anyone out there wanna work on some cheap SCSI drives? ;]
Can't sleep, the clowns will eat me...
Just Talking out my ass here, but havent we been hearing great things every now and then about holographic crystal storage? Didn't they want those out within 5 years? I can't remeber.
:)
As a second note, with faster/cheaper/denser ram comming all the time, ram based static hard drives might come to replay magnetic media. (might) I know I want one.
-Crutcher
-- Crutcher --
#include <disclaimer.h>
Cars have gone nowhere? First of all, let me introduce you to a new technology called fuel injection. Drastically reduced emissions. Tiptronic. Viable electric autos. Drastically improved gas mileage. A gasoline powered car can now get as high as _47_ mpg freeway. 3 and 4 cylinder engines. A car that squeezes a very respectable amount of power out of a v6. From asia. Not only have there been the steady advances that we've seen from the computer world, we've also seen a drastic, universal adoption of a new technology.
You can only make a mechanical device operate so fast before the laws of physics start destroying it - or making it prohibitively expensive and large.
If we want to get out of the slow storage days, solid-state mass storage devices will have to become practical for everyone. Or some of the other fancy stuff that only exists in labs right now will have to become a parketplace reality. Hard disks as we know them appear to be coming to the end of their evolution, at least in terms of speed.
Maybe some engineer will find an inexpensive way to put many heads over a platter.
And despite what you said, you would have to spin head-per-track. Maybe you're thinking of the obvious extension of head-per-bit, but without motion there is no signal for a magnetic change detector.
The head positioning times have not changed as much because it still takes time to move a physical head across the platters. However in the 1970's many hard drives used stepper motors while the top-of-the-line used voice coils, while now most hard drives use the faster voice coils.
The obvious question becomes how to alter head position time. All the disk drive engineers for 20 years have been working on this and I can't improve on their efforts.
There are some obvious possibilities for Optical or CD-ROM/RW/RW disks. Perhaps a laser can be scanned over a whole disk to read the data faster than with current designs (even if the laser has to be several feet from disk to be able to scan). Or several lasers on several heads (this is already being done, although I don't remember the CD drive using it).
Or image the entire CD or DVD without focusing, but the compute power required for this will require extremely fast device interface hardware.
Except loading at boot time the contents of the HD to mem, this is what good OSes do.
When data is required they load it to the buffer cache, serve it and leave it in the buffer cache till it is replaced by other data because there's not enougth memory for other services or because that data hasn't been used for a long time.
Writting data is done as you described: updating to HD when necessary and periodically.
So to improve HD caching performance you do as if it were normal memory (and cache):
- Reduce first instance misses using prefetch.
- Reduce conflict misses incresing main memory size.
- Increase the data bandwidth using RAID.
With RAM prices falling the way they do, we'll soon be able to keep the whole contents of a HDD in the main memory. After all, it's almost trivial to have as much as 1GB in your home machine right now (I don't, but the RAM in my PII machine is already 1.5x the HD of my secondary system, a 486DX2).
The storage while the machine is off could still be done on a regular HD (or flash memory); upon boot, it gets loaded in RAM; regular update of the HD from buffers in the ramdisk takes place, but this can be done asynchronously, and should not affect much the performance.
Argument: how about power failures?
Reply: can you say UPS?
some playstation games did this and it worked great. it made a 2x cd-rom seem like a hard drive.
Drive speeds have certainly been improving over the last twenty years, but I agree that they haven't followed the same blistering rate of improvement that memory and CPU power have. (I remember having a 10MB, 70ms seek time drive on an XT.)
But more importantly, drive prices have been continuing to fall. And caching technologies have been getting better (as CPU and memory prices continue to fall). So even though the core drive technologies might not have changed in the last ten years, the performance that I can expect from a drive array has. The numbers on the EMC Symmetrix disk arrays and Celerra file servers are astoundingly fast. They do it with lots of drives (drives are cheap enough these days that you can have lots of little ones), combined with great caching and fast networks.
(I'm just using EMC as an example because I know them best. There's lots of disk array vendors out there.)
In short, I'm not sure that disk I/O or capacity is a bottleneck for a lot of computer systems these days. As much as CPU and memory have been getting faster and faster, they still are the limiting factoring in a majority of systems.
I would be suprised if the problem was the interface. My educated guess is that it's the solid-state storage. That stuff is INCREDIBLY SLOW compared to the RAM you find in your PC.
http://www.bullnet.com
There's ongoing research ( http://nanonet.rice.edu/research/boy d_res.html) to use photopolymers as a cheaper holographic medium. If such research comes to fruition, you're more likely to see CD like disks coated w/ a holographic layer than the typical science fiction "data crystal."
Other problems w/ holograms:
- materials are not totally transparent, so "cubes" might be out of the question
-materials must be chemically resistant to the atmosphere (e.g. oxidation, humidity), which might necessitate that they are coated. Such a coating might have deleterious effects on the substrates optical properties.
- storing a hologram changes the structure of the crystal, which can cause limits of data density and beam penetration.
- multiple holograms can be stored at the same location by rotating the crystal, but each hologram attenuates the possible intensity of subsequent holograms in that location.
- holographic "efficency" is a funciton of the difference between the refractive indices of the substrate components. photopolymers have a very small range of refractive indices as oppossed to inorganic crystals.
Overall the medium might not be rewritable, but a high density, long lasting storage medium would be ideal for back-ups.
Anyway, it's been awhile since I "got out of the business" of chemistry, but this is what I remember.
Solid State is a very good point, but don't the new G3's have Firewire connectors?
Imagine Solid-state/Firewire HD's...
It's a thankless job, but I've got a lot of Karma to burn off
Actually that's spelled MEEEPT ;-)
www.anandtech.com - review & benches a couple of months ago
Let's give the guys a break. If we do our part maybe the Linux kernel will be so sleek we'll ratify it's functions as the new laws of physics.
The ship sank. Get over it. (This sig was cut out from another's shirt and painstakingly hand-posted)
Make it a two layer process.
Have a volatile RAM Storage mechanism that pretends to be the hard drive.
Have a non-rotating plate be your Non volatile storage medium.
Have the CPU and everything read/write from/to the volatile portion only.
On its own without any disturbance of the processor, have the drive scan and update whole contents of the drive from the volatile storage to the non-volatile storage at intervals depending on the latency of the medium. Let the volatile and non volatile technologies develop independently and have the factory test the latencies to be written into a uniform driver.
By the way how would a guy like me learn VHDL?
URLs?
The ship sank. Get over it. (This sig was cut out from another's shirt and painstakingly hand-posted)
I don't see why not.
The ship sank. Get over it. (This sig was cut out from another's shirt and painstakingly hand-posted)
Breaking bottlenecks:
One way that probably will come in handy when trying to break or at least
avert bottlenecks is paralellism. And with todays prices it's also becoming
more and more feasible due to the facts that low end isn't as far removed from
the top end as it used to be. And the top end research is becoming more and
more expensive and affected by the law of diminishing returns.
For disks: Striping will go a long way and is not that hard to implement -
even today there should be a market for PCI card which could accept, for
example, 4 ATA-interfaced disks. Add intelligent caching which could be
more or less helpful but my experience is that the program/data I use tend
to be divided in to more or less recognisable groups.
For computing: wider and/or paralelled buses combined with multiprocessor
machines. Using a setup like this with a micro- or nano-kernel OS will
encourage a more distributed way of programming and in a multi-processor
machine with a high-speed bus you can also do dynamic load balancing a lot
easier by process migration.
Just my 20 milli-Euro. Jari
Have you ever seen a harddrive from 1970? I have one. It's has a silver, aluminum case, which is rather odly shaped. It's 6"x5"x"6. And it weighs quite a lot. It has three plastic screwes with heads that are about 3/4" in diamter. The data cable looks like those brown unsheilded, translucent cables that can be found in inkjet printers (the thing connected to the place where you put the ink). It has a capacity of 100MB. It was taken out of an IBM cluster controller (if you know what that is [was]).
Hard Drives have come a long was since 1970. Now CARS. CARS have gone nowhere since 1970.
Very good comment, that some technology cannot be the issue. When your model of efficient data communication between computer components cease to work, you adjust your model (ie change architecture). An indication of the need for this is (the earlier mentioned) 'infostress' (bravo for that). My 20 mEuro
nosig today
http://www.spie.org/web/oer/june/jun98/opcwg.html
I read the above link about two years ago, and was amazed by the technology. Quarter-sized 100TB holographic discs. They're supposed to have something viable in another year; here's hoping!
Read: Rabbit Rue - Free serial nove
Hey all,
I'm no physisist, but hey, kick me if you want..
How about if your new PC came with x number of hard drives, say five or six, and then some clever person came across a way of using the drives in parallel... would that help ? The only problem would be the lookup time for reading, but perhaps the index could be stored in the cache or something.
Or then again, how about having a few gigs of memory, and then when shutdown comes, the memory is written to the disk, read again upon bootup. Possible ?
But don't forget that there are limits on this stuff, absurd though they may be. For example, the speed of light (sorry can't remember... somthing about it goes round the Earth eight times per second). Once one approaches the speed of light ( I mean *closely*) the mass of the object increases more and more, and thus eventually you gain so much mass you slow down (you would going up to this, but hey, more POWER!!!) to the point that you can never reach the speed of light... could be a bit of a bugger for all those spacemen...
You can bet that as soon as faster storage and any technology improvement is usable, the OS vendors will add enough features to make the hardware crawl. I've had opportunities to rollout batches of new desktops that were years newer than what my users were using at the time. They saw little or no difference. This happened with P133 to P350s, with Mac603/200s to G3/300s.. even a R4400 Indy to an O2R10k/180.
I always thought this could be explained by my users being idiots, but I don't think so anymore. We just did stuff that would tax the hardware. Commercial software upgrades have new features, not neccessarily improvements. Thus new software doesn't take advantage of fast hardware. It just uses more, faster resources to suck less.
By contrast, Linux/OSS seems to have been designed to run efficiently, and upgrades improve performance, not just add features. I've actually seen faster performance running Redhat 5.2 vs Redhat 4, and I have not seen that kind of difference from NT3.5.1 -> NT4 or any MacOS upgrade. I hope that as Linux matures, it won't become bloated.
-jpeg
It's all well and good saying that hard drive access speeds have come up and that so have ram and cpu speeds, but the format of the data on the drives that is being retrieved hasn't altered all that much. We're still stuck with fat-16/32, ntfs, efs and the like.
Caches are good but the software can flood these.
Just listen to the hard drive activity when Windows *.*, Novell load, they make the drive active seeking for most of the time that they are loading. (linux settles down a lot faster)
Games are another thing like this.
My pc has 64mb memory, if things were ordered good and God was smiling on game developers, then the game would be loaded in just over the time that the drives could transfer 64mb of data.
(I know that I'm dreaming a tad here)
Maybe what we need is a way of doing speculative file access. Looking at files that an application needs and accesing them before the file actually requests them, possibly in a time while it is waiting for user input.
Well
They'd certainly be insane to charge only $500 for the first terrabyte drive.
A better plan $1500 for the terrabyte drive. Cut the prices on evrything else you got in half.
That way even if they can't afford you're terrabyte drive the consumer will still be gobbling up your other drives. You'd smother the competition in no time.
Then they can start regulating the advances since no one else will be left out there to compete with them.
-- Rambar
Looks pretty cool....
Article was written in 1996... And says will come out "about 2000"...
I would love to have one of these, but is this for real? First of all, I tend to discount revolutionary products that say their time of arrival is "about 2000", usually means "probably never". Especailly if written 4 years before with no real updates...
2ndly, thought I am no real harware guy, isn't that a lot of memory for that die size?
Blessed are the pessimists, for they have made backups.
Basically, the ideal would be to have all computer parts run at core processor speed, but this just isn't goint to happen. We have registers, L1 cache, L2 cahce, (k6-3 L3 cache in some boards), main memory, hard drives...
Sure, it would be ideal for all storage to be as fast as the processor, but the current cached model works pretty well provided you have enough rams and a OS that makes good use of that ram...
If you really need more performance, use RAID...
www.cdrom.com doesn't seem to be disk limited, that's because it has boku memory and raid arrays.
Besides, hard drives today are about 10x as fast (10x more throughput, initial latency has been cut about 4x) as those of 10 years ago, which isn't too bad in machanical engineering terms...
Blessed are the pessimists, for they have made backups.
I tried a couple of these boards in an SMP WinNT configuration. Both consistently corrupted data on two systems with different hardware, and I was unable to get any real help from Promise Support ("Could you try moving the drives farther apart, then call me back?").
Unfortunately, it is not "all done in the hardware". It turns out to be a rather ordinary UDMA/33 card with a BIOS that simulates RAID hardware, and needs special drivers for OSes like Linux that don't use the BIOS. I have heard that works in a nonraid fashion with late Linux kernels, which would allow software RAID.
An IBM hard drive research-and-developer spoke last week about their current work. He says their plattens are flat to within 90 angstroms, which means that the head is floating less than the mean free path length of air away from the disk surface. The head-to-disk distance seems to be one of the more important limiting factors of how much better disks can be, and I doubt whether we can expect too much more improvement using the current technology.
Play and design text adventures online.
Not magnetics!!!!!! Optical storage!!!!!
MEEPT!!!!
What ever happened to the Ram Disk/Ram Drive concept? With cheaper RAM, it seems one's desktop could function like one's PDA?
Steve
I would have to agree with a comment made earlier - hard drive access times are not the issue. More attention needs to be paid to bus speeds or a new architecture all-together. The Von Neuman (sp?) architecture is all well and good but it is the only thing that has remained constant in the computing industry. Perhaps a new architecture needs to be investigated.
----- this is my sig, do you like it?
Good advice. I said *next* best peforming RAID level, as in it will do 50% worse than RAID 0. (You do know you can do RAID 1 with an arbirary number of drives right? It's called RAID 1+0)
RAID 5 will do 50% worse again than RAID 1 for the same number of drives. Price / MB has gotten so low that if you are worried about performance, you shouldn't purchase drives based on the amount of storage you need, you should buy them based on the number of drive spindles you need for the minimum level of performance. Only if you have a server where performance isn't much of an issue, e.g. a file server, should you consider RAID 5.
It is supposedly based on technology found in a crashed UFO!
More ram? That would not gain much in terms of performance. If you study cache design, and virtual memory performance in relation to cache, the size of the cache doesn't matter past a certain threshold. Most hard drives today have at most a meg of cache, in relation to their size 19 gig divided by one meg is an one thousand to one ratio of storage to cache. These drives that have massive caches don't perform much better then the drives that have an 128k cache. What I think is going to have to happen to the next generation of physical (long term) storage is for it to move away from the hard disk mechanism. Some leap in the technology, be it hologram or whatnot, will lead the way and speed up computers. Until then, RAID arrays and larger main memorys will have to do.
You can buy a commodity off-the-shelf intel box with a couple G of RAM for amazingly little. Now, a lot of applications can't fit their data in 2G. But for those that can, in-memory databases produce astounding performance for anyone who's struggled with conventional databases. If you can't fit in 2G, go buy an Alpha or an AIX box or whatever and 8 or 10 G of RAM. If your data doesn't fit in that... read the other threads in this talk.
Note that the optimal data structures for in-memory DBs are really different from those on-disk.
-T.Here is a web page to check they have basicaly a 90gig solid state hard drive that looks like it could plug into a PII slot one processor slot you would just need a funky mother board or a expansion card I would prefer if it was on the mother board or through and AGP slot for fastest throughput ofcourse they could attack it just like a cpu and get the fastest through put the storagge would run at the same speed as the cpu almost eleminating the need for ram.
http://www.accpc.com/tcapstore.htm
"Never judge a man till you've walked a mile in his shoes. This way when you do you are a mile away and have his shoes."
I read smoewhere about crystal holography.From memory the theory seems to be that two lasers, when fired into a crystal, will generate an interferance pattern where they meet: a lower-powered laser can read this interferance pattern back. We can use this to represent binary within a crystal and this have some form of pseudo-solid-state storage.
I also saw somethong similar in the Bowie film the man who fell to earth - perhaps a preminition!!