Slashdot Mirror
Breaking the ATA Addressing Barrier
BitMan writes: "If you haven't heard, there has been a new disk geometry limitation looming for some time at 128GB (gigabytes of 2^30 bytes), which is 137GB (gigabytes of 10^9 bytes). As many will note, there have been various BIOS and OS limitations in disk geometry before -- e.g., 512/528MB, 2GB, 8GB, 32/33GB, etc... But what makes the latest 128/137GB "limit" different is that it revolves around the "hard, physical addressing" limitations of the ATA (AT Attachment) interface. 28-bits are used for addressing, which results in the 2^28 sector * 512 bytes/sector = 128/137GB limitation. As such, hardware fiends like myself were wondering when the industry would get around to addressing this "hard" limitation in the ATA interface.
Fortunately, the solution is already in the works. The ANSI ATA committee has accepted a proposal from Maxtor that extends the ATA bus addressing to 48-bits. This allows for up to 128 pB (petabytes of 2^50), which is 144pB (petabytes of 10^15), sizes. This should tide the PC world over until the 2TB (terabytes of 2^40) limit is reached, which is the maximum number of sectors a 32-bit OS can address -- i.e. 2^32 sector * 512 bytes/sector = 2TB.
In addition to breaking the addressing limitation, another addressing limitation was overcome for performance considerations. The maximum number of sectors transferrable in any command was boosted from 8-bit = 256 sectors/command (~128KB max. transfer/command) to 16-bit = 65,536 sectors/command (~32MB max. transfer/command). This should increase ATA/IDE performance in burst transfers and many other operations.
A whitepaper on the new proposal can be found here from Maxtor. Small correction in the article: Maxtor says 144 pB (petabytes) = 144,000 GB (gigabytes), which is quite incorrect. 144pb (petabytes of 10^15) = 144,000 TB (terabytes of 10^12) = 144,000,000 GB (gigabytes of 10^9).
Thanx goes to the most excellent StorageReview site where I first heard of this."
Every lame PC "standard" has continuous artificial limitations like this. Remember the 8086's memory addressing? Go with SCSI on a 64bit processor and you won't have to deal with this crap.
A. C. Clarke says you can store an entire human consciousness in a petabyte. This should be a big enough address space, then, for a while anyway.
Why can't the hardware and bios just use a 64-bit sector number. There, problem solved, and no more stupid limititions because someone is trying to save a buck or two since they only used a 12-bit address, er, now 16, er, now 28-bit address.
ENOUGH already. Design it RIGHT the 1st time.
SCSI!
How much longer are we going to be stuck with I/O interfaces that bog the CPU (and cripple the user inteface) during heavy disc access?
Apparently only about 1% of us actually want to use our computer to do more than one thing at a time.
--
- Dan I.
Sounds lovely doesn't it?
but then you'd have to pay a whole lot more for lovely SCSI hardware, which may not be an option for many of us (especially people in other not so fortunate countries)
and you'd have to recompile many applications to run on 64 bit, not to mention buying the lovely 64 bit stuff Intel and AMD can't see to agree on, which might I add is pricey, which may not be an option for many of us (especially people in other not so fortunate countries)
doing all of that? ... or a simple FLASH of the BIOS?
Sunny Dubey
I'm sure it has to do with physical limitations and performance. Anyway, this is a different problem then the BIOS one, ATA has been able to handle 137gig drives since it's creation. And now were talking about increasing that capability by A Million Even if we apply more's law to hard drives, that's still over 30 years away. 64bit on the other hand would be useful for about 54 years. Either solution would will work for quite a while, and I doubt being able to use the same hard drive standard between 2030 and 2056 is really that much more then performance considerations today.
ReadThe ReflectionEngine, a cyberpunk style n
Is a gigabyte 10^9 bytes, or is it 2^30 bytes? It depends what you're talking about.
For computer memory, the SI prefixes are certainly used to refer to powers of 2: 640 kB of RAM means 655360 bytes, not 640000 bytes.
For networks or clock speeds, the SI prefixes are certainly used to refer to powers of 10: 10Mbps ethernet can carry 10^7 bits per second, not 10*2^20 bps; similarly, a 1GHz processor runs at 10^9 Hz, not at 2^30 Hz.
And disk space? The manufacturers all specify their sizes in terms of decimal powers. And why not? Everything else, with the exception of computer memory, is expressed in terms of decimal powers.
Let's put this silly argument to rest; I'm sure people have much more important things to argue about (vi vs emacs, BSD vs linux, bash vs ksh...)
Tarsnap: Online backups for the truly paranoid
Maybe just once they should make the painful switch to a simple flat 128-bit address space and be done with it.
It would be difficult for me to accept this as a solution to the upcoming limits of ATA hard drives. what kind of transfer rate can you get off an ATA drive! If I'm going to need a 100TB hard drive it isn't going to be from my mp3 collection. Its going to be massive data files for rendering and databases. I know this is a little offtopic but shouldn't ReadWrite mass storage move away from simple linear devices before we have 1TB capacity?(Not to the certain loss of life span on a mechanical drive spinning at 50K rpms)
"Get them before they get....
What's the point of setting those standards if it's physically impossible for bits at that density to maintain cohesion?
Repeal the DMCA!
Revamping the ATA spec is like dusting off the plans for the pyramids for revision. ATA, and even SCSI are showing their age in more ways than one. We should be investing in Infiniband, RapidIO, etc.
Someone you trust is one of us.
I'd like to know, why, exactly, does my Win2k box get horribly bogged down when reading and writing the Floppy? Does this happen with Linux too? I can't do a damn thing when formatting a 1.44 disk, and for the life of me, I can't understand why manipuliting such a small about of data is so demanding.
Alcohol, Tobacco and Firearms should be the name of a store, not a government agency.
6 bits will be taken up by Big Brother's "digital rights management" software.
To achieve this, I propose the creation of the "GATE-A29" to allow older software that wrap around the addressing space to access the first sector of the hard disk to continue to function properly. This gate could be controlled with one of the keyboard controller's free lines (to save one cent), and could be turned on and off from the BIOS. Also, there should be a new INT21h function to control the "GATE-A29".
What a perfect way to extend the PC-AT architecture in the totally unencumbered and elegant fashion it has evolved so far !
[Seriously people, just buy SCSI drives, they already do the work properly]
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Whereas everything else you read on Slashdot is accepted as fact?
Ahem, ah, yeah, that should be "20MByte" not "20MBit" SCSI system.
I don't think a "20MBit" SCSI system would stand much of a chance at competing with anything except my TI99/4A.
--
Slashdot did not post your article because this is not guarenteed to happen. If Apple were to take over development of powerPC processors then I'm sure the article would be posted. As of now it is only speculation.
So please, stop posting this...
SerialATA was supposed to be hitting in the same time frame as this. Why are they dicking with 48bit just to make the hardware implimentation cheaper?
Also, has anyone checked to see if CPRM is being "stealthed" into the spec?
This sig left unintentionally blank.
wah, wah, wah, none of my articles ever get posted, wah, wah, wah, I post off-topic and it gets moderated as off-topic
Are they going to try to bundle the copy protection stuff with this upgrade? Or are the two issues completely disconnected?
It's easy to say that we'll stick with the uncrippled technology we already have. But as it ages and becomes obsolete (ie., can't handle normal sized disks), we'll be pushed into the next generation whether we like it or not. If that next generation includes copy protection, we'll have to live with it.
I propose a new unit of measure:
How about the Bogo-Gig
For instance, 137 BG Bogo-Gig (base-10 or marketing-style), which translates to 128 GB (base-2, fair and actual non-marketing gigabytes)
Any ideas on better units for monitor sizes?
Maybe ideas for MTBF (Marketing Time Between Failures?) I was told that hard-drive manufacturers actually count on several returns per drive. It's definitely not 100,000 hours like they like to say.
-d1: Use DMA
-c3: 32-bit IO
-m16: Transfer 16 sectors at a time
-u1: Unmask interrupts
According to the hdparm man page, the -u1 option will greatly increase system responsiveness. The other options mainly improve throughput. Many people also use the "-X66" flag to select UltraDMA mode2 transfers, although my BIOS seems to do that automatically.
To test your IDE transfer rate (do this before and after tuning), use "hdparm -tT /dev/hda". It is recommended you test your IDE settings on a read-only filesystem before you start using them regularly - usually the commands don't cause problems, but they can cause major data corruption with a few buggy chipsets (having a recent kernel might help).
You may also be able to recompile your kernel, having it use IDE drivers specific to your chipset, rather than the generic IDE drivers. Kernel 2.4 is probably needed, and you should know what kind of IDE chipset you have (check your motherboard manual). Go to "ATA/IDE/MFM/RLL support" in the kernel config menu, and make sure it is set to "Y". Then go to "IDE, ATA, and ATAPI block devices", make sure the following options are set to "Y":
Then save the config, compile, and install the kernel. You may still need to use the hdparm commands after doing this, just put them in a startup script.
If you want your post to be scored 1+, simply don't post AC. As for the rest of your whine, there's only so many stories that get posted here, and if you have to cut, hey, why not cut a fairly speculative story? It's not like it won't get posted here if and when it really happens. And yes, I've had a shitload of submissions rejected as well.
:-)
Oh, and just for the record: Linus is Finnish
Would you pay $24K for a Pontiac or $8K for a Yugo?
Higher quality (reliability, performance, etc) means higher cost.
Just because it CAN be done, doesn't mean it should!
Not very logical IMHO (and I think the thousands of newbies asking where the 2 gigs from their new 30 gig HD went will agree)
--
128GB (gigabytes of 2^30 bytes), which is 137GB (gigabytes of 10^9 bytes)
Calling both of these "gigabytes" is confusing. The second figure should be referred to as "metric gigabytes"!
I believe that would be a RAID.
Your welcome.
I don't remember since when, but LILO now supports linear address mode, and my linux bootpart happens to live at 10GB just fine.
Power to the Peaceful
Drives spinning at 7200 rpm will still kick data out at the same speed, and will still be as prone to failure, regardless of capacity. For performance and redundancy (I can't imagine that nothing in your 128GB dataset is not valuable), you need to go multi-spindle, i.e. RAID. Case in point:
Local screwdriver shop has 40GB deskstars for $120. They have a 75 GB for $240. The obvious solution = buy 2x40 and stripe them. Same $$, more capacity, and 2x the speed.
I want to delete my account but Slashdot doesn't allow it.
A petabyte should be PB. Peta- is not p. Internation System (SI) prefixes are case-sensitive Peta- is P, and pico- (which is very, very small, 10^-12), is p.
Liberty in your lifetime
Isn't a "pB" a *pico*byte? A "Petabyte" would be a PB, no? Same way that a mm is a millimeter, and a Mm is a megameter. It's case-sensitive.
Personally, I don't want any picobyte-capacity hard drives. How about you?
Lemon curry?
I beat you to it by four minutes. ;)
Liberty in your lifetime
Actually, that's only 8.0 zebibytes, which is the SI binary prefix for 2**70.
9.4 Zettabytes is correct. Zetta is the decimal prefix.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
"This should tide the PC world over until the 2TB
(terabytes of 2^40) limit is reached, which is the maximum number of sectors a 32-bit OS can address -- i.e. 2^32 sector * 512 bytes/sector = 2TB."
Well, you make some wrong assumptions here, a 32-bit OS can do 64 bit arithmetics quite well.
(many do so for file accesses e.g.).
Wouldn't anything that divides a byte by powers of ten be an imaginary concept anyway? You can divide it into 8 bits, but smaller than that isn't it like discussing the "integer-ness" of pi?
I see even classic Slashdot is now pretty much unusable on dial up anymore.
Serial ATA is not planned to be included on an Intel chipset until some time in 2003. It's unlikely manufacturers will produce SATA discs before then, or even for some time afterwards.
These post-modernistic bathroom stall scrawlings have got to stop... Can't you folks go back to unobtrusive, one-line trolls?
The MPAA and the RIAA will surely kill this technology!!!!
--
Knowledge is, in every country, the surest basis of public happiness.
So what if SerialATA won't be on an Intel chipset until 2003? FireWire will never be on an Intel chipset (as they want USB2.0 - to which they hold the patents - to become the standard) but you can buy a 4-port FireWire PCI card for £25.
Remember ATA66? Intel was the LAST vendor to adopt that standard into their chipsets - Via, ALi et al all had solutions in the marketplace while Intels BX was their champion and CaminoGate was giving us all a jolly good laugh.
Remember too PC133 memory. Other chipset vendors have been supporting this for ages, but Intel have only just "gotten off the dime".
You should also give the drive makers more credit. They will realise that SerialATA is a change of maybe 15% to the drives controller board - just a change to the physical interconnect and the silicon that drives it. They're already doing this to produce both SCSI and ATA drives, so rolling out another is not that big a deal.
This sig left unintentionally blank.
Not keyboard-controller, that's already used. Let's use the PC-speaker instead. If there's no connection to the HD, we'll use a microphone on it. It's not like machines haven't steadily become more noisy and annoying, especially after we got CD-ROMs. People don't care for anything but the price anyways.
- Steeltoe
http://www.debunkingskeptics.com/
For performance and REDUNDANCY (I can't imagine that nothing in your 128GB dataset is not valuable), you need to go multi-spindle, i.e. RAID. Case in point: Local screwdriver shop has 40GB deskstars for $120. They have a 75 GB for $240. The obvious solution = buy 2x40 and stripe them. Same $$, more capacity, and 2x the speed.(emphasis added)
To save people needless grief from following this advice, I feel that something should be explained.
There is a very good reason why striping is called RAID Level 0 (that's a zero), which is because it's not redundant! By utilizing RAID-0, you double the probability that one of the two hard drives will fail, and since 50% of all the data is on the other drive, even the drive that doesn't fail is basically unrecoverable! Sure, it doubles read speed, but even hardware implementations will have slightly slower write speed unless both drives are absolutely identical in geometry, access speed and spindle speed. RAID-0 is essentially useless for anything where reliability is concerned. There are numerous RAID tutorials that explain the differences. Just check Google, or see this quick explanation of RAID levels.
(Moderators, I'm expecting at least one mod point for this short but informative post which includes a link to an informative web page. If I do not get at least one mod point, I will go postal and kill my boss. Urm, I'm self-employed, nevermind.)
--RedBear
=============================================
The Peta prefix, modifying by 10^15, is written in uppercase P. The author here is using a lowercase p prefix, which means pico, or 10^-15.
:-) We haven't been using millibytes for some time, much less micro- or picobytes...
Technically, this means the author got the article 30 magnitudes off.
Why are we consumers still fighting these damn flat cables? I hate them! Get rid of them! Oh, and I'd loved to have 8 drives without a mess of cables and add-ons or something.
--- Hajotkaa siihen, kapitalistit!
While Maxtor should be applauded for creating such a huge capacity hard drive, I think we're now running into a big problem: how do we back up such a big hard drive?
It's already bad enough backing up today's 20-30 GB hard drives. You need 31 to 47 650 MB CD-R/RW discs to back up today's hard drives; high-end tape drives that store over 30 GB per tape starts costing thousands of dollars for models that run off SCSI or IEEE-1394 interfaces.
I'm not looking forward to backing up a 128 GB hard drive with 183 700 MB capacity CD-R/RW disks. (thud)
I actually have a RAID-0 strip (2x 7200 RPM IDE), and much prefer my single 7200 RPM SCSI drive. The reasoning is simple. Latency. It takes much, MUCH longer for two disk heads to seek to the proper position than it does for one to seek there. Every disk you add to a RAID-0 array adds a couple milliseconds to the latency. Now if your working in an environment where pure transfer rates is important (i.e. audio/video editing), then a RAID-0 strip is great. But for most of us, where our hard drive heads have to constantly jump all over the place, the added latency will actually result in a slower over-all experience, despite the high available transfer rate. If your really enough of an über-geek to be playing with RAID arrays, why not just spend the extra money and go for a 10,000+ RPM SCSI drive? With these drives you get the best of both worlds: high transfer rates & low latency.
/dev/random
How many people keep only one copy of any really important file? If *I* was storing the entire human consciousness on magnetic media *I* would certainly keep more than one copy. (Of course, keeping them in sync would be a real pain....)
On a more serious note, the real problem with increasing disk capacity is the attendant increase in waste. You could theoretically double the maximum capacity of a drive interface by doubling the sector size, but there comes a point of diminished returns due to poor addressing of the storage space.
The long term solution is to abandon hard limits and use a hardware/software API that allows the drives and controllers to negotiate the addressing scheme.
There have been a lot of improvements to ATA since it first appeared. But when I see so many changes and improvements year after year just to approach the same level of performance (and not really even try for the same flexibility) as SCSI has had since day 1, I can only conclude that ATA was, and presumably still is, misdesigned from the start. That's why I don't use it, and why you shouldn't either. Use SCSI, or 1394, or FC, or one of the hot new technologies that you can't buy yet. It's long past time for ATA to die. They blew it from the start and have never really recovered.
Of course, in that regard, ATA and the peecee are a match made in hell. Neither one has ever let the cost of competent design work raise the price by even 1 dollar. You might say you get what you pay for, but if you're buying this stuff you must be paying for rape.
And yes I agree its really annoying when trolls mod you down...
Realizing that G=(10^3)^3=1,000,000,000 bytes, the NIST (a governmental body that sets standards) decided to formalize binary multiple prefixes, which of course are based on powers of 2 rather than powers of ten. They defined Gi (gibi) to be (2^10)^3=1,073,741,824, which is quite different from 1,000,000,000. In this article the reader confuses gibi with giga. Please use GiB for multiples of 1,073,741,824 bytes, and GB for multiples of 1,000,000,000 - this will help avoid confusion, as well as keep drive manufactures honest.
When I read your thoughts here, two questions pop into my mind immediately:
1) Just what the hell are you smoking?
2) Why ain't you sharing?
But this won't be the end of "storage barriers". As a quick example, Linux 2.4.0 had a maximum size for any volume (even LVM & RAID) of only 1 TB last time I checked. Maxtor's white paper mentioned 2 TB limits in nearly all OS's due to use a 32 bit integer to store the sector address.
Microsoft undoubtedly has many similar implementation limits, even if the raw format of FAT32 and NTFS can handle more. If past history is any indication, PC bios code will also consistantly have problems with "next years" drives, simply due to bugs and short sighted coding, even though there's no "real reason" for it.
PJRC: Electronic Projects, 8051 Microcontroller Tools
Actually, the metric definition of Giga = 10^9 has been around a lot longer than the binary variation. Somewhere in the dim mists of computer history, someone noticed that 2^10 ~ 10^3, and started talking about "kilobytes". If anything, we should call these "binary kilobytes", "binary gigabytes", etc.
[100% ISO 646 Compliant]
SVM, ERGO MONSTRO.
Addressing is no longer the primary problems in hard disk creation. The primary problem is throughput to the system. Disks are plenty big, and the price-per-gigabyte is agreeable to most, but hard disk data transfer to memory has not significantly increased in a very long time. Someone needs to find a way to speed that up. Hey, maybe we could get the Rambus people on that problem! (Just kidding.)
Comment removed based on user account deletion
Looks like the year 2370 or so we'll still only have a few petrabytes of storage available on our Space Stations!! In "Our Man, Bashir" they had to wipe nearly all of DS9's computers to store the brain patterns of the Defiants crew..
*shrug* Just proof that anybody stupid enough can always fuck up a good thing. I could probably fuck up $bootloader so that it won't boot from any disk. Does that mean the bus is broken?
Also for that sparcstation make sure you have that 180GB drive well cooled. There is nothing quite like spending $2000 on a drive including controller card and having it fail in six months.
Why would I need a new controller? There's one built in. Granted, it won't give me much performance, but the point I was making is that it will work, unlike plugging your miracle 150 gig IDE crap-o-rama into a 4-year-old controller. In any case, cooling isn't a problem if you attach the disks externally in a proper enclosure. Which in fact virtually all real computers also support out of the box, and something most IDE controllers still don't. Probably because the 2-device per channel limitation makes external attachment fairly pointless given the cost of reasonable quality cables.
The moment I can justify spending over five times per GB for something inherently less reliable and not appreciably faster I'll talk to you.
I wasn't claiming that buying a 180GB disk was necessarily the right solution. Like any first-generation disk it's bound to be slow and less reliable. I was instead indicating that SCSI is infinitely more extensible and flexible than IDE, without having a new hackjob "standard" every six months. Personally I would be more likely to use 16 18GB disks in a 0+1. Similar capacity, much faster, and much more reliable for not much more money. But hey, the fact is that I have a choice and either will work if I do it right.
I don't know about you but I know four 7.2k 75GB IDE doing RAID 0 (striping if you didn't know) are always going to be faster than one 180GB Ultra160 even in an Adaptec 29160 in a 64 bit PCI slot.
Hmm, RAID 0 with consumer-grade disks. Fucking brilliant. Can I hire you to run my production database servers? By the way, downtime is 300 large an hour. And if the outage is due to your choice of inadequate equipment that deviates substantially from accepted industry best practices, you *will* pay it out of your own pocket.
Oh wait a minute there are no consumer grade PCs with 64 bit PCI slots.
Nobody who actually needs a computer uses peecees anyway. And peeceei is hardly the only bus in the world. Get over yourself; the peecee is dead because the technology is grossly inferior and was misdesigned (if designed at all) from day one. I really get a kick out of all you people who jabber on about how great "commodity" hardware is and how cost-effective it is to stick it to the makers of Real Computers by buying cheap peecees and using them in ways they were never intended. You people are complete idiots; if you have a job in IT my hat is off to your boss; he or she must be the biggest fuckwit the world has ever seen. Repeat after me: peecee hardware is fundamentally and entirely inferior to Real Computer equipment and is unsuitable for all but the least important of tasks (personally, I wouldn't even use a peecee for games - even relaxation is too important to let a flaky CPU or cosmic-ray-ified DIMM interrupt it). It is not cost-effective. It is not "power to the people." It is a bunch of shit that costs more than it should and doesn't deliver even the minimal results it promises. Get with the fucking program. How can a person be as stupid as you? I'm seriously baffled as to how you survived childhood.
> How many people keep only one copy of any really
> important file? If *I* was storing the entire
> human consciousness on magnetic media *I* would
> certainly keep more than one copy.
Sounds reasonable, unless you're trying to hide humanity from evil aliens. Of course, the first generation won't be robust enough to survive the banging around.
> Of course, keeping them in sync would be a real
> pain....
Oh it was way worse than that! He kept losing the finger storage pods.
I am for the complete Trantorization of Earth.
I believe that would be a word. A byte is 8 bits. Period.