HDD Manufacturers Moving To 4096-Byte Sectors

Luminous Coward writes "As previously discussed on Slashdot, according to AnandTech and The Tech Report, hard disk drive manufacturers are now ready to bump the size of the disk sector from 512 to 4096 bytes, in order to minimize storage lost to ECC and sync. This may not be a smooth transition, because some OSes do not align partitions on 4K boundaries."

Comments

Factors of 10

[suso]

Why not just move it to 1000 byte sectors, then we could minimize the space lost to advertising.

(Note to accuracy nazis, this is meant to be funny)

Re:Factors of 10

[rolfwind]

If they want to use base10 the first thing they should do is respecify a byte to be 10bits.

How about leaving the word byte alone and using another, distinct group of letter to do the job? Respecifying only confuses the issue, even those who know, because you're still be working with two different definitions in the same field for a long time.

Re:Factors of 10

[drainbramage]

Mine goes to 11.

So only XP is out of luck?

[7o9]

According to the Anandtech article, only the pretty much end-of-life Windows XP is out of luck. Linux, OS X and modern Windows versions all work ... Non news?

Re:So only XP is out of luck?

[gbjbaanb]

whoooooo. WinXP is end-of-life? You'd best tell that to all the millions of users (including big businesses) out there.

What that's you say? Upgrade to Windows 7 and use its perfectly infallible XP mode?

Ah, I understand now. Hi Bill, how's Steve getting on, still a bit sweaty and concerned he's not selling enough?

Re:So only XP is out of luck?

[AlecC]

Why wasn't it done before? Sheer inertia. 512 bytes has been the HDD sector size since time immemorial. Some HDDs in the past could be re-sectored to different sizes, and sometimes were. I did it on one generation of disks to optimise storage for a particular reasons, but it didn't work reliably on the next generation of disks, so I dropped it. Some disks had a sector of 1080 bits, I think to handle the 33rd bit on IBM System/38.

What is the advantage? Every sector has a preamble, a sync mark, a header, the payload data, ECC, and postamble. These can amount to tens of bytes, especially as you have stronger ECC for weaker signals. By having fewer sector, you recover this space from most of the sectors. This could easily add 10% to the capacity of a drive. And, as posted elsewhere, most OSes do 4K transfers most of the time.

Re:So only XP is out of luck?

[kill-1]

The new hard drives will have a compatibility mode. It will be slower though because it has to read-modify-write behind the scene.

Re:So only XP is out of luck?

[lorenlal]

Eventually, you have to put a line in the sand. If you push off the deadline, manufacturers will still take their time, and they'll be in the same place 9 years and 11 months from now.

Example: IPv6.

Re:So only XP is out of luck?

[drinkypoo]

Except that pretty much every OS in use now has IPv6 support.

Except that name resolution is broken for IPv6 on Windows XP, which is the operating system not supporting 4k sectors that people are complaining about... so IPv6 was a super shitty example for you to try to defend.

Re:So only XP is out of luck?

[petermgreen]

Sidestepping your ignorance or deliberate deception on periods of typical Linux support contracts
He didn't say if he was stating lengths from release or length of overlap (to me the latter is the more important figure)

Who cares if support goes out 10 years
It's 10 years (5 mainstream, 5 extended) minimum from release, 7 years (2 mainstream, five extended) minimum overlap between releases and 2 years (all extended) minimum overlap if you skip a release. IIRC XP will have exceeded all of those.

if you can't buy a new hard drive that will work with the OS?
These "advanced format" drives will work fine with XP, they just require a little extra effort (either using a third party paritioning tool, fitting an extra jumper to change the sector mapping or using the WD tool to realign the partitions after setup) if you want maximum performance. Besides I can still by PATA drives so I doubt these drives will be the only ones on the market any time soon.

Similarly if I go to almost any major vendor I can still get computers and computer parts that are supported with XP, some of the consumer crap isn't but virtually every buisness machine and seperately sold peice of hardware i've seen lists XP as supported.

It's articles and comments like this that give me difficulty discerning what exactly Microsoft "support" entails.
For most of us the most important part of the support is continuation of security updates (though they have occasionally refused to release one that they really should have released by claiming that it's not nessacery in a default environment), I would be very uncomfortable running exposed systems (and I coun't any machine used to browse the web as exposed) on an OS that was no longer getting security updates.

There is also problem support and non-security hotfixes (free if created while in mainstream support, pay for if created during extended support) but for most of us these are fairly irrelevant.

As I alluded to above though what really matters is support from third party vendors, I can still buy the latest hardware and run XP on it with no problems, just try doing that with a comparable aged linux distro (e.g. debian woody).

WD is already shipping them

[daha]

There are certain models of the Western Digital Caviar Green drives that are already shipping with a 4K sector size, such as this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136490

Re:Care to provide examples?

[andrewd18]

I realize this is Slashdot, but both of the articles linked talk about the affected operating system. Hint: It shares an ending with a colloquial name for urine.

Re:Care to provide examples?

[tepples]

I realize this is Slashdot, but both of the articles linked talk about the affected operating system. Hint: It shares an ending with a colloquial name for urine.

Wii? PSP?

disable ECC?

[Anonymous Coward]

I heard some talks from the ZFS folks at Sun about how they were floating the idea to HD mfgr's of just disabling ECC on the drives. ZFS checksums every block, and in a RAID configuration, it would be able to transparently correct any checksum errors. I think this may have also been the motivation behind bringing triple-redundant RAID to ZFS.

The motivating idea was that this would reduce the overhead involved on ECC and gain extra space.

Thoughts?

Re:disable ECC?

[MobyDisk]

That wouldn't work with existing file systems that assume the drive does this. That's like deciding to remove the checksums from TCP and IP because a few protocols provide their own checksums. Might work in specialized cases. Probably just adds risk though for no benefit.

Re:disable ECC?

[Waffle Iron]

It doesn't seem like a great idea to me. There are a lot of different ECC algorithms and implementations. It seems to me that it would be better to let the hard drive manufacturer select one that closely matches the expected signal and noise characteristics of a particular disk drive rather than some generic algorithm in the filesystem.

Re:disable ECC?

[butlerm]

That's insane. ECC at the hardware / firmware level corrects the vast majority of bit errors transparently in a manner that is invisible to the operating system. If you took out sector level ECC, the drives would be useless in anything other than a ZFS RAID configuration, and even then performance would drop in the presence of trivially ECC correctable errors, due to the re-reads and stripe reconstructions at the filesystem level.

Drive performance would probably drop because the heads would have to stay in closer alignment without the ability of ECC to correct data read errors caused by small vibrations and electrical noise. In addition, sector relocations would probably increase because tiny flaws that do not impair the ability of a drive to write an ECC correctable sector would force the drive to remap that sector to another part of the disk.

It is a similar issue with various wire level data transmission schemes. If DSL connections did not use error correcting codes, they would suffer much higher packet loss rates than they do now, especially at distance. Most those packets would generally get retransmitted due to transport level checksum errors, but why resort to performance impairing fall back measures when the problem can be largely eliminated at a lower level?

Re:Why are there sectors?

[JordanL]

A sector on a HDD is the minimum writeable space. Think of it as a lot in a subdevelopment. If each lot is 50,000 sq. ft. on a 20 acre plot, and you move to 60,000 sq. ft. lots instead, the plot is still 20 acres, but the development now has less lots on it.

In computers, larger sectors are often better for large files, while smaller sectors are better for smaller partitions and smaller files. If a sector is 4096 bytes, and you create a 1024 byte file, it still occupies 4096 bytes on the disk, as the HDD won't write anything else but that file to the sector. If you have files that are hundreds of megabytes though, you can access the file, with minimum wastage, by using fewer sectors, which reduces thrashing and similar issues.

The discrepancy between file sizes and sector sizes is what the difference is in Windows when you view a hard drive and it displays "size" and "size on disk". "Size" is the actual file size, while "size on disk" is the amount of space the file occupies on the hard drive.

Re:Looks like 512

[butlerm]

Those are "logical" sectors, which can be different from the physical sector size. According to the Anandtech article the Western Digital hard drive model numbers that end with "EARS" use the larger, 4KB physical sector size, while presenting a 512 byte logical sector size to the operating system for compatibility reasons.

Please note, of course, that the logical sector size is a drive interface level concept distinct from the filesystem cluster or block size. Filesystem block sizes have generally been larger than the logical or physical sector size for quite some time.

Re:intelligent interfaces

[Thanshin]

Do you really believe your hard drive has 256 heads?

It had only six, at first, but we didn't know the thing about burning the stumps.

Tail packing

[tepples]

Unless HDD makers were going to create firmware, and programmers made partition formats, which address each bit individually (which itself would require an enormous amount of space... much larger than the HDD in fact), you will always be unable to live without sectors. The subdivision idea is again relevant. Imagine if every part of the 20 acre plot had to be "addressable" down to the square inch.

It's called block suballocation: store a small file in its entirety in another file's slack space. And yes, it's a "killer" feature.

Re:Care to provide examples?

[Anonymous Coward]

Solar... isssss

D:

Actually no.

[Mashiki]

Most of the drive manufactures are releasing tools to align the drives to 4k clusters so they can be used under XP. WDC already has theirs out here: WDC Adv Format Plus instructions on all of their new 1TB and higher drives on how to set them up properly. You do have to jumper them, then format them specially but the drives work fine with 4k clusters. I put one in my work machine on Saturday, works flawlessly.

*I only used WDC because that's the brand I picked up recently. I do know other companies have similar tools and jumper settings on their newer drives as well.

Characters were not always 8-bit

[klubar]

I believe that some of the early CDC machines (a company that is no longer around) had a 6-bit character. The Digital Equipment Company (DEC, alos a company that is no longer around) PDP-1, maybe the PDP-20, and some others also had a 6-bit character. The PDP's had 36-bit words, packing 6 characters into a word. And of course, the IBM machines (a company that is still around) used EBCDIC rather than ASCII (but did use an 8-bits per character). Some of the earlier (and even the 370's) IBM machines used BCD (binary coded decimal) for arithmetic (packing a number from 0 to 9 in 4 bits, with some sign and unassigned bits left over).

Also, back in the IBM JCL days, when allocating disk space for a file you could specify the number of cylinders (or tracks) that you wanted, the block size and the packing factor.

please stop spreading FUD

[Anonymous Coward]

What a bunch of misinformed drivel. That article is missing a couple of things:

firstly) The issue affects all Windows versions based on a 5.x kernel. That means Windows 2000, XP, 2003 server and Windows Home Server.

1) These drives are NOT strictly-4k-sector. The platters may be organized in 4k sectors, but the drive only talks to the OS in terms of 512 byte-sectors. And since we're discussing old Windows versions: NTFS has defaulted to using 4k (logical) sectors since its introduction, so there is NO performance penalty when using NTFS on these drives. You shouldn't be using FAT32 anyway.

2) The issue can be worked around by creating partitions with a tool that understands 4k sectors, or by re-aligning the partitions after creation/installation. If you only use a drive in those systems (i.e. no repartitioning), the drive will work as it should. Even if you create partitions that are unaligned, the drive will still work - you will only lose some performance.

3) The one genuine problem raised in the linked article comes when you want to use these drives in closed-firmware devices. In this case you still have two options: either you use the WD-provided jumper setting, or you pre-create the partitions before you insert the drive.

I fail to see what the fuss is all about.