Long Block Data Standard Finalized

An anonymous reader writes "IDEMA has finally released the LBD (Long Block Data) standard. This standard, in work since 2000, increases the length of the data blocks of each sector from 512 bytes to 4,096 bytes. This is an update that has been requested for some time by the hard-drive industry and the development of new drives will start immediately. The new standard offers many advantages — improved reliability and higher transfer rates are the two most obvious. While some manufacturers say the reliability may increase as much as tenfold, the degree of performance improvement to be expected is a bit more elusive. Overall improvements include shorter time to format and more efficient data transfers due to smaller overhead per block during read and write operations."

Comments

Re:Higher Reliability?

[5pp000]

The longer block sizes add reliability because the error correcting codes have more to work with at a time (more data bits, but also more ECC bits).

As for wasted space, that's under the filesystem's control, not the drive's.

Error correction better over larger blocks

[EmbeddedJanitor]

If you're working with a certain number of ecc bits per data bit, then the number of corrections you can perform increases with an increased data block size. Oversimplifying, just for explanation here:

Let's suppose you can fix one error per 512 byte block or 6 errors per 4096 byte block. Intuitively that might seem like a step back because 6/8 is smaller than 1, but that is not so. If you have 512-byte blocks and get two errors in a 512-byte sequence then that block is corrupt. However if instead you're using 4096 byte blocks then a 512-byte sequence within that block can have two errors since we can tolerate up to 6 errors in the whole block.

Or put another way, consider a 4 k sequence of data, represented by a sequence of digits dependent on the number of errors in each 512 bytes. 00000000 means no errors, 03010000 means 3 errors in the second block and 1 in the fourth block (ie a total of 4 errors in the whole 4096 bytes). With a scheme that can fix only one error per 512 bytes, the block with 3 errors cannot be corrected (because 3 > 1), but in the system which fixes up to 6 errors per 4096, the errors can be fixed because 4 6. This means that the ECC is far more reliable.

Re:Why 4096?

[42forty-two42]

Using 4MB blocks for everything would kill memory performance - and more specifically, mmap performance. Each library loaded in your system would require at least 4MB of ram - probably more, as they have code, data, and zeroed data segments. Additionally, each process would require another 4MB*n. There's no gain for doing this either, except under specialized circumstances, as the OS can already request a batch of sectors from the drive in one operation.

Re:Oh great

[avxo]

Now when I want to update just 256 bytes, instead of reading 512 bytes, changing 256 of them, and writing 512 back, I now have to do this with 4096 bytes. So I end up transferring 3584 more bytes than I otherwise needed to.

So, your O/S requires that you issue all read and write operations using the hard drive's native block size? That must suck. What else must you do? Setup DMA manually in your app? Solder a microcontroller onto the board perhaps? Sarcasm aside, you seem to have a fundamental misunderstanding of what this change achieves, who it will affect, and how. Other posters have addressed those very issues eloquently, so I won't go into that.

They really could do this transparently. Let the driver write anything in any range.

Sorry to burst your bubble but it already is done transparently. The O/S lets you write anything -- from a single byte, to gigabytes -- transparently; all you do is tell the O/S read n bytes of file F so and so into buffer at x, or write m bytes from buffer at y into file F, which is the interface that 99% of programmers use. And after what you wrote above, I find it hard to believe that you are writing the specialized software, low-level drivers and/or controller microcode that could potentially be affected by this change.

Re:Why 4096?

[Scott Wood]

No, x64 and ARM both use 4K pages (though ARM has 1K subpages that you can set permissions on individually). Alpha and sparc64 use 8K pages, though.

blocks and clusters

[ceroklis]

To all the posters complaining about the loss of space when they will be forced to use 4096 instead of 512 bytes to store their 20 bytes file:

• The cluster size (unit of disk space allocation for files) need not be equal to the physical block size. It can be a multiple or even a fraction of the physical block size. It is fairly probable that you are already using 4K clusters (or bigger), so this will not change anything. This is for example the case if you have an NTFS filesystem bigger than 2GB.
• Not all filesystems waste space in this manner. Reiserfs or EXT3 can pack several small files in a "cluster" .

Re:What about the MBR?

[ottffssent]

The word you're looking for is GPT. It has nothing to do with 4k hardware sectors, but it does support up to 128 partitions. Which ought to be enough for anybody (says the man with a 1 average number of partitions per disk in his household).