Slashdot Mirror
Oracle Adds Data-integrity Code To Linux Kernel
jazir1979 writes "ZDNet is reporting that Oracle has added code to the Linux kernel for ensuring data integrity. The code has been developed in partnership with Emulex and was recently accepted into the 2.6.27 kernel release." According to the article, "The code adds metadata to data at rest or in transit, to monitor whether that data has been corrupted. It helps make sure that I/O operations are valid by looking at that metadata — which acts as verification information — exchanged during data transmissions."
How badly does this affect performance?
The Year of Linux on the Database? Nah, that happened a long time ago.
My blog
It adds a 2nd layer of metadata that is used to verify the first layer of metadata wasn't corrupted so you can be EXTRA confident that your original data was actually handled correctly.
No sig for you!!
It pre-corrects a future corruption in the as-yet-unimplemented third layer of metadata. Kernel developers have decided to add the third layer and accept the patch on the grounds that the corruption might still have occurred even if Oracle hadn't said anything.
Block integrity patches were discussed in excellent article on LWN in July 2008. Kernel 2.6.27 was released in October 2008. This is old news.
:wq
You've invented the Checksum
On a more serious note (yes I did RTFA), somebody please explain where this fits. Other than network or disk errors (which generally already have error detection schemes), I'm not sure what the target problem is that this is supposed to fix. The article says "the code helps maintain integrity as data moves from application to database, and from Linux operating system to disk storage", that it checks I/O operations, and that "code contribution includes generic support for data integrity at the block and file-system layers". That's still not clear what they think the problem is. Don't most of the modern file systems already check data operations?
I don't know where it fits either, but ZFS and eventually BTRFS actually have checksums at the block level, and can heal over corrupted blocks using redundant copies whose checksums do work. That alone is enough reason to use ZFS for a file server, but similar improvements could be made inside the Linux stack without a new filesystem on top. However ZFS' reliability also comes from copy-on-write updates which is not trivially installed into an existing filesystem.
Sam ty sig.
Integrity is a security principle, and that is the sense that they're using the word in the summary. It's pretty much the only definition of the word that makes sense in a computing context. More precisely, we're talking about confidence that the data stored in the system is the same as the data retrieved at a later time. The only difference between this and a more cryptographic sense of the word is that this doesn't attempt to guard against malicious attacks if an adversary had offline access to the disk. (Or so I presume, having not RTFA'd).
Evidently, the key to understanding recursion is to begin by understanding recursion. The rest is easy.
I'm not certain but it appears to be checksumming data while it is moving around the kernel after a write or read call is made.
Seems like something that should be handled in hardware with ECC, but what do I know.
I'm not certain but it appears to be checksumming data while it is moving around the kernel after a write or read call is made.
Seems like something that should be handled in hardware with ECC, but what do I know.
Kernel bugs can cause data to get corrupted and hardware ECC won't correct that. Likewise with transfers from memory to disk. Ultimately it'll need to be a hardware/software thing but the software portion is needed as well.
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
The purpose of a journal is to make sure that operations either happen or they don't happen - i.e. you don't leave the filesystem in some half way state if the power goes out.
It doesn't verify the actual data written or anything.
One of the problems that this is supposed to detect is blocks getting written to the wrong place or being read from the wrong place. I think it's one of those rare problems that stops being quite so rare when you have huge amounts of data stored on cheap hardware.
Mea navis aericumbens anguillis abundat