Slashdot Mirror
Linux 4.0 Has a File-System Corruption Problem, RAID Users Warned
An anonymous reader writes: For the past few days kernel developers and Linux users have been investigating an EXT4 file-system corruption issue affecting the latest stable kernel series (Linux 4.0) and the current development code (Linux 4.1). It turns out that Linux users running the EXT4 file-system on a RAID0 configuration can easily destroy their file-system with this newest "stable" kernel. The cause and fix have materialized but it hasn't yet worked its way out into the mainline kernel, thus users should be warned before quickly upgrading to the new kernel on systems with EXT4 and RAID0.
I'll stick with Windows Vista, thanks.
this is obviously some strange usage of the word "stable" that I wasn't previously aware of.
RAID 0 is unstable to begin with. Medium case scenario here (for legitimate use) is some data gets corrupted on a compute node. Run the program on two nodes; if you get the same result on both, that result is probably fine. If you're running RAID0 on any filesystem that isn't temporary or at least easily replaceable, you're doing it wrong.
If your running a brand spanky new kernel, with data you do not care about why an old FS. Plenty of newer better FS's to choose from.
No sir I dont like it.
This is the new 4.0 kernel, A Major version update , less than a month old, that most Linux systems will not have yet ...and the issue has already been patched
Bleeding edge builds get what they expect, stable builds don't even notice
Puteulanus fenestra mortis
md raid. The bug was in commit md/raid0: fix bug with chunksize not a power of 2 causing, you guessed it, a bug with a chunksize not a power of two. I guess "fix" was a bit diversionary.
The real problem was a macro modifying arguments that were later expected to be the unmodified version.
I have 4 drives in a RAID 10, so two RAID 1 arrays of two drives each combined together in a RAID 0. I did it mostly because I can add new drive at any time and just chain them onto the RAID 0.
No. There was a minor bug introduced at 3.14. The patch to fix that, completely different issue, went into 4.0 and caused this corruption issue.
"National Security is the chief cause of national insecurity." - Celine's First Law
Tunneled down into the articles, http://git.neil.brown.name/?p=... has the patch. I'm building a system with 4.0.4 right now so this was material to me
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Or just use a power of 2 chunk size?
What idiot configuration did someone have to have to trigger this bug?
That fix is actually in the wrong place. The fix for that is tracked in kernel.org's bugzilla # 98501. I'm not linking directly as linking to bugzilla tends to place too high a load on those systems. It's impolite.
Neil Brown said that he'd push the fix to Linus "shortly" at 2015-05-20 23:06:58 UTC. I still don't see the fix in Linus' tree.
Watch for a fix titled "md/raid0: fix restore to sector variable in raid0_make_request"
Or it could work just fine. RAID 0 is not dangerous, you may just as well loose your data even if you only use a single drive. Hard drives and SSDs don't go bad that often that it's a problem.
Well, there goes that slogan.
Escher was the first MC and Giger invented the HR department.
Losing data goes with the territory if you're going to use RAID 0.
In particular, RAID 0 combines disks with no redundancy. It's JUST about capacity and speed, striping the data across several drives on several controllers, so it comes at you faster when you read it and gets shoved out faster when you write it. RAID 0 doesn't even have a parity disk to allow you to recover from failure of one drive or loss of one sector.
That means the failure rate is WORSE than that of an individual disk. If any of the combined disks fails, the total array fails.
(Of course it's still worse if a software bug injects additional failures. B-b But don't assume, because "there's a RAID 0 corruption bug", that there is ANY problem with the similarly-named, but utterly distinct, higher-level RAID configurations which are directed toward reliability, rather than ONLY raw speed and capacity.)
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Based on the commit fixes, it's in a function called raid0_make_request, which is only used in raid0.c
raid 10 is in raid10.c, so it doesn't use this function.
The bug is based on the fact that a macro "sector_div" modifies it's first argument [and returns the remainder]. I've removed the obligatory backslashes for clarity:
# define sector_div(n, b)( /= (b);
{
int _res;
_res = (n) % (b);
(n)
_res;
}
)
This is used in some fifty files. Some just want the remainder [and they don't want the first arg changed so they do]:
sector_t tmp = sector;
rem = sector_div(tmp,blah);
This is effectively what the code wanted, but the actual fix was to do a restore afterwards:
sector_t sector = myptr->sector;
...
rem = sector_div(sector,blah);
...
sector = myptr->sector;
... // use sector [original value only please ;-)]
The last line to restore sector with the original value was the fix.
They should do a full code audit as their may be other places that could be a problem. I've reviewed half the files that use this macro and while they're not broken, some of the uses are fragile. I paraphrase: "sector_div considered harmful"
What they really need are a few more variants which are pure functions that could be implemented as inlines:
rem = sector_rem_pure(s,n)
s2 = sector_div_pure(s1,n)
Or, a cleaner sector_div macro:
sector_div_both(s,n,sector_return,rem_return)
Like a good neighbor, fsck is there