Serious Bug In 2.4.15/2.5.0

John Ineson writes: "There is a bug in the latest kernel releases, that causes fs corruption on umount. A lot of people have already been hit by this, so for now I suggest you hold fire on booting those new kernels. More dead-duck than greased-turkey. Two possible fixes are being discussed on linux-kernel." Colin Bayer adds links to a story at the Register and Al Viro's fix. Update: 11/25 00:39 GMT by T : Tarkie writes "Linux 2.4.16-pre1 is out, as detailed at NewsForge. If you've been having the filesystem corruptions, might be worth a try so that 2.4.16 can be out ASAP!"

Comments

Re:Filesystems

[MShook]

You're correct, it is regardless of filesystem. If you happen to be running 2.4.15 or 2.5.0, just remember to force a fsck for the next reboot (shutdown -F) that's the only way to clear the fs because it will be marked clean even if it's not). Right now, the developpers don't know how reseirfs would deal with this bug...

A fix...

[phyngerz]

You can find Andrea Arcangeli's fix at:
ftp://ftp.us.kernel.org/pub/linux/kernel/people/an drea/kernels/v2.4/2.4.15aa1/00_iput-unmount-corrup tion-fix-1

Re:Does anyone know...

[Colin Bayer]

This bug was introduced when the kernel coders were trying to fix a bug that existed earlier (but, AFAIK, didn't cause fs corruption). It was introduced in pre9, but the final kernel was released within a few hours, so I guess nobody caught it in time.

A Workaround

[kanelephant]

Al Viro gave this comment and workaround on lkml.

Breakage happens when you umount filesystem (_any_ local filesystem, be it ext2, reiserfs, whatever) that still has dirty inodes.

As a workaround - sync before umount (and don't boot unpatched 2.4.15/2.4.15-pre9 again, obviously).

IOW, if you are running 2.4.15 - build a patched kernel, install it and do the following:
* switch to single-user
* sync
* umount everything non-busy
* remount the rest read-only
* turn the thing off
* boot with patched kernel or with anything before 2.4.15-pre9

The filesystem corruption can be fixed by a forced fsck. (The fsck must be forced since the filesystem is marked clean.)

Re:A Workaround

[kanelephant]

sorry I didnt make that clear. If you follow the above advice you should not get any filesystem corruption. The last line is what to do if you have already got a corrupt filesystem!

the patch from the kernel list

[MentlFlos]

I hope /. dosent mangle this up too bad, but if it does:
http://marc.theaimsgroup.com/?l=linux-kernel&m=100 658174003122&w=2

List: linux-kernel
Subject: Re: 2.4.15-pre9 breakage (inode.c)
From: Linus Torvalds
Date: 2001-11-24 5:55:42
[Download message RAW]

On Sat, 24 Nov 2001, Andrea Arcangeli wrote:
>
> --- 2.4.15pre9aa1/fs/inode.c.~1~ Thu Nov 22 20:48:23 2001
> +++ 2.4.15pre9aa1/fs/inode.c Sat Nov 24 06:30:20 2001
> @@ -1071,7 +1071,7 @@
> if (inode->i_state != I_CLEAR)
> BUG();
> } else {
> - if (!list_empty(&inode->i_hash) && sb && sb->s_root) {
> + if (!list_empty(&inode->i_hash)) {
> if (!(inode->i_state & (I_DIRTY|I_LOCK))) {
> list_del(&inode->i_list);
> list_add(&inode->i_list, &inode_unused);

I have to say that I like this patch better myself - the added tests are
not sensible, and just removing them seems to be the right thing.

Linus

The discussion isn't over

[Carnage4Life]

The last post in that thread is this one by Andrea Arcangeli sometime this morning and from the looks of things (if you read the entire thread) there is conflict between Alexander Viro and Andrea on which is the better solution.

Linus saying he prefers a patch on an initial viewing isn't the end of the situation for now. I'd suggesting waiting a week and revisiting the thread to find out what the final word was.

Patch download here

[DeeKayWon]

The mailing list converted tabs into spaces, causing patch to choke. Get the patch here.

Re:This is why I use FreeBSD

[FattMattP]

This is not a stable kernel, as there is no development tree to iron out all the bugs.

Well, I disagree with you there. The way things have always been done, and the way we tell people that they are done is that x.<even#>.x is a stable kernel and x.<odd#>.x is a development kernel. Once you make that second number even, then it's interpreted by the whole community as stable, whether there's a development kernel or not, because that's what we've been taught and that's the way Linus has always done it. Continuing to put new features into the 2.4 tree rather than opening up 2.5 has led us to this unfortunate position. Hopefilly, in the future, the development tree will open as soon as the next major stable release is made and we can avoid things like this.

Re:If you are already running it...

[PeterM from Berkeley]

I wouldn't do what this guy says.
You're pretty much guaranteed to corrupt your
filesystem this way. Probably nothing fsck
couldn't fix, but still.

Other posters have suggested that you use
"shutdown -F" after running "sync",
and rebooting into a NON-2.4.15 kernel.

"sync" will write all the unsaved data to
the disk, and "shutdown -F" will cause
an fsck to start after rebooting.

PM

Re:Please spare us

[Shane]

First: This linux bug does not the loss of the ENTIRE FILE SYSTEM. It leaves .lock files with invalid INODES which can be repaired by manully running fsck. As to you're challenge, these are just a few corruption problems with windows 2000 that I found doing a simple search on www.microsoft.com.

http://support.microsoft.com/support/kb/articles /Q 268/8/97.ASP

http://support.microsoft.com/support/kb/articles /Q 258/0/75.ASP

http://support.microsoft.com/support/kb/articles /Q 273/2/45.ASP

http://support.microsoft.com/support/kb/articles /Q 298/9/36.ASP?LN=EN-US&SD=gn&FR=0&qry=file%20system %20corruption&rnk=16&src=DHCS_MSPSS_gn_SRCH&SPR=WI N2000

http://support.microsoft.com/support/kb/articles /Q 261/1/22.ASP?LN=EN-US&SD=gn&FR=0&qry=file%20system %20corruption&rnk=19&src=DHCS_MSPSS_gn_SRCH&SPR=WI N2000

http://support.microsoft.com/support/kb/articles /Q 255/5/69.ASP?LN=EN-US&SD=gn&FR=0&qry=file%20system %20corruption&rnk=23&src=DHCS_MSPSS_gn_SRCH&SPR=WI N2000