Linux 2.3.48 Released

Turambar let us know that Linux 2.3.48 is out. If you know where to get it, go for it. If you don't know, you probably shouldn't poke at it. Gotta be getting close to the release by now, right? I gotta say I'm really looking forward to the integrated PCMCIA getting released and hopefully put into woody.

Comments

Interesting? How?

[E1ven]

IS there anything particulary noteworthy about this kernel?

ObRant: I suggest that Slashdot creates a software release section, as they have with BSD, and Your Rights Online, and move these stories there. We only get 10 or so stories a day, I do not want them used to point out every development kernel...

Freshmeat whiners, let me see if I get this

[Stalemate]

OK, here goes:

1) Slashdot != Freshmeat (I'll go along with this one)

2) Slashdot should not post any of the same stories as Freshmeat.

OK, now number 2 lost me. I don't see what the reasoning is that makes #2 follow from #1. Let me use this argument in some more examples.

Slashdot != StarWars.com, so no stories about star wars movies should be here.

Slashdot != Suse != RedHat != Debian, so no stories about these should be on slashdot either.

Slashdot != GoHip.com, so no information that is at GoHip.com should be on Slashdot. Since GoHip.com has a license agreement that tells what their "browser enhancement" does, it should not be on Slashdot.

So, do you people think that slashdot should only contain news about things that aren't on any other website? Most of the news posted here has a link to some other site, so most of the news could be found somewhere else on the internet.

Personally, I look at Slashdot as a repository for news. It gives me one place to look instead of having to go to 100 different pages to find interesting stories. I just don't get why you people are so upset about this. I don't like every little story that pops up on here, but I don't have to read every one of them either.

I think a lot of the problem is in the assumption that all Slashdot readers are also Freshmeat readers. I haven't heard anyome come right out and say this, but it is the impression that I get.

OK, I'm finished now.


--

Some of it is already stable...

[Ami Ganguli]

DevFS, for example, has been stable for ages and Richard has dutifully been releasing updated patches against current kernels. It was just a matter of convincing Linus that it was the `Right Thing'.

The softnet stuff is, in my mind, too radical a change for a feature freeze, but if it's really as good as people say then it might be worth it. I'm sure it will push the stable release back a month or so.

The most exciting new feature for me is the Logical Volume Manager included in 2.3.47. I've spent a lot of time administering AIX systems and the LVM is a Godsend for the harried system administrator. I don't know yet what the Linux LVM can do, but on AIX you can expand volumes while the system is running. I've heard that on HP you can shrink volumes as well. Even if the Linux LVM doesn't have all the bells and whistles, you can bet they will appear quickly now that the feature is in the mainstream kernel.

It looks like 2.4 will be a really nice release all-around. Not a lot of radical changes, but lots of performance improvements and useful little things.

News from the Linux frontlines

[Jesus Christ]

Torvalds begins work on Linux 2.3.48.9.2.7.43, possibly
Posted by CmdrTaco on Sunday February 27, @10:36AM
from the rob-sucks-tarballs dept
Linus Torvalds, creator of Linux, accidentally hit his keyboard with his elbow today. We have yet to receive confirmation that the resulting code will be be included in the next development kernel, but we can never be too sure. Here is the code in full:

kjnlkmf ,m58u45knm ,9804
8v793oy5n9*(&V(*N&

This won't compile under GCC, so we can only assume the code is pretty experimental. Look for the tarballs to be released this evening.

Torvalds comments, "What? Oh, yeah, I accidentally hit my keyboard with my elbow when I reached to get my tea. What? Is it part of the new kernel? You're kidding, right?"

We'll update the article as soon as we get more information. The Linux world hasn't been in such frenzied anticipation since the release of kernel 2.3.48.9.2.7.42, which was about ten minutes ago.

Interview: Alan Cox farted
Posted by Hemos on Sunday February 27, @10:34AM
from the whats-that-smell dept
Linux guru and hacker-extrodinaire Alan Cox farted earlier today. What do you think this says about the future of Linux development? Alan's ass will respond to the highest moderated posts later this week.

ESR and JonKatz to participate in "Zealot Deathmatch"
Posted by Roblimo on Sunday February 27, @10:33AM
from the die-bitch-die dept
Open source proponent Eric S. Raymond and Slashdot nutcase JonKatz are reportedly organizing a "Zealot Arena Deathmatch" to raise money for the Apache Software Foundation. The fight is expected to be a tough one, because while Katz is genuinely insane, ESR has the power of girly, elfish looks. A spokesman from Apache says that, "while we don't encourage violence, we'll do anything for money."

VA Linux aquired by Klingons, Rob bows down to new alien masters
Posted by emmett on Sunday February 27, @10:32AM
from the star-shit-enterprise dept
VA Linux Systems, owner of Andover.net, owner of Slashdot.org, owner of Rob's ass, was officially aquired by the Klingon Empire earlier this morning. The Klingons, who have recently taken over Kellogs, GM, and Disney, are looking forward to absorbing more major corporations in the near future. The US Government is discussing investigating the Klingons for holding a monopoly over "every aspect of our lives", to which the Klingons responded, "Puny human scum! I will crush you like a bug and feast upon your steaming entrails." Finally, some competition for Microsoft!

Red Hat and VA stock at all time high!
Posted by CmdrTaco on Sunday February 27, @10:31AM
from the i-am-so-rich dept
Dude, have you heard the market reports today? I am so fucking rich! If this keeps up, I'll be able to stop doing this Slashdot crap! Hell yeah!

I am the Lord.

7 minutes to boot, with a full fs check: 46gb used

[Convergence]

These are normal ext2fs filesystems.. Even despite the fact that e2fsck isn't fully parellelizing the checks, I have clocked a full bootup at 7 minutes.

Among the other partitians, I have a 23, 16, and 20gb partitians. (on seperate drives). I have about 75gb of disk space total, with 46gb of that currently in use (723484 files/directories). My trick is twofold:

First, the default inode allocation is a bit insane.. Inodes are 128 bytes each and there's one inode for every 4kb of diskspace. So for every 10gb of disk, the default format uses 320mb of inodes, capable of storing over 2.5 million files! And e2fsck has to scan each and every byte in each and every one of these inodes. So why not drop that to 1/4, or one inode for every 16kb? Then for every 10gb of diskspace e2fsck only has to scan 625,000 inodes or 80mb worth. Can you say 4 times faster? :)

Some might claim that they could run out of inodes with an allocation that small? Unless the server has lots of small files (mail, news, proxy), its highly unlikely that you'll have even 500,000 files on the whole thing. You can get this info very quickly by using 'tune2fs -l /dev/foo'.

If you're like me and you notice that you're using only 1/8 or even 1/15 the total number of inodes, and you don't the file charactaristics [number of files, directories, average size, ...] that you're going to store on that partitian to change radically. Then get rid of 3/4 of them and speed up the fsck.

In my case, I've got a total of 4.2 million inodes, with only 700k used, had I formatted normally, I would have had around 19 million. (multiply by 128 bytes/inode to see how much storage they need, and how many hundreds of megabytes e2fsck would need to scan.) I also tuned my partitians seperately. Based on how they were currently being used and on the risk of that changing radically. (For instance, /tmp has the stock 4kb/inode.. I never know if I'll suddenly stuff lots of small files in there.)

Ok.. That's trick #1.. The second trick is the default blocksize. Changing this speeds up every filesystem operation, from allocation to fsck to reading to writing to unlinking. This trick does waste more diskspace.

Normally, ext2fs allocates storage in 1kb blocks. But changing that to 2kb has many advantages. First, a file requires only half the number of drive transactions, which will improve speed. Second, since all allocations are now done in 2kb sizes, I can allocate (and remove) twice as fast. Finally, due to the subtletly in I, II, III blocks that form the allocation BTREE, (These are diskblocks which point to diskblocks that point to diskblocks containing data.) Having twice the size of allocation means that the btree has twice the fanout AND each leaf holds twice the data. I'm not sure how much impact this factor has on speed.

For those of you who don't know how ext2fs inodes are layed out.. They're actually curious.. The inode itself points to the first 12 blocks of the file directly (normally the first 12*1024). Then it points to an I block that contains pointers to the next blocks in the file. (normally, the next 1024/4 = 256 blocks, or 256kb). Then there's the II block, which contains pointers to I blocks. Finally, there's the III block that contains pointers to II blocks. You don't need an IIII block because with only an III block, you can handle files up to about 16tb, which is larger than the maximum possible filesystem size.

Now, the reason to get into this big long explanation is to make a fascinating point about diskspace usage.. If you have a blocksize of 1kb, then files less than 12kb in size don't require any I blocks. While if you have a blocksize of 2kb, files less than 24kb in size don't require any I blocks.

So, if your filesystem has files between 12kb and 24kb in size, if you compare the disk usage between a filesystem of 1kb blocksize and 2kb blocksize, The worst you could do is waste an extra kilobyte in the last block, but that wasted diskblock is made up for the fact that you don't have an I block. :)

And that's the worst you could do. In fact if you have luck, you can actually come out pretty far ahead! Formatting with a blocksize of 2kb may actually waste LESS space AND require fewer seeks! :)

Now combine this with the tidbit that the average file tends to be around 13kb. If the majority of the files on the partitian are between 12 and 24kb in size, you can't lose with this!

As files get bigger than 24kb, the relative size of wasted space in the last block becomes much less relevant, (for files around 24kb, the maximum percentage of wasted space is 2kb/24kb ~~ 8%. For 128kb, its 2kb/128kb ~~ 1.6%) So a 2kb blocksize has a decreasing affect on wasted space, while at the same time increasing the bandwidth and speed of handling large files. So at files >24kb in size, you start winning, for files >1mb, you start winning a whole lot.

If the partitian is only intended for very large files, (Ones where any wasted space in the last block is irrelevant with respect to the total size.), then a 4kb blocksize makes perfect sense. I don't suggest this idea too strongly because its not as applicable as a 2kb blocksize.

Those are just a few characteristics of ext2fs with regard to blocksize. There's no magic bullet for speeding up ext2fs, but depending on how the filesystem is used, you can frequently speed it up. Look at your drive, the average file size, and the filesize distribution. ``find /foo -size +12k -size -24k | wc ; find /foo | wc ; find /foo -size +24k | wc ; find /foo -size +128k''. Then decide if changing the blocksize makes sense.

For my personal system, the overhead of increasing the blocksize to 2kb is around 3-7%, 3% in most places and 7% where there tend to be many small files (/home/http).

Closing remarks:

If you use both tricks together, they almost cancel themselves out. The overhead of having 1 inode for every 4kb is 128b/4kb, or about 3%, if you format with 16kb/inode, the overhead drops to .75%. You save 2.25%. And as it just so happens, the overhead of the bigger blocksize is loss of about 3%. So overall, you break even; within one or two percent of the origional disk usage. This is how I formatted most of my system.

And if you actually need millions of 4kb files, well, unjourneled ext2fs is not the filesystem I would reccomend.

So, a quick summary. My system takes 7 minutes to boot. It has 723484 used inodes, out of a total of 4.2 million inodes. I have 46gb of drivespace used, out of a total of 75gb. A boot with a full filesystem check takes 7 minutes and requires reading about 500mb worth of inodes. A boot without a full fsck takes one minute (about 20 seconds of that just mounting).

Had I formatted it normally, I would have saved 500-1500mb (1-3%) of drivespace, had 18 million inodes. Fsck times would probably take 4x-8x as long and requrie reading about 2.3gb worth of inodes.

I considered the trade well worth it for me, and I suspect that it would be well-worth it to many other people. (Excluding those who's boxes have multi-year uptimes. :)

[PS: I may turn this into a mini-faq.]