Slashdot Mirror

Alright, Just got done with some testing of UT2004 between 2.6.23-rc1 CFS and 2.6.22-ck1 SD. This series of tests was run by spawning in a map while not moving at all and always facing the same direction, while slowing increasing the number of loops.

CFS generally seemed a lot smoother as the load increased, while SD broke down to a highly unstable fps count that fluctuated massively around the third loop. Seems like I will stick to CFS for gaming now.

--
Kenneth Prugh

My experience was quite similar. I noticed after launching the second loop that the FPS stuck down to 15 for about 20 seconds, then climbed back up to 48. After that it went rapidly downhill. This is similar to other benchmarks I've done of SD versus CFS in the past. At a "normal" load they're fairly similar but SD breaks down under pressure.

The only other thing of interest is that the -ck kernel had the WM menus appear in about 3 seconds rather than 5-8 under the other two.

Personally, I'd like things like schedulers to be pluggable. But that's just me. Or maybe not.

OMG people even have to steal comments these days?

I wrote this comment on kerneltrap.

Christ, that's incredible. Nice one "bconway".

Funny how OSS is always about 'choice' until someone has the gall to choose something other than it.

Without OSS, the only choice for PC's would be Hobson's: Microsoft, or nothing. Let me illustrate from Linus himself, commenting on the latest brouhaha over the Linux scheduler:

> As far as im concerned, i may be forced to unofficially maintain SD for
> my own systems(allthough lots in the gaming community is bound to be
> interrested, as it does make games lots better)

You know what? You can do whatever you want to. That's kind of the point of open source. Keep people honest by having alternatives.

But the the thing is, if you want to do a good job of doing that, here's a big hint: instead of keeping to your isolated world, instead of just talking about your own machine and ignoring other peoples machines and issues and instead of just denying that problems may exist, and instead of attacking people who report problems, how about working with them?

(Emphasis added.)

The difference, though, is that Microsoft doesn't always work with people who report problems; sometimes they simply ignore them forever. Apple has been guilty of the same thing. That's what makes OSS different: the demand for accountability will always be met, either with compliance or by a forked project that will comply.

Do you also go as "Charles Goodwin" or "Free Gamer". The parent was also posted to http://kerneltrap.org/node/14008 , apparently 8 hours earlier.

"If a scheduler makes games better but hurts general server performance..."

IIRC that is the reason Con together with another person, whose name I can't
can't be bothered to look up, wanted to merge plugsched to which they got a
reply along the lines of "too much choice will split contributors" or some such

And then Ingo turns around on himself, and claims something along the lines of
"Oh okay, you should work on plugsched, may be it'll get merged"

"If a scheduler makes games better but hurts general server performance..."

IIRC that is the reason Con together with another person, whose name I can't
can't be bothered to look up, wanted to merge plugsched to which they got a
reply along the lines of "too much choice will split contributors" or some such

And then Ingo turns around on himself, and claims something along the lines of
"Oh okay, you should work on plugsched, may be it'll get merged"

Anyone else notice the two pics of a young Linus were are linus1.gif and linus3.gif?
http://kerneltrap.org/files/linus2.gif

When it comes to drivers, closed isn't pragmatic. Closed drivers don't get audited. Closed drivers don't work if you upgrade your kernel (or need to run an old one), or want to run on an arch that the vendor didn't bother to compile for. And then worst of all, closed drivers enslave humanity!

Haven't learned our lesson regarding security or portability have we?

Popular binary drivers had some unresolved, severe exploits and couldn't be bothered to address them for about two years. That's just an anecdote, but illustrates that the problem is real and not just theoretical. Anecdotes aside, there are inherent problems with binary-only drivers (or binary-only anything). For the obtuse, the interview with Theo de Raadt interview with Jonathan Gray and Damien Bergamini go into more details.

Production mistakes and design flaws aside, happen. That's why we get the effect that "given enough eyeballs, all bugs are shallow". But with binary-only that also means that nearly anything, from back doors to monitoring, can be piggybacked into the blob. You'd be hard pressed to find out. And depending on the vendor for the binary also leaves you dependent on their choice of architectures - not yours, and their lifecycle timeline - not yours.

Some, like the GP, may prefer the GPL, others may prefer other open source license. Whatever. Any of them is a far cry better than no source code.

Also, remember the open source is not just a license, but a development model. Popular hardware will gain development speed and quality for the drivers. It's not like the drivers have any inherent value without the hardware. Opening up the drivers would most likely boost the sales of the hardware they use.

Haven't learned our lesson regarding security or portability have we?

Popular binary drivers had some unresolved, severe exploits and couldn't be bothered to address them for about two years. That's just an anecdote, but illustrates that the problem is real and not just theoretical. Anecdotes aside, there are inherent problems with binary-only drivers (or binary-only anything). For the obtuse, the interview with Theo de Raadt interview with Jonathan Gray and Damien Bergamini go into more details.

Production mistakes and design flaws aside, happen. That's why we get the effect that "given enough eyeballs, all bugs are shallow". But with binary-only that also means that nearly anything, from back doors to monitoring, can be piggybacked into the blob. You'd be hard pressed to find out. And depending on the vendor for the binary also leaves you dependent on their choice of architectures - not yours, and their lifecycle timeline - not yours.

Some, like the GP, may prefer the GPL, others may prefer other open source license. Whatever. Any of them is a far cry better than no source code.

Also, remember the open source is not just a license, but a development model. Popular hardware will gain development speed and quality for the drivers. It's not like the drivers have any inherent value without the hardware. Opening up the drivers would most likely boost the sales of the hardware they use.

Linus Torvalds had something to say about the evil beaches:

So what's a pasty white nerd to do? You can't go out on the beach, because the goodlooking people will laugh at you, and kick sand in your face.

I'm not bitter.

...

Beaches are overrated anyway, the sand gets into the laptop fan and soon it won't work.

So, I will ask the obvious question: Why would you bring a wifi enabled device would you bring to an area with a lot of sand and water?

How does Ingo's new CFS compare to the code Kolivas wrote? Which design is superior? Does Ingo's design actually borrow from Con's code, or does it just do more or less the same thing?

Kernel Trap has an ongoing archive of CFS-related discussion from the LKML, including this detailed email exchange betweeen Ingo and Con.

The gist of the exchange is that the two schedulers are similar, but not identical. I'm not a kernel expert and I don't know squat about schedulers, but in my opinion, the exchange seemed to indicate that CFS is superior.

How does Ingo's new CFS compare to the code Kolivas wrote? Which design is superior? Does Ingo's design actually borrow from Con's code, or does it just do more or less the same thing?

Kernel Trap has an ongoing archive of CFS-related discussion from the LKML, including this detailed email exchange betweeen Ingo and Con.

The gist of the exchange is that the two schedulers are similar, but not identical. I'm not a kernel expert and I don't know squat about schedulers, but in my opinion, the exchange seemed to indicate that CFS is superior.

How does Ingo's new CFS compare to the code Kolivas wrote? Which design is superior? Does Ingo's design actually borrow from Con's code, or does it just do more or less the same thing?

Kernel Trap has an ongoing archive of CFS-related discussion from the LKML, including this detailed email exchange betweeen Ingo and Con.

The gist of the exchange is that the two schedulers are similar, but not identical. I'm not a kernel expert and I don't know squat about schedulers, but in my opinion, the exchange seemed to indicate that CFS is superior.

The article really focuses on how quickly the desktop responds to user operations. I haven't personally found this to be a problem on the 2.6 kernels; however, to say that work is not being done in this area is unfair. Kernel Trap has had several articles on people working on CPU schedulers to address this problem, recently the Completely Fair Scheduler was merged to potentially solve this problem: http://kerneltrap.org/node/11773.

...[involving] numerous strange computations involving a number of magic constants; it is difficult to understand, much less improve. --Linux Weekly News

"Before Linux you often needed to be in a clique to work on a high profile project."

Really? Check out the recent plight of the genious programmer Con Kolivas, and Ingo Molnar's theft of his ideas.

http://kerneltrap.org/node/8059

Yes.

iabervon (1971) said:
They took too long to publish this. Linux 2.6.21 (released in April) added support for using one-shot timers instead of a periodic tick, so it avoids the problem like OS X does ... The CFS additionally removes the interactivity boost in favor of giving interactive tasks no extra time but rather just quick access to their available time, which is what they really benefit from.

Anonymous Cowered said:
from reading the CFS documentation, I suspect Ingo read (or at least heard) of this paper, which is available on-line for more than a year according to one of the comments above. this is probably what Ingo means by saying "the CFS scheduler is not prone to any of the 'attacks' that exist today" see http://kerneltrap.org/node/8059

iabervon (1971) said:
On the linux-kernel mailing list, there was a lot of discussion of patterns that cause bad scheduling decisions with various schedulers, generally focused on making test cases for interactivity problems for workloads people had seen. Since the authors of the paper got their initial hint from having problems with a particular real load, this and the work that Ingo is referring to independantly encountered the same issues.

Anonymous Cowered said:
Perhaps you are right, but AFAIK, the LKML doesn't contain any mention of a systematic "attack": Much like the initial hint upon which the paper is apparently based, the workloads that are described in the LKML discussions are "legitimate", in that no application is doing anything malicious. (Also note that process hiding is never discussed.) So the fact Ingo chooses to use the term "attack" in this context suggests he knows something that was not mentioned in the LKML.

from reading the CFS documentation, I suspect Ingo read (or at least heard) of this paper, which is available on-line for more than a year according to one of the comments above. this is probably what Ingo means by saying "the CFS scheduler is not prone to any of the 'attacks' that exist today" see http://kerneltrap.org/node/8059

I suspect libATA is the problem. Are you using the NF4 or do you also have an SIL3114? And which are supported by the newer libATA IDE mechanism? Check here: http://kerneltrap.org/node/11695

Run 2.6.16 and quit complaining.

Or, just use whatever your favorite distribution publishes for you.

See http://kerneltrap.org/node/553/2131 for explanation. In short, Linus has good reasons to use goto.

There's a Linux filesystem under development that might be able to compare favourably with ZFS if shown some love by developers:

* http://oss.oracle.com/projects/btrfs/
* http://kerneltrap.org/node/8376

Avoid the license squabbles and do what we do best: build it ourselves, only better.

Single-threaded SPARC performance on modern processors is perfectly fine. In fact it's better than on Intel / AMD in that it's always the same. ie - it runs the same speed, regardless of load.

What? What the hell are you talking about?

"runs the same speed" regardless of "load"? Could you please use some technical terms here? x86 instructions complete in a given number of cycles (barring branch misprediction, to which SPARC is not immune) so intel/AMD chips also always run at the same speed (barring throttling.)

I've had horrible experiences with RedHat in particular

Well, that's fair - so has everyone else. (Some people are simply willing to overlook them)

They aren't making Solaris look like Linux, they are however, making Solaris cross platform (Sparc/AMD/Intel)

*cough*bullshit*cough* As a newborn Sun employee, Murdock is thinking about making Solaris more Linux-like. "When people say Linux what do they mean? Linux is a kernel. Cool apps are not written to the kernel. The OS powers higher levels of the stack. What we want is an open OS platform and to make sure that the existing skill sets and knowledge and training investments are leveraged. We don't want to make them learn a new product or rip and replace," Murdock said. "You can make a real argument that Solaris innovated more than Linux in the last few years--such as DTrace and ZFS--but usability stands in the way of appreciating that," Murdock said. "Part of what we are working on is closing the usability gap so that it doesn't stand in the way." (next para, emphasis mine:) "There is no reason we can't make Solaris look and feel more like Linux," he continued. "There are a couple of ways we could do it. We could stick a penguin on it or take a Linux distribution and put a Solaris kernel in it. There are a few Solaris-based distros that have done that. Personally, as the person charting the course and looking at the strategy question, it becomes how to keep the competitive differentiation of Solaris while closing the usability gap."

Perhaps you should try to be informed before you attempt to refute my statements? Especially since you're wrong.

Also, it's worth noting that there's Sun SPARC-based hardware that OpenSolaris doesn't run on, because Sun won't give out sufficient specifications. Theo's way of putting it was "Sun released CPU docs, but that's useless. It is kind of like trying to fix a car engine with the owner's manual. The rest of the hardware is not documented."

Now go away, or I shall taunt you a second time.

Very interesting! I hadn't heard this before, but found a thread with some brief mentions of patented optimizations on kerneltrap: http://kerneltrap.org/node/486

Do you happen to have a link to any complete stories that talk about these patents?

Actually I've never seen them on a Linux platform. Zombies are not a problem though. See here for example.

What I've seen that is a problem are processes in "D" state (ininterruptible sleep waiting for the end of an I/O IIRC), usually happening with bad drivers / bad hardware.

But contrary to the windows platform, it never clutters your desktop, as you can "xkill" X ressources even if the program still uses ressources in the background.

'Alan Cox [interview] suggested, "the real test of whether Sun were serious about ZFS being anywhere but Solaris is what they do to license it - they've patented everything they can, and made the code available only under licenses incompatible with other OS products. Their intent is quite clear, and quite sad. Compare it to what the old Sun company did with NFS, which is now a standard used everywhere."' http://kerneltrap.org/node/8066

I understand that there is a popular OS written in C++. Here's what Linus has to say about it:

"we did try C++ once already, back in 1992. It sucks. Trust me - writing kernel code in C++ is a BLOODY STUPID IDEA."

C'mon what are we talking about here, a few minutes? AFAIK, better power savings comes through a good acpi config, which I don't see a whole lot of discussion on.

"The tickless kernel feature (CONFIG_NO_HZ) enables 'on-demand' timer interrupts: if there is no timer to be expired for say 1.5 seconds when the system goes idle, then the system will stay totally idle for 1.5 seconds. This should bring cooler CPUs and power savings: on our (x86) testboxes we have measured the effective IRQ rate to go from HZ to 1-2 timer interrupts per second.

"This feature is implemented by driving 'low res timer wheel' processing via special per-CPU high-res timers, which timers are reprogrammed to the next-low-res-timer-expires interval. This tickless-kernel design is SMP-safe in a natural way and has been developed on SMP systems from the beginning."

If Microsoft put as much energy into making a good operating system as they did in planning how to "defeat" open source, they might have a halfway decent product. Sadly, they keep dicking around with crappy models and outdated notions. If M$ released a product which was as interoperable as Linux, as customizable as Linux, as modular as Linux, and as user-supportive as Linux, there would be no pressing need for Linux. Imagine if M$ sold Windows in boxes at a retail store for one base price (for individuals and families,) where you could install updates/optimizations/customizations/etc without having to buy a new OS to get said features. M$ would indeed make more money selling a license in this manner due to sheer popularity. (For comparison, see http://kerneltrap.org/node/8197 where Linus talks of all the new architectures and stacks available on 2.6.22-RC1) Imagine if users of M$ products could get new, optimized stacks for free. Would that not make you love Windows? Sadly, M$ is committed to charging more and providing less. That's why we have Linux.

Yeah, like Windows "just works".

• A lot of people expect Office to be included with Windows. It isn't.
• A lot of people expect Outlook to be included with Windows. It isn't (but it is free).
• A lot of people expect all their hardware to work first time. It doesn't. Even if you get an OEM bundle, sometimes just the order you actually start to use stuff / plug it in can cause glitches. A noob could hose a USB pendrive by just unplugging it during a big write, for example.

I don't think Linux is any different from Windows in that regard, especially given that this is an OEM offering, not a DIY install. Funnily enough, in a curious world, if Dell support "get" Linux, they may be able to better support it - compared to Windows - over time. If they have a standardised distro, then being able to read logs from clients (via email, VNC, whatever) may be more useful than the crap that Windows gives you in guise of error messages & debug information. They could recommend alternative free software, rather than having to continue supporting old apps "because they came with the machine and I don't want to upgrade", etc etc.

Shame the linux kernel took "printer on fire" out though, huh?

1. Millions of hackers? There isn't a single FOSS project that millions of hackers have contributed too.

2. There are very few people with the experience to write a good much less great 3d driver.

3. Even with the specs I am guessing that the majority of contributions will be security or code clean up and not performance optimizations.

How? The code is all open source. Hans owned the company called "Namesys" that does all the development, apparently that company is still alive to some degree anyways, because developers continue to post patches from @namesys.com addresses. All I know is that there still seems to be a decent amount of activity surrounding it (note the date, April 25th 2007).

My understanding is that Hans was the "brains" behind the operation as far as high level ideas and direction is concerned, but he spent a good portion of his time looking for funding and trying to raise money in other capacities, while he paid other programmers to work on Reiser4 itself.

As far as your comment regarding journaling and htree's, you are obviously severely lacking in knowledge as far as what Reiser4 is capable of, it goes FAR beyond fast access to many files and journaling. Speed, encryption and compression are just a few of the other major features it offers... According to the benchmarks, with the speed of CPUs today you can almost double your disk performance by enabling compression, contrary to what most people would think.

Version 3 of the Completely Fair Queueing IO scheduler was put into the mainline Linux Kernel in 2.6.13 and has been the default IO scheduler since 2.6.18. Jens Axboe, who wrote and maintains it, says that CFQ "has also grown more advanced features such as IO priority support, allowing a user to define the IO priority of a process with ionice, similar to CPU scheduling." in an interview at KernelTrap (http://kerneltrap.org/node/7637).

Ingo the ego maniac? Lots of egos among the kernel hackers.
People should read the thread ( http://kerneltrap.org/node/8059 )
to see the politics of Linux kernel development.

Linux really doesn't need a new process scheduler. What it could really do with is I/O prioritisation. Windows now has it, so there's no excuse.

I dunno. Linux has has some changes recently in the scheduling department, and an O(1) process scheduler can only be "a good thing". Recently, the I/O block layer got a new scheduler (linky http://kerneltrap.org/node/7637). Regarding other I/O prioritization, I can't say with authority that this is needed or not.

Maybe all of these things are related, but in my selfish world, I want Linux to scale better. Scaling well beyond 64 CPUs or cores, but I believe this is a much greater task than ~60 hours of coding. It took a while before Linux got SMP clean, and today SMP is normal, not something that reserved for servers. The monolithic high clockrate CPU is dead, and this has been overtaken by multi-core processors, and more than one of them inside of one box. In my eye, this is where all OSes should be focusing their attention.

I think processor companies already do something similar w/o an FPGA.

The difference between the Pentiums and the Celeron (or whatever they called now) used to be mainly cache size -- this might have been motivated from yield considerations (in terms of the cache, since that is a large portion of the chip area). I remember reading something along the lines that they might have a few extra cache lines that can be used to replace a bad one (during the time of manufacture), by blowing a tiny fuse, etc. And I guess that if a particular chip doens't have enough good cache lines for a Pentium, then it becomes a Celeron...

I'm not a hardware designer, but I would think that a general-purpose FPGA wouldn't map well for processor implementation. My guess is that heat/power and/or speed considerations would prevent anything very reconfigurable/not-very-specialized to be used in a modern desktop CPU.

OTOH, modern Intel processors do still use some kind of microcode that can be updated via software (e.g. when the OS is running). I *think* microcode is only used to implement really complex operations or weird instructions that are seldom used. It isn't going to be used for fast things like simple arithmetic operations, etc.

Interesting reading: http://kerneltrap.org/node/2678

Gosh, I don't know. That reminds me, did Linus ever put ESR's CML2 patches into the kernel tree?

http://kerneltrap.org/node/17

It was possible, dunno if it still is, to use a Firewire device for kernel debugging in FreeBSD.

Macs a few years ago didn't restrict Firewire access, and there was a demo of vandalizing the video display by plugging in an iPod.

Ref. Dornseif, CanSecWest 2005. His results about writing were
OS X: works
FreeBSD: works
Linux: works
Windows 2000: crashes
Windows XP: doesn't work

Except that Adam Boileau demonstrated write access to RAM under Windows from Firewire by having the device lie about its configuration.

This sort of thing is why security people sometimes act so devoid of hope.

This is not about your one-line patch. The issue of adding hardware support to a stable branch is something that people have been arguing about for a while. See, for example, this, which is about 2.6.16.y but talks about some of the issues that adding hardware support can cause.

I agree that Vista isn't the last Windows OS. Case in point is OS/2, thought to be DOA in 1995 is still around and Windows will probably be too in 10 years.

Linux? Yes, the argument is very good but he doesn't take into account the raw power, quality, robustness and flexibility of Linux especially the Kernel. He expounds on the lack of drivers which indicates that the author isn't quite up to date on Kernel development.

As far as the OS being outdated I think not, maybe for the casual consumer a transparent OS will come true but there will always be a OS it is in the nature of computing machines. Running all your applications on the Internet will probably come true however accessing the Internet will still requite an OS AND an application to connect to it.

No benchmarking of the media, though, so I guess that would be the unique part.

You forgot another thing, Windows Vista has it Out of The Box [or the torrent if u prefer.]. Yeah, With Linux you could do /anything/. You could compile your kernel with Genetic algorithms to make it automagically tune according to your computers and resources, you could also get all the stunning visual 3D effects (compiz and all those beta-state applications). But you would have to spend at least 2 hours *after* installing your OS. While I do not know how long does Vista takes to install, I know that, what they are advertising is something you get /out of the box/ and although for you it might not matter, for the _average user_ it is the *big* difference.

Seriously. I do not plan to run Vista aaaaanytime soon (not even piratebayed) as I am completely fine with my Xubuntu 6.10 but at least I can acknowledge when they innovate (hint, the innovation is not in the *creation* of the technology, but in the implementation experience ... yep out of the box).

http://kerneltrap.org/node/7238

Why no comparison against VMWare or native?

This is likely to boost QEMU's popularity, the virtualizer accelerated by KVM. An interesting coïncidence is that I released the very first version of Qemudo on Jan 4th while being totally unaware of the existence of KVM. Then three days later the KVM project released their first version too, and I read about it on this kerneltrap article.

I am thrilled at the idea of using KVM + QEMU + Qemudo together. To put it simply, and to quote my README, Qemudo is "a Web interface to QEMU offering a way for users to access and control multiple virtual machines running on one or more remote physical machines." Qemudo makes use of two important features in QEMU: native support of VNC, and copy-on-write disk images for instantaneous VM creation. If you are interested go check out the website (and download the tarball which contains more detailled doc). </shameless-plug>

> [T]he Linux 2.6.20 kernel will include a full virtualization (not para-virtualization) solution. Yep. But Molnár Ingo (yes, the hungarian kernel hacker) Ingo Molnar announced a new patch introducing paravirtualization support for KVM.