Slashdot Mirror

The legal analysis is here. The argument is basically that Bionic copied directly from the kernel, like this clever implementation of a byteswap. It's the cleverness that makes it copyrightable, whereas headers that only include defines used for interopt are not copyrightable. The glibc version of the same file is original; it doesn't come from the Linux kernel (at least I can't find it). So the problem is there may be original ideas that come from the linux code that got put into bionic. At the same time, it would be hard to argue that a program linking to a standard C library is a derivative work of that particular implementation. I believe such an argument would get laughed out of court.

Now, this guy is a lawyer, and he understands copyright law more than me, but his understanding of this particular issue is weak. The example given earlier, of the bytewaps, does not apply at all to any developers on ARM because that was x86 specific code.

His second argument is extremely weak. He says the API is also copyrightable (which is sometimes true), and that if it were not, someone could use it to re-implement a closed source version of Linux. Which is true, but it's a rather braindead thing to worry about: getting the API is not the hardest part of that (and indeed, BSD has a linux compatibility layer that responds correctly to the Linux APIs).

This header was automatically generated from a Linux kernel header of the same name, to make information necessary for userspace to call into the kernel available to libc. It contains only constants, structures, and macros generated from the original header, and thus, contains no copyrightable information.

The rational is here; basically, kernel headers are typically used in compiling the kernel, so using it to build userspace apps is a typically untested configuration that is prone to breakage. The script (according to the readme) tries to strip the bits that cause problems which they didn't need in the first place. It looks like they just remove all comments, so the removal of the license is a side effect of that.

I don't actually do anything with Android (or have a device), this is just from reading that file.

That link seems broken. Did you mean this?

But Google claims to have removed the interesting elements leaving only interface details that are needed to operate with the kernel.

There are (at least) two mistakes in your statement.

First, the person to whom you're responding specifically said that, at least according to his or her definition of "interesting", those interface details are "interesting":

As a programmer, I have to disagree with this. The header files should be very interesting. They should provide all of the information that I need in order to properly use the C code or library. I really hate looking through an implementation in order to figure out how to use a function (and these functions do not have 'man' pages).

Second, as that person notes, Google didn't just leave "only interface details that are needed to operate with the kernel":

Some header files described in the articles contained inline function calls. This makes them a bit more than a standard header.

and, as I've noted in response to about a zillion posts all saying "butbutbut the headers just have function declarations and stuff such as that!", as the README.TXT file for the Android build script that munges the kernel header files says, "the 'clean headers' only contain type and macro definitions, with the exception of a couple static inline functions used for performance reason (e.g. optimized CPU-specific byte-swapping routines)" (emphasis mine).

Google should know very well by this point that you can't just copy GPL code and change the license on it.

That's not what they are doing. They have provided clean header files (interfaces) to link with the associated libraries. Same as you writing your own function declarations for a library that doesn't provide headers.

They have, at least according to the README.TXT file for the Android build script that processes the Linux kernel header files to make headers for Bionic, provided header files that include "a couple static inline functions used for performance reason (e.g. optimized CPU-specific byte-swapping routines)". That's a bit more than "clean header files (interfaces)", as it includes some implementations.

So? I can add inline functions to my interfaces to whatever library i want and there's no problem with that.

If you read the rationale section of Bionic README.TXT it doesn't seem that Google's trying to claim these new headers are re-licensed. They're simply providing an (automatic) way to create cleaned headers that won't cause compilation errors due to a variety of reasons that the original headers can/will cause them.

Except that, according to said README.TXT, said headers also include "a couple static inline functions used for performance reason (e.g. optimized CPU-specific byte-swapping routines)".

Google should know very well by this point that you can't just copy GPL code and change the license on it.

That's not what they are doing. They have provided clean header files (interfaces) to link with the associated libraries. Same as you writing your own function declarations for a library that doesn't provide headers.

They have, at least according to the README.TXT file for the Android build script that processes the Linux kernel header files to make headers for Bionic, provided header files that include "a couple static inline functions used for performance reason (e.g. optimized CPU-specific byte-swapping routines)". That's a bit more than "clean header files (interfaces)", as it includes some implementations.

Here's the very first lines at the top of /usr/src/linux-2.6.37.1-1..2/COPYING

NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work". Also note that the GPL below is copyrighted by the Free Software Foundation, but the instance of code that it refers to (the Linux kernel) is copyrighted by me and others who actually wrote it.

So just using the normal api/abi is specifically exempted from the whole "derived work" question by the authors

Using the normal API/ABI by normal system calls is exempted.

Using, for example, the "optimized CPU-specific byte-swapping routines" that, according to the README.TXT for the Android build script that munges the Linux kernel header files into headers for Bionic, are still in the "clean headers" is not necessarily exempted.

Google's action here is the equivalent of copying an entire book, then replacing the title page with their own -- claiming authorship and copyright.

Or perhaps more like copying the title and chapter names of an entire book, and a small number of piquant passages. What the script from the Android source that "cleans" the Linux kernel header files does, according to the README.txt for it, is "process a set of original and unmodified kernel headers in order to get rid of many annoying declarations and constructs that usually result in compilation failure", with the result being that "the 'clean headers' only contain type and macro definitions, with the exception of a couple static inline functions used for performance reason (e.g. optimized CPU-specific byte-swapping routines)".

The part that those who think Google is in the wrong here might find most important here is the "couple static inline functions used for performance reason"[sic] - that's not just a specification for an interface, that's an implementation of an interface.

Except by this logic all Linux applications must either be GPLv2, or never make any syscalls.

No, the license for the Linux kernel has an extra bit at the top, put there by somebody named "Linus", that starts out saying

NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work".

Well you're obviously not a copyright genius, otherwise you'd know that header files usually only contain structural information about functions that doesn't contain the level of creativity required to clear the threshold for copyright protection.

Yeah, it's not as if, for example, <linux/socket.h> has inline functions for accessing control message headers.... (Hint: look for the word "inline" in the Linux headers. You may find it a few times....)

Google's attitude seems to be that copyright is merely a hassle, an obstacle to be routed around. Even if they are not found to be legally in violation of the GPL, they obviously Bionic with the deliberate intent of routing around it.

If you read the rationale section of Bionic README.TXT it doesn't seem that Google's trying to claim these new headers are re-licensed. They're simply providing an (automatic) way to create cleaned headers that won't cause compilation errors due to a variety of reasons that the original headers can/will cause them. While I'm not a copyright expert, this doesn't sound anything like what these guys are claiming Google's doing.

Also, I've never heard of anyone having to release their source code because they used header files from Linux. You kinda have to use them to compile, and there's plenty of non-GPL software that compiles and runs on Linux that hasn't been forced to release its source code.

Bionic comes with a set of 'clean' Linux kernel headers that can safely be included by userland applications and libraries without fear of hideous conflicts. for more information why this is needed, see the "RATIONALE" section at the end of this document.

these clean headers are automatically generated by several scripts located in the 'bionic/kernel/tools' directory, which process a set of original and unmodified kernel headers in order to get rid of many annoying declarations and constructs that usually result in compilation failure.

In other words, they're complying with the GPL by including the scripts necessary to generate the code. Florian Mueller is a liar and enemy of open source, has been for years. This is the guy who went around saying open source destroys value.

Read the RATIONALE section at the bottom.

After you've read it, if you still have a problem, to put this into perspective ... let's take a proprietary platform, Windows. There is nothing to prevent me from creating a set of optimized headers that wrap around the windows api, leaving out unnecessary parameters, constants, and code definitions that will conflict with my userland application code, and defining new "convienence" methods, functions, classes and structures.

If I then use the "cleaned up" headers to create an application, that application is no less my work because I used the "cleaned up" version as opposed to the original kernel source.

Proprietary applications can use these "cleaned up" header files without falling afoul of the GPL, the same as if they had used the original headers. Otherwise, the "binary blob" device driver problem would clearly not exist.

From the GPL v2:

identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works.

An application distributed separately, just like a driver distributed separately, is not "derived" from the kernel source.

Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.

It's a tricky balance. The kernel is gpl, and the code is available, including the "cleaned up headers" code, as per the gpl.

Just remember, an application that calls kernel functions is not "derived from" the kernel any more than an application that calls the windows api is "derived from" windows.

I believe this is just more FUD on the part of certain parties ... ask yourself "cui bono", then follow the money to the usual suspects.

I've been running using dmraid in RAID 5 configuration - I need to use Intel's fakeraid to facilitate dual-booting to Windows. Does this commit

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=9d09e663d5502c46f2d9481c04c1087e1c2da698

mean that I no longer need to hunt patches for dmraid5 but can just use vanilla kernel from now on?

Can't remove it without breaking backward compatibility. But any competent developer should have already done a global search of their code base for strcpy, strcat, etc. and made sure they either did appropriate up front checks or replaced them with strncpy, strncat etc. -- preferably the latter, to keep the issue from having to be revisited in the future.

You'd think so - but did you read the article I linked to? http://git.kernel.org/?p=linux%2Fkernel%2Fgit%2Ftiwai%2Fsound-2.6.git&a=commitdiff&h=eaae55dac6b64c0616046436b294e69fc5311581 - obviously that change was made less than a month ago.

I absolutely abhor the phrase "dumbing down" when used in this context.

Linux used to be something used by a tiny minority of people who were primarily interested in hard-core computer science testing and research. It was their playground in which they could work their art. By making it more user-friendly, it has gotten it into the hands of people who are brilliant in other ways so that they can work their art. Are you a graphics guru? A UI wiz? A scripting genius? A music prodigy? A 3D design master? A business star? A poet laureate? If so, then Linux is now for you, too!

It hasn't been "dumbing down" anything. If anything, it has been dumbing up--more and more people using it in smarter and smarter ways.

And the beauty of the situation? If you're a hard-core computer scientist wanting to do testing and research with new stuff, it's still there for you, too.

I absolutely abhor the phrase "dumbing down" when used in this context.

Linux used to be something used by a tiny minority of people who were primarily interested in hard-core computer science testing and research. It was their playground in which they could work their art. By making it more user-friendly, it has gotten it into the hands of people who are brilliant in other ways so that they can work their art. Are you a graphics guru? A UI wiz? A scripting genius? A music prodigy? A 3D design master? A business star? A poet laureate? If so, then Linux is now for you, too!

It hasn't been "dumbing down" anything. If anything, it has been dumbing up--more and more people using it in smarter and smarter ways.

And the beauty of the situation? If you're a hard-core computer scientist wanting to do testing and research with new stuff, it's still there for you, too.

yes, Linux is the powerful transformer. Flexiable to fit in phones, PCs, servers, chips. from computer specialists to computer dummies.

I absolutely abhor the phrase "dumbing down" when used in this context.

Linux used to be something used by a tiny minority of people who were primarily interested in hard-core computer science testing and research. It was their playground in which they could work their art. By making it more user-friendly, it has gotten it into the hands of people who are brilliant in other ways so that they can work their art. Are you a graphics guru? A UI wiz? A scripting genius? A music prodigy? A 3D design master? A business star? A poet laureate? If so, then Linux is now for you, too!

It hasn't been "dumbing down" anything. If anything, it has been dumbing up--more and more people using it in smarter and smarter ways.

And the beauty of the situation? If you're a hard-core computer scientist wanting to do testing and research with new stuff, it's still there for you, too.

I absolutely abhor the phrase "dumbing down" when used in this context.

Linux used to be something used by a tiny minority of people who were primarily interested in hard-core computer science testing and research. It was their playground in which they could work their art. By making it more user-friendly, it has gotten it into the hands of people who are brilliant in other ways so that they can work their art. Are you a graphics guru? A UI wiz? A scripting genius? A music prodigy? A 3D design master? A business star? A poet laureate? If so, then Linux is now for you, too!

It hasn't been "dumbing down" anything. If anything, it has been dumbing up--more and more people using it in smarter and smarter ways.

And the beauty of the situation? If you're a hard-core computer scientist wanting to do testing and research with new stuff, it's still there for you, too.

yes, Linux is the powerful transformer. Flexiable to fit in phones, PCs, servers, chips. from computer specialists to computer dummies.

I absolutely abhor the phrase "dumbing down" when used in this context.

Linux used to be something used by a tiny minority of people who were primarily interested in hard-core computer science testing and research. It was their playground in which they could work their art. By making it more user-friendly, it has gotten it into the hands of people who are brilliant in other ways so that they can work their art. Are you a graphics guru? A UI wiz? A scripting genius? A music prodigy? A 3D design master? A business star? A poet laureate? If so, then Linux is now for you, too!

It hasn't been "dumbing down" anything. If anything, it has been dumbing up--more and more people using it in smarter and smarter ways.

And the beauty of the situation? If you're a hard-core computer scientist wanting to do testing and research with new stuff, it's still there for you, too.

The command does exist. It even includes security provisions for insisting on finishing the process before it allows doing anything else.

But of course whether it actually does what it's supposed to depends on the firmware.

> The difference is that Google is not calling their system Java.

Really? Android docs and source beg to differ:
http://source.android.com/source/code-style.html
http://android.git.kernel.org/?p=platform/packages/apps/Calendar.git;a=tree;f=src/com/android/calendar;hb=HEAD

...you do realize that everything is exactly as you say it should be, right? That's pretty much universal with all SSDs that I'm aware of.

How to perform an SSD secure erase to via ATA commands: https://ata.wiki.kernel.org/index.php/ATA_Secure_Erase

I apologize if your post was sarcasm. It's hard to tell on the Internet, and other people may not recognize it as such.

That said, it would be nice if there was some standard way of doing this.

Wouldn't it be even cooler if they made it part of the ATA standard itself, so you could send a single disk command to immediately commit to secure destruction of the entire volume.... such that even if someone powered down in the middle of an erase and powered the drive back up, the circuit boards inside the drive would just continue the committed physical media secure erase rather than respond to any further read commands, or enable any type of inquiries/recovery efforts?

Note: that's the source for the SDK. If you want the source for the full distro, you will find it here, including all drivers, even graphics card drivers, which is actually kind of surprising.

If everyone clicked the link in those "work from home" scams 100 times, or replied to every "your webmail account is about expire" email with bogus details then it would drown the enemy in useless information.

If you then take it a step further and have an automated system that clicks links a million times automatically and replies to the emails with bogus information a million times then it would be even better.

Until someone gets the idea to send out a "I made a billion $$$ working from home. Click http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.38-rc2.tar.bz2 for details!" and you're suddenly part of the problem.

can we please get back to arguing now?

Yeah, right after I finish beating Doom III by "I.D." software running on my "MAC".

I wouldn't be surprised if the poster pronounces daemon as "daymon" either. The geek ego sure seems to get in the way of learning how to pronounce things. It looks like devs these days will have to provide audio files to show how to pronounce their work.

*shakes his head at correct pronunciation deniers*

That's already been done...

boot.kernel.org

You're probably running into this long standing IO bug, which despite complaints for many years, has still not been properly diagnosed. A big mystery, evidently.

https://btrfs.wiki.kernel.org/index.php/Problem_FAQ has the answer: cp --reflink=always ... You will need a non-ancient version of coreutils for this to work. Cheers, Michael

Indeed, Linux has been doing this for years. But PXE is not a correct analogy, since that's only an execution environment, like efi.

boot.kernel.org is what you're looking for. And yes, it can use PXE to netboot your machine directly onto the 'Net.

Incidentally, if anyone out there has suggestions on how to reliably test for race conditions, please speak up.

The foolproof way is to write your software as a state machine that has well defined atomic transitions between states. If that's not an option, put mutexes on all shared objects and use a deadlock detection algorithm like the Linux kernel has for proving the (in)correctness of the locking operations.

The actual code is here.

Reading the various compare() implementations definitely leaves room for doubt about correctness. The compareStrictly() code is a lovely illustration of the ambiguity that exists in the world of phone numbers. The comparison implementation mentioned above is found in compareLoosely() and is characterized in comments as "similar() not equals()", meaning collisions are possible. Which of compairStrictly/Loosely is actually use is subject to configuration; the caller can't know which is used without examining configuration resources.

Haven't yet seen evidence that this is the cause of the problem folks are having; does the SMS code rely on this? The comments claim the compareLoosely() method is "identical enough for caller ID purposes." One could imagine that when the user hits 'reply' on a message the code might hunt through the phone book using compareLoosely and stop on the first "match", which may be incorrect due to a collision. There seems to be some correlation between reports of this phenomenon and the 'threaded' 'conversation' stuff in Android, which could mean people are relying on 'reply' and getting wrong results.

Who knows. Bugs will happen and phones aren't trivial. The real problem in my mind is that this one has been on the books for a looong time now (six months, approximately) and it's not getting the attention it clearly deserves.

http://android.git.kernel.org/?p=platform/packages/apps/Mms.git;a=commit;h=7bb3d8cf74ec1e4ae18cb814c17e12a00816f105

Though I guess it'll take a while to get into builds/updates for existing handsets.

First thing I on a fresh system (and I install a lot of fresh systems due to testing that goes horribly wrong :) Just put this in your sources.list and your fine. deb http://mirrors.nl.kernel.org/debian/ squeeze main contrib non-free deb-src http://mirrors.nl.kernel.org/debian/ squeeze main contrib non-free deb http://security.debian.org/ squeeze/updates main contrib non-free deb-src http://security.debian.org/ squeeze/updates main contrib non-free deb http://deb.opera.com/opera-beta/ squeeze non-free deb http://www.debian-multimedia.org/ squeeze main non-free After that I down the catalyst drivers from ati. And only then I start using the system. With all my closed-source goodies :D I love it!

First thing I on a fresh system (and I install a lot of fresh systems due to testing that goes horribly wrong :) Just put this in your sources.list and your fine. deb http://mirrors.nl.kernel.org/debian/ squeeze main contrib non-free deb-src http://mirrors.nl.kernel.org/debian/ squeeze main contrib non-free deb http://security.debian.org/ squeeze/updates main contrib non-free deb-src http://security.debian.org/ squeeze/updates main contrib non-free deb http://deb.opera.com/opera-beta/ squeeze non-free deb http://www.debian-multimedia.org/ squeeze main non-free After that I down the catalyst drivers from ati. And only then I start using the system. With all my closed-source goodies :D I love it!

Are you able to guarantee IP lawyer would NOT be able to spin this on the line of:

-Well, the kernel was first published in 1991. 2011 makes a couple of decades from publication, formally I can take the kernel and the bug fixes... you know, there are some bug fixes in 20 years... and use it as I please.

Yep, easily.

All it takes is one kernel contributor to point to one feature that they implemented more recently and own the copyright to and say "You're using code that I own the copyright to, without my permission."

But, really, in the case of the Linux kernel, it has changed so much that there's really no possibility of a question. Even if every line in a 1991 kernel were completely unmodified, it would still only represent a tiny fraction of the code in a 2010 kernel. The last 1991 release, version 0.11 only HAD 13K LOC as compared to the 13.5 million LOC in 2.6.36. So if every line were unmodified, that 1991 kernel code would represent only 0.1% of the current codebase. And, it's unlikely that those 13K lines are umodified. It wouldn't surprise me if analysis were to show that a 2010 kernel and a 1991 kernel only have a few hundred lines in common, and that those lines are almost entirely trivial.

For that matter, if we just look at the changes in Linus' git repository (which only goes back to April 2005) we see that the kernel has grown from 6,718,838 lines to 13,913,125 lines(*), a net increase of 7,194,287 lines -- more than doubling the size of the codebase in 5.5 years. Running git diff --stat to find the differences between the April 2005 version and today's version shows that in 5.5 years there have been 10,683,974 lines added and 3,493,136 lines removed.

Honestly, for the Linux kernel, a copyright duration of even 5 years is probably perfectly adequate to ensure that no one is interested in building on a public domain kernel. And even if someone did, so what? Do you think progress on the GPL kernel would stop, or even slow? The BSD kernel continues moving, and not that much more slowly than Linux.

Linux is something of a special case -- and a very poor choice as your example -- but if you go with 10 years, much less 20, the same will hold true of any open source software. If software has gone that long without changes, it's because no one cares about it any more. Even codebases as marvelously stable as Knuth's TeX undergo changes quite regularly -- and if they didn't, if they so perfectly do their job that there's no need to update them, there's not going to be much value that can be added by a closed source version.

No, copylefted software has NOTHING to fear from 10-20 year copyright terms.

(*) Line counts included all files in the source tree, excluding only the contents of the .git tree. Excluding the git tree was done by the simple expedient of mv'ing it out of the way, then line counts were produced with find . -type f | xargs cat | wc.

I will do some research on GIT for fronting SVN in the future when I have time to play outside in the OpenSource domain again.

This will get you started: http://www.kernel.org/pub/software/scm/git/docs/git-svn.html

I believe the parent is talking about the iowait bug #12309, which, maddeningly, has nothing to do with swap, or filesystem type. You can turn off the swap entirely and still trigger the bug. Of course there are use cases where heavy swapping brings the system down, so there is a perceived improvement by most people when turning off the swap.

It's been widely reported that there's a duplicated file, and indeed it there is something close to that. BUT! One thing you'll find missing in Oracle's Exhibit "J" are the package headers at the top of the file. There's a good reason for that. On the Android side the file is in package org.apache.harmony.security.tests.support.cert, in directory support/src/test/java. You can see this in the git repository for android. It's sitting in a directory of test support classes.

So the matching file that we have here is part of the test suite to ensure compliance with the interfaces. It is NOT part of the implementation itself. So the real question is, is it OK to have this kind of file sitting in the test branch, to ensure that the real implementation of it complies?

The fact that the package headers have been removed and that this file is from the test suite can't be anything other than a deliberate attempt to deceive, well, someone. ;)

It's rather unbelievable that with thousands of stories out there on this file nobody is talking about WHERE it fits into the android tree.

It's been widely reported that there's a duplicated file, and indeed it there is something close to that. BUT! One thing you'll find missing in Oracle's Exhibit "J" are the package headers at the top of the file. There's a good reason for that. On the Android side the file is in package org.apache.harmony.security.tests.support.cert, in directory support/src/test/java. You can see this in the git repository for android. It's sitting in a directory of test support classes.

So the matching file that we have here is part of the test suite to ensure compliance with the interfaces. It is NOT part of the implementation itself. So the real question is, is it OK to have this kind of file sitting in the test branch, to ensure that the real implementation of it complies?

The fact that the package headers have been removed and that this file is from the test suite can't be anything other than a deliberate attempt to deceive, well, someone. ;)

It's rather unbelievable that with thousands of stories out there on this file nobody is talking about WHERE it fits into the android tree.

An AC below pointed me to an article at zdnet that has a copy of the complaint and excerpts of the exhibits.

The one example listed is in PolicyNodeImpl: OpenJDK version, Android version.
It was released under the GPL as part of the OpenJDK, but the license has been changed to Apache 2.0 in the Android version. Unless the original author authorized this, that is not kosher. I can't find in in the HEAD of the Harmony repository. Also note that it appears in a test directory of the Android tree - probably included to test against their own implementation.

Well, this one example does look pretty copied. Sun wrote sun.security.provider.certpath.PolicyNodeImpl - notice it's fully documented, and authored by Seth Proctor and Sean Mullan. It's not part of the standard Jaa library, it's part of the JVM's private implementation, and was released later on as part of OpenJDK.

Google have exactly this same code, minus the comments in their copy of Apache Harmony, but it's not in the official Apache Harmony, at least since 2005 (Don't believe me? Run svn log -v http://svn.apache.org/repos/asf/harmony | grep PolicyNodeImpl).

However, this code isn't central to Android. It's part of the test suite, it doesn't run on any phones.

On the one hand - this looks like a cut and dried infringement. On the other hand, it's a pretty trivial part of the project. Is that the best Oracle can find? If it is, then it's on a par with SCO holding up malloc.h as the "smoking gun".

These are not fs blocks but actual disk sectors (as the documentation states), so the 512 byte size is the correct one.

Yeah, that's what the discussion was about - we improved that particular case, see this commit (which can be found in v2.6.36), and Phoronix reported about that upstream fix.

Thanks,

Ingo

Ingo, I find delays of 29-45ms to be pretty noticeable. To put it another way, if you had a delay of 10ms before, and you're now getting a delay of 50ms due to some background copy, all of your applications went from running at 100fps to 20fps, which I think even non-sensitive people can pick up on, even outside of games and smooth scrolling. VIM feels different over a 10ms LAN connection vs. a 45ms connection from my home.

Yes i agree with you that if a 45 msecs latency happens on every frame then that will snowball and will thoroughly ruin game interactivity - but note the specific context here:

you can see the commit referenced by Phoronix here

(hm, my first link above was broken, sorry about that.)

Those 45 msec delays were statistical-max outliners - with the average latency at 7.3 msecs. This got cut down to 25 msecs / 6.6 msecs respectively via the patch. Note that it's also a specific, CPU overloaded workload that was measured here, so not typical of the desktop unless you are a developer running make -j build jobs.

We care about optimizing maximum latencies because those are what can cause occasional hickups on the desktop - a lagging mouse pointer - or some other non-smooth visual artifact.

Thanks,

Ingo

I'll chime in too about not having this problem.[...] Now that everyone on Slashdot seems to be having this problem, I can't help but wonder how I managed to be so lucky all these years. WTF is going on?

Put differently, where is the simple test which demonstrates the problem? It may or may not be the one someone linked to: https://bugzilla.kernel.org/show_bug.cgi?id=12309

I tried that dd(1) line. It doesn't make /everything/ else "slow down to a crawl", but everything that does I/O is affected ... and as someone remarked things like fancy web browsers and bloated desktops environments do a lot of I/O.

Ingo,

I believe most desktop users run into this problem when they complain about IO schedulers. Is there any immediate plan to address it?

Thanks,

Jason

Regarding plans you need to ask the VM and IO folks (Andrew Morton, Jens Axboe, Linus, et al).

Regarding that bugzilla entry, there's this suggestion in one of the comments:

    echo 10 > /proc/sys/vm/vfs_cache_pressure
    echo 4096 > /sys/block/sda/queue/nr_requests
    echo 4096 > /sys/block/sda/queue/read_ahead_kb
    echo 100 > /proc/sys/vm/swappiness
    echo 0 > /proc/sys/vm/dirty_ratio
    echo 0 > /proc/sys/vm/dirty_background_ratio

    or use "sync" fs-mount option.

If you can reproduce that problem with a new kernel (v2.6.36 would be ideal) then please try to describe the symptoms in a mail to linux-kernel@vger.kernel.org, and also point out whether the tunings above improved things. Please Cc: Jens, Andrew, me and Linus as well.

To turn interactivity woes on the desktop into actual hard numbers you can use Arjan van de Ven's latencytop tool. It will measure your worst-case delays with and without big copies being done in the background, which numbers you can cite in your email.

Thanks,

Ingo

It's more than that ... it's about the reclaimable memory, often it will chose to throw much more recent data to swap because it's priority algorithms think that is a better idea than reclaiming (which costs CPU). Linux's VM developers seem to me to be a very myopic bunch ... leaving me with no other choice than not letting their code not make too many decisions ... ie. turn off swap.

https://bugzilla.kernel.org/show_bug.cgi?id=12309

PS. why the hell was drop_caches dropped, it almost feels like the VM developers just couldn't stand the plebian end users second guessing their brilliant code.

Ingo,

I believe most desktop users run into this problem when they complain about IO schedulers. Is there any immediate plan to address it?

Thanks,

Jason