Slashdot Mirror

distributors are definately in the process of getting io down. So is Linus himself. quote from http://lkml.org/lkml/2007/8/5/171 : "change relatime updates to be performed once per day." It's not only the livetime of flash memory that benefits from this. also power consumption and noise goes down for hdds. and overall io performance benefits fromsuch improvements,too. About the swap: just keep it big enough so the Kernel can free the ram of some unused data, but not a lot bigger. Twice the size of the ram is nonsense these days.. if you run out of buffers and cache you don't have enough ram. if you have enough ram swap is hardly used.

Unless Solaris is released under the GPLv3 and Linus sees some stuff he wants.

Really, he said that.

Everything is a rumour until it is posted to the LKML.

Everybody knows that.

On July the same discussion was on lkml. I guess the opinion of Linus Torvalds himself is pretty clear:

"I think that people who argue for splitting desktop kernels from server kernels are total morons, and only show that they don't know what the hell they are talking about."

You're assuming that the 2GB a day could be spread evenly over the disk. This would vary depending on how much free space you have on the device. If your drive is 1% full then you can distribute your writes over the other 99%. But most people don't keep their storage mainly empty. In fact people tend to run just under the limit - hence the saying that crap always expands to fill the available space. If your drive was 99% full then you can't distribute the writes over the parts with data (as it would have to be moved somewhere else negating the benefit), and then you run into the problem with the limited duty cycle.

Wear levelling happens inside the device - the OS can't see it.

So even if you keep writing to the same logical sector over and over again that physical location moves around so that the erase count remains the same for each erase unit. The device has just needs to keep track of the mapping from logical blocks that the OS uses to physical blocks which are an actual location in flash.

It has to be like this, since all filesystems tend to write to the area at the start of the disk very frequently. On FAT the FAT needs to updated everytime a file grows. On EXT2/3 or NTFS it's actually worse - the inode needs to be updated on file growth or when the file "last accessed time" changes. People have worked on "last accessed time" problem though - XP only updated it with a one hour granularity, and Vista disables it by default. Linux has a relatime mount option.

Please take a look at Linux Device Drivers, Third Edition. It's available for free but you of course will be purchasing it (you only mentioned it had to be free - not that you would be using the free version if there were multiple versions available).

I'll be expecting to see your name popping up on the Linux Kernel Mailing List in a few months announcing fixes and new features for my Intel, AMD/ATI, Via and NVIDIA graphics cards (some of which you will be reverse engineering). No need to thank me (you may also need to find a good book on X).

Anyone who has studied filesystems in depth knows about NetApp's WAFL, that NetApp has defended its patents, and that ZFS uses a lot of ideas from WAFL. Questions about ZFS violating WAFL patents came up on LKML months ago, and probably earlier elsewhere. People have been wondering why Linux doesn't have anything like ZFS, and a large part of the answer to that is patents; The Tux2 filesystem would have been a lot like ZFS but was stopped due to patent grumblings. I wish Sun luck in overturning the patents, since that would help everyone, but this lawsuit should not come at a surprise to anyone.

In 2000, Daniel Phillips started developing a new Linux filesystem that
would have many of the features netapps WAFL has, and ZFS has now.

This filesystem was called Tux2.

He was quite sure that the patents NetApp had on this weren't valid,
because of prior art, and because his algorithm was quite
different and quite a bit smarter:

http://uwsg.iu.edu/hypermail/linux/kernel/0010.0/0 343.html

Yet somewhere in 2002, he gave up on Tux2, presumably due to pressure
from netapp: http://lkml.org/lkml/2002/8/26/138 .

I wonder what will happen to BTRFS in light of this new NetApp
legal action: http://oss.oracle.com/projects/btrfs/

Mike.

* Copyright (c) 2002-2007 Sam Leffler, Errno Consulting
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer,
* without modification.
* 2. Redistributions in binary form must reproduce at minimum a disclaimer
* similar to the "NO WARRANTY" disclaimer below ("Disclaimer") and any
* redistribution must be conditioned upon including a substantially
* similar Disclaimer requirement for further binary redistribution.
* 3. Neither the names of the above-listed copyright holders nor the names
* of any contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* Alternatively, this software may be distributed under the terms of the
* GNU General Public License ("GPL") version 2 as published by the Free
* Software Foundation.
*
* NO WARRANTY
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTIBILITY
* AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
* THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY,
* OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
* IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
* THE POSSIBILITY OF SUCH DAMAGES.

* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:

* [T]his software may be distributed under the terms of the
* GNU General Public License ("GPL") version 2 as published by the Free
* Software Foundation.

Check it out.

Flame war averted.

link

There's also a new reply from Ingo dated today, that RFS is more complex (increased code size), is not so fast and is less fair:

link

While it 'removes complexity' by removing comments, debugging code and other stuff.

link

There's also a new reply from Ingo dated today, that RFS is more complex (increased code size), is not so fast and is less fair:

link

While it 'removes complexity' by removing comments, debugging code and other stuff.

Here's a post on how the above patchset can improve the responsiveness of the system under heavy write load:

huge improvement with per-device dirty throttling

And the thread referencing the latest version of the patch posted to lkml:

per device dirty throttling -v9

Here's a post on how the above patchset can improve the responsiveness of the system under heavy write load:

huge improvement with per-device dirty throttling

And the thread referencing the latest version of the patch posted to lkml:

per device dirty throttling -v9

Date Fri, 31 Aug 2007 12:54:47 +0200 From Ingo Molnar Subject Re: [ANNOUNCE/RFC] Really Fair Scheduler Digg This * Roman Zippel wrote: > Hi, > > I'm glad to announce a working prototype of the basic algorithm I > already suggested last time. As I already tried to explain previously > CFS has a considerable algorithmic and computational complexity. [...] hey, thanks for working on this, it's appreciated! In terms of merging your stuff, your patch looks a bit large and because you did not tell us that you were coding in this area, you probably missed Peter Zijlstra's excellent CFS work: http://programming.kicks-ass.net/kernel-patches/sc hed-cfs/ The following portion of Peter's series does much of the core math changes of what your patch provides (and which makes up for most of the conceptual delta in your patch), on a step by step basis: sched-update_weight_inv.patch sched-se_load.patch sched-se_load-2.patch sched-64-wait_runtime.patch sched-scaled-limit.patch sched-wait_runtime-scale.patch sched-calc_delta.patch So the most intrusive (math) aspects of your patch have been implemented already for CFS (almost a month ago), in a finegrained way. Peter's patches change the CFS calculations gradually over from 'normalized' to 'non-normalized' wait-runtime, to avoid the normalizing/denormalizing overhead and rounding error. Turn off sleeper fairness, remove the limit code and we should arrive to something quite close to the core math in your patch, with similar rounding properties and similar overhead/complexity. (there are some other small details in the math but this is the biggest item by far.) I find Peter's series very understandable and he outlined the math arguments in his replies to your review mails. (would be glad to re-open those discussions though if you still think there are disagreements.) Peter's work fully solves the rounding corner-cases you described as: > This model is far more accurate than CFS is and doesn't add an error > over time, thus there are no more underflow/overflow anymore within > the described limits. ( your characterisation errs in that it makes it appear to be a common problem, while in practice it's only a corner-case limited to extreme negative nice levels and even there it needs a very high rate of scheduling and an artificially constructed workload: several hundreds of thousand of context switches per second with a yield-ing loop to be even measurable with unmodified CFS. So this is not a 2.6.23 issue at all - unless there's some testcase that proves the opposite. ) with Peter's queue there are no underflows/overflows either anymore in any synthetic corner-case we could come up with. Peter's queue works well but it's 2.6.24 material. Non-normalized wait-runtime is simply a different unit (resulting in slightly higher context-switch performance), the principle and the end result does not change. All in one, we dont disagree, this is an incremental improvement we are thinking about for 2.6.24. We do disagree with this being positioned as something fundamentally different though - it's just the same thing mathematically, expressed without a "/weight" divisor, resulting in no change in scheduling behavior. (except for a small shift of CPU utilization for a synthetic corner-case) And if we handled that fundamental aspect via Peter's queue, all the remaining changes you did can be done (and considered and merged) evolutionarily instead of revolutionarily, ontop of CFS - this should cut down on the size and the impact of your changes significantly! So if you'd like to work with us on this and get items that make sense merged (which we'd very much like to see happen), could you please re-shape the rest of your changes and ideas (such as whether to use ready-queueing or a runqueue concept, which does look interesting) ontop of Peter's queue, and please do it as a finegrained, reviewable, mergable series of patches, like Peter did. Thanks! Ingo

Ingo's reply can be found here. Roman's reply to that is here and here

Ingo's reply can be found here. Roman's reply to that is here and here

Ingo's reply can be found here. Roman's reply to that is here and here

It may seem that the licenses let one _distribute_ it under either
license, but this interpretation of the license is false...

In http://lkml.org/lkml/2007/8/29/183, Alan Cox managed to summarize
what Jiri Slaby and Luis Rodriguez were trying to do by proposing a
modification of a Dual Licenced file without the consent of all the
authors. Alan asks "So whats the problem ?". Well, Alan, I must
caution you -- your post is advising people to break the law.

Funny, but I would have thought that 'Alternatively, this software may be distributed under the terms of the GNU General Public License ("GPL") version 2 as published by the Free Software Foundation.' would mean, well, that alternately the software could be distributed under the terms of the GNU General Public License ("GPL") version 2 as published by the Free Software Foundation.

If he had made the argument that some of the files (i.e. the headers) lacked that notice and thus couldn't be distributed under the GPL he would have been a point. If he said that even though it was allowed it was rude he might have had a point. Instead he's just being obtuse.

It appears that you can't read this particular patch style. The lines with + mean added, the lines with - mean removed.

The lines without either mean that's context for the differences.

If you look at the original patch, no attribution was removed. The attribution was in the context lines.

It looks like the .c files were handled appropriately and it was merely the .h files that had the license completely ripped out. The .c files were dual-licensed and said you could choose either. They just removed the BSD license as that was "choosing" GPLv2. The .h files are just some interfaces and don't change often anyways, so the BSD license is good enough for them (they should have left those). The .c files are the actual implementation, which would change between operating systems.

Here's a link to the actual diff as provided in the original article:

http://lkml.org/lkml/2007/8/28/157

You'll also note that the dual-licensed code had the committer's copyright notice on it. In some cases it was only his notice, originally. With the data immediately available, maybe he stripped it out in a commit before this one, but they don't seem to be accusing him of that. They are mainly accusing him of ripping out the BSD license from a couple .h files since they didn't have the dual-license notice in them. If they aren't dual-licensed under both, you can relicense as GPLv2, but you have to include the BSD notice under its own terms. The GPL itself even says not just attribution, but the original notices themselves must be preserved. One additionally might say that since the GPL says to preserve the original notice, that even in the dual-license case you must preserve the BSD license in order to initially comply with the GPL, although that's a requirement of the GPL and not a dual-licensing/BSD provision. A dual-licensing (as you can see in this case) clearly says you can pick either, since the word "Alternatively" (e.g. the ath5k_reg.h license) implies if you chose the following path, you can ignore the provisions of the previous path.

In summary, it looks like a lot of this was nit-picking over how to actually do the license notice preservation, rather than preserving somebody's attribution. I imagine it'll be fixed up in very little time and few people will care about this in more than a day or two.

Problem solved.

Problem solved.

Technically the best we can do is to leave the license as dual
licensed, but keep in that technically that means nothing and is just
for show, the GPL is what would apply as its derivative work and is
the most restrictive license. This applies to any other driver in the
kernel right now with a dual license tag.

http://lkml.org/lkml/2007/8/29/69

Someone pointed out the problem and a patch is likely on its way.

The response from the person involved is at least much more responsible and reasonable than in the earlier incident: http://lkml.org/lkml/2007/8/29/69

I dont know, perhaps it runs a 64bit kernel?
Take a read of:
http://lkml.org/lkml/2006/1/19/73
That explains it better...

How much of that is because some versions on some platforms only report 497 days uptime before resetting to zero?

He said Con wasn't following up on bug reports wheras Ingo did. In commercial terms he sacked Con and moved hi responsibilities to Ingo who he trusts.

However, what Linus said about Con not following up on bug reports is disputed as false by some people.

Quoting Grzegorz Kulewski from the above link:

"I don't really want to keep all that -ck flamewar going but this sum-up is a little strange for me:

If Con was thinking SD was "perfect" why he released 30+ versions of it? And who knows how many versions of his previous scheduler?

Besides Con always tried to help people and improve his code if some bugs or problems were reported. Archives of this list prove that. I reported several problems (on list and privately) and all were fixed very fast and with very kind responses."

• Higher maintenance in having to maintain two similar subsystems within the kernel.
• More choices and options means that debugging and enhancing/fixing the kernel is harder.
• IMPORTANT: Supercomputer users with 1024 CPUs require a much lower memory footprint and this would benefit desktop users as well - but only if the two schedulers aren't split. This is the benefit of "cross-pollination" (having everyone use the same subsystem for various uses).

• -ck users are a special subset group who have very narrow aims and viewpoints (desktop computing). They need to show that they're prepared to accommodate other users who run Linux on supercomputers and take their concerns very seriously (even if it means compromising on desktop performance).
• He is not compromising the development of the kernel in favor of a political outcome that makes everyone happy, but which also creates a huge technical mess (plugsched).
• Linus has to work at a very high level when making technical decisions. He has to look at everything from embedded users through to supercomputer users and therefore needs to do the hard work of making compromises where necessary. His viewpoint is very wide and open - not the narrow and directed view that a lot of other developers have.

• He has very good argumentative/debating skills and this is often mistaken as abusive behavior by people with weaker skills in this area. These skills are vital in technical environments where politics and indecision will lead to poor engineering outcomes.
• Linus is not showing unwarranted pride. By nature of his experience, he does know more about kernel development and management than most people on this planet. This is not something to hide away. It should be taken far more seriously than what I am reading from the "Linus ego" crowd. He is describing his reasoning and is allowing the opportunity for others to provide input. His experience is VERY IMPORTANT, don't trash it. It is fine to question his views, and as far as I see, he hasn't come back with a "don't question my knowledge/experience noob" response (which is vastly different to ignoring posts from newbies - although I haven't seen much of this either).
• He is decisive, direct and open to argument. All the skills and more you need in a real engineer. Sales tactics such as avoiding argument, avoiding hard questions, compromising at the request of the customer even if it is bad for them, etc HURT engineering efforts.
• Kernel development isn't for the weak. Your actions directly impact MILLIONS of users and TRILLIONS of dollars worth of economic activity. Your actions filter down into life support systems, nuclear warheads and banking systems. It isn't a happy picnic in the middle of a field of flowers. It is hard (and rewarding) work with a huge sense of responsibility. Changing the scheduler in the kernel has HUGE consequences for many users. Will it work on a 1024 CPU supercomputer running a large bank? An embedded microcontroller in the warhead of a nuclear missile?

A lot of useful discussion seems to have generated in response to people's
_interpretation_ of my interview rather than what I actually said. For
example, everyone seems to think I quit because CFS was chosen over SD (hint:
it wasn't). Since it's generating good discussion I'll otherwise leave it as
is.

Here is what Linus said in lkml

lkml full quote of Linus:
> People who think SD was "perfect" were simply ignoring reality. Sadly,
> that seemed to include Con too, which was one of the main reasons that I
> never ended entertaining the notion of merging SD for very long at all:
> Con ended up arguing against people who reported problems, rather than
> trying to work with them.
(from Linus's post to lkml)

The other misrepresentation is in this quote
> As a long-term maintainer, trust me, I know what matters. And a person who
> can actually be bothered to follow up on problem reports is a *hell* of a
> lot more important than one who just argues with reporters.
Because Kolivas was quite good with feedback--arguably better than Ingo or Linus (for example with Linus vs. SCSI-emulation cdrecord)--and recently had the single instance to which Linus refers. Ironically, Kolivas rejected a request to have code renice X processes for the same reason Linus rejected a request to keep SCSI emulation as de-facto: the design and code is cleaner and more correct. In fact, while Con would argue about small design issues and change his views, this renicing instance (some people called out as trolling because of the insistence and seeming insincerity) is the only time I've seen Con (and I have followed his work since he started) flatly and repeatedly reject a request.

There is merit in Linus's argument that a comparison of CFS and SD showed no "significant difference" in performance.

My personal take is that several years of minor spats between Ingo and Con made a better -ck patchset and -mm patchset, but never brought -ck closer to mainline. It came down to the good-old-boy system where Linus knew and trusted Ingo better than Con, and if there was a major disagreement, Ingo's side would be favored. But, honestly, Linus could also be considering that Ingo's resume has always been a programmer's resume (even if Ingo was one of the youngest maintainers at his start) while Con is a self-taught programmer (just to improve kernel responsiveness nonetheless!) with a primary passion as a physician. If Con decides not to continue with the -ck patchset, I am sure he will refocus the extra dedication and time to new patients.

Having lurked on http://www.lkml.org/ for several years, I find Linus to be rather rude.

I think you mistake brutal honesty for rudeness. The post referenced is a bit harsh, but it's honest and to the point about how he feels. Politeness can often get in the way of expressing a point. That's not to say politeness isn't a valueable skill at all, it certainly is for a salesman or customer service person. I can be for many jobs, but being to the point is often more valueable in science and technology.

I don't read lkml very often, but from what I've read I think Linus is just strong willed about the things he has strong feelings for. Politeness has the potential to spill over into letting "bad" code into the kernel. Politeness can also hide peoples true intentions, which for anyone that just wants to understand the value system used to judge an idea can be maddening.

Imagine a scenario where there's a pushy person who overwhelms a person with a polite instinct. The polite person might just eventually give in instead of stating harsh realities defending what they believe to be the best idea. The pushy person might never learn why the polite person won't just accept what they think is right. It's not the best option for an honest discussion.

Anyway, I think you need to look at the situation as a discussion about what the best code is, and who does the best job at producing that code. Falling in love with your code (or anything you produce really) is a bad idea. It seems the kernel maintainer of the SD scheduler did just that, and Linus is only pointing that out.

Having lurked on http://www.lkml.org/ for several years, I find Linus to be rather rude. May be it's because English is not his first language...so words are not well chosen. I must say though, that I excuse him because he produces, [or helps produce] a very useful product on the world today. That is the Linux kernel.

As it seems many others don't agree to his opinions of Con, http://lkml.org/lkml/2007/7/27/426

I hope he picks "leader" and gets off the kernel kick long enough to make corrections suggestions that he can 'lead' with.

The Linux kernel is a design disaster. This might be a consequence of the development style. However, the result is that I have dropped any intentions to support LRW on Linux.

So you don't have LRW support on Linux.

Meanwhile, I've yet to see any expert review of either system. The closest I ever saw was David Wagner (well known crypto expert at Berkeley) saying that Clemens "does not understand cryptography very well".

I use loop-aes myself, partly because it's actually pretty easy to use, and mostly because of the extraordinary effort that Clemens put into denying the existence of the problems with his dm-crypt code. Just to be clear, the "well known" problems with loop-AES were shared by dm-crypt, and they were only well known because the loop-AES author acknowledged them loudly and fixed them while Clemens kept shouting "no no no there is no attack and you are stupid".

Linus's email was dated June 20th, 2007. That was less than a month ago. I suspect the text of the GPLv3 license he was commenting on didn't change between that date and the date it was released: June 29th, 2007.

http://lkml.org/lkml/2007/6/20/223
http://www.fsf.org/news/gplv3_launched

The "pretty ok" article you link, however, is dated several months ago: March 28, 2007.

About the 32k threads, read this.

If you ask me, that is complete bullshit. In the specific case, Linus is not conveying software to his children. Not even in a TiVo way: the computer end up being his. He can take it away if the kids don't want to do homework, or brush teeth. It ends up being private use, wich GPLv3 permits him to do whatever he wants.

Reading that makes me feel that either he doesn't understand the license or is using subtle falacies to pursuit an agenda(hello mister obvious).

Every legal system enforces ideas of morality. Why is murder wrong? Why do countries restrict hate speech? Why can't you have sex with your sister? These are all moral concepts enforced through legality.

You're right, but these examples actually refer to moral consensus, a set of moral principles shared by majority. What Linus referred to was moral principles of the individual, enforced to the majority in the form of a law by that same powerful individual.

Maybe Linus is having a bad day. And what exactly does he mean by:

Does Linus sell have a sideline selling PCs? And he uses some DRM to stop users modifying the software he supplies? What?

The article is bad for not referring to the actual Linus' post on LKML. In that post Linus was talking about putting Linux on his kids computers and prohibiting them (the kids, a.k.a users) to update the software on their own. He says that, as a legal guardian of his kids, he has the right to do so, and that he effectively controls the hardware.

> To call people who draw the line in a different place than you hypocrites is BS.

That was *not* what I did.

I don't think it's hypocritical to prefer the GPLv3. That's a fine choice, it's just not *mine*.

What I called hypocritical was to do so in the name of "freedom", while you're at the same time trying to argue that I don't have the "freedom" to make my own choice.

See? THAT is hypocritical.

The GPLv3 can never do that. Quite fundamentally, whenever you extend the "reach" of a license past just the derived work, you will *always* get into a situation where people who designed two different things get into a conflict when they meet. The GPLv2 simply avoids the conflict entirely, and has no problem at all with the "Do unto others as you would have them do unto you". In a very real sense, the GPLv3 asks people to do things that I personally would refuse to do. I put Linux on my kids computers, and I limit their ability to upgrade it. Do I have that legal right (I sure do, I'm their legal guardian), but the point is that this is not about "legality", this is about "morality". The GPLv3 doesn't match what I think is morally where I want to be. I think it *is* ok to control peoples hardware. I do it myself.

I find it silly to compare his kids with himself - the comparison (in the context of http://lkml.org/lkml/2007/6/20/223) clearly does not make sense.

Then he starts ranting GPLv2 not being about "morals" although he himself considers anything written to Linux kernel ("thinking about Linux") must be GPLv2 but ported drivers (e.g. from Windows) does not need to be.

Sour, said the fox ...

http://lkml.org/lkml/2007/6/20/223

This was in the comments on the article page: http://lkml.org/lkml/2007/6/20/223