Slashdot Mirror
Linux Gains Support for NUMA
soosterh writes "CNet has an article about a NUMA patch from IBM. It says that the improvement adds some support in Linux for nonuniform memory access, or NUMA, a design for higher-end servers with many processors. Linus Torvalds, the original creator of the operating system and still its top authority, accepted the update this month into version 2.5, the current test version of the software."
I thought that I'd seen some other NUMA stuff in previous runs of 'make menuconfig'-- Can anyone explain what's already there and what this patch adds?
get 0wned. irc.w30wnzj00.com
Seriously, this is something that will close one of the last remaining gaps between Linux and Solaris. Not that it will do much good for 99% of users out there, but if you need this, you *really* need it.
And, of course, also support for the Hammer architecture, which is (smaller scale) NUMA. Each processor in an x86-64 system has its own memory bus, so time to access memory depends on whether the memory is directly connected to a given processor, or whether another processor needs to mediate, the definition of NUMA.
I've had this sig for three days.
Someone correct me if I'm wrong.
This article says the code was submitted by Martin Bligh, not Dirk Pitt.
Clearly it's a typo. I'll have to e-mail Clive about this.Rot-13 my address to e-mail me.
"So I hurry back to little earth / For another life another birth"
Not only is this beneficial for large computers, but also on smaller SMP systems with hyperthreading. On CPU's with hyperthreading,
it's often faster for a process to reside on the same CPU but not necessary the same 'virtual' CPU when accessing memory.
And alot of 8way+ systems are NUMA whether or not they are advertised as such.
Also, I seriously doubt if any desktop machine will use NUMA; it's primarily about systems which use system boards, where there are CPUs & RAM on a board which slots into the system & a CPU can access memory on a local board faster than that on other boards. Desktops tend to use one "system board" (i.e. the motherboard) so there isn't the difference in speed for accessing the data.
Oh, for cryin' out loud. Dude, there's this thing called Google. Try it out some time.
That said, I'll give you a hint: non-uniform memory access. If you've got a computer that uses different banks of memory as a single physical address space, then that computer has a NUMA architecture.
If you want to maintain cache coherency across a NUMA system, you have to employ some tricks. These tricks are sufficiently complex to warrant their own name: ccNUMA.
I write in my journal
I'd imagine it's mainly for 64-bit as that's the kind of systems which tend to ship with NUMA (usually with MIPS or Itanium). Without knowing more, I couldn't comment as to whether it will work under 32-bit or not, but I can't see how it would be so limited.
That is an incredibly naive comment. NUMA systems have been around for quite a while (think Sequent), the current generation of IBM x440 are NUMA. These are all 32-bit Intel architectures.
This patch didn't even address memory, it only dealt with scheduling processes anyway.
You misunderstand the way Open Source works. Linux actually has perfect democracy - *anybody* can take charge. Anybody - even you - could fork the kernel any time you wanted to. Of course, until you show why your stream is better than the main stream, you'll be pretty lonely on your fork.
The only reason that Linus is still the controlling authority on the Linux kernel is that he is doing it pretty well. And he isn't without advice from others - there are hundreds of people only too willing to favour him with their advice.
Every functioning organisation needs a chief executive - someone who makes the final decision. Even when you have executive committees - and, informally, most Open Source projects do - someone still has to make the final decision, to jusdge what the consensus actually is.
Anybody can "call an election" on an Open Source project any time, by proposing a fork. That is much closer to perfect democracy, it seems to me, than one where you only choosw the Chief Exec every few years.
As it is, Linux is a roaring success with Linus in charge. It ain't broke. don't fix it.
Consciousness is an illusion caused by an excess of self consciousness.
the MIPS/Itanium systems the parent refers to are (I assume) the SGI Origin and Altix multiprocessor servers, both 64bit, the first MIPS/IRIX, the second Itanium/Linux:
Origin
Altix
"we demand rigidly defined areas of doubt and uncertainty!"
Thank you for explaining who Linus Torvalds is.
Unfortunately it's not JUST showing someone you're better, it's "marketing" too. That's pretty hard in the case of Linux, because you can't use the name "Linux" anywhere - it's trademarked by Linus. If i made some funkyass Linux fork and called it Finnix, it wouldn't get the press Linux would, and you'd effectively be ten years back in time, building up a name etc. With BSD it's a little easier, because "BSD" in and of itself isn't trademarked, though it's doubtful whether you'd be able to get away with calling your Soviet Russia fork "NyetBSD".
That said, i pretty much agree. Linux has so much stuff in it right now as a kernel (in the sense of running on a zillion architectures with a zillion features) i'd say Linus is pretty much pleasing everyone in the end. Most Alan Cox stuff gets rolled in, most commercial stuff does too. It's all good.
I got a sig so you would remember me.
You do not have to run Linus stock kernel.
Not two vendors ship the same kernel. So in the end it's up to the vendor you use to tweak your kernel. Redhats are heavily patched to suit (what they belive) is there users needs..
I think thats a good system.
the decision of linus to not accept the NUMA patches right away was correct. we now have a patch that doesn't affect non-NUMA archs...
i don't like style guides
Of course, actually doing this would involve jumping through a few hoops, and I have to think hard to come up with situations why this would be the way to go.
Gees, any MB I've bought in the last several years can take more RAM than I'm willing to buy for it. I wonder what kind of memory limits the OP was asking about.
"A language that doesn't affect the way you think about programming, is not worth knowing" - Alan Perlis
Linus Torvalds, the original creator of the operating system and still its top authority.
You know Slashdot is going "mainstream" when people have to explain who Linus is.
----- rL
SGI must have added NUMA support for their Itanium-based Altix-servers as well. On their web-page it says: "Enhanced Operating System for High-Productivity Computing" [aka Linux] with "High-performance NUMA support".
Anyone ever seen a patch for this?
- jarman
...can be found here.
they are copying Linux related news from CNET.
You underestimate the intelligence of Linus, he initially refused the NUMA patch as it affected performance on non NUMA systems. So the author of the patch got some help with it and the patch now works without hindering non NUMA systems.
Surely this is quality control at its finest? every project has its project manager. Everyone is free to write their own applications without Linus controlling them, he just looks after his project which happens to be the kernel. Without strict quality control the kernel would be a right old dogs dinner by now.
It's not a SPF - if you want to run a kernel without the NUMA, go right ahead. If you want to run a kernel with DRM support, go right ahead. It's opensource - that's the whole point.
Linus decides what goes into his tree. You decide what goes into your tree. If you don't have the time or skill to build your own tree, pick someone you trust, and use thiers.
"More recently, the NUMA scheduler patch has been reworked (by Martin Bligh, Erich Focht, Michael Hohnbaum, and others) around a simple observation: most of the NUMA problems can be solved by simply restricting the current scheduler's balancing code to processors within a single node. If the rebalancer - which moves processes across CPUs in order to keep them all busy - only balances inside a node, the worst processor imbalances will be addressed without moving processes into a foreign-node slow zone. A simple (three-line) patch which did nothing but add the within-node restriction yielded most of the benefits of the full NUMA scheduler; indeed, it performed better on some benchmarks. Real-world loads, however, will require a scheduler which can distribute processes evenly across nodes. Occasionally it is necessary, even, to move processes to a slower node; a lot of CPU time on a lightly-loaded node will give better performance than waiting in the run queue on a heavily-loaded node. So a bit of complexity had to be added back into the new scheduler to complete the job."
Extracted from:
http://lwn.net/Articles/20741/
However, the main way you might be able to add RAM over and above the MB limit is via some kind of PCI card with DIMMS on it. I'm not sure how that would work over PCI (even 66MHz/64bit) or how it would work at a lower level, but it might get by some limits. The limits OP was asking about may be of the order of trying to get over 1GB of RAM for some simulation code. Of course if you need over 1GB of RAM, buy a system which supports it.
In any event, from what people are saying, the NUMA patch is a change to the scheduler, to ensure that processes run on the CPU nearest the RAM bank storing the data. I don't think it addresses trying to add RAM from other sources (either disk or hypothetical PCI card)
Richard Stallman has sent an angry e-mail demanding a name change to GNUMA.
You can only drink 30 or 40 glasses of beer a day, no matter how rich you are.
-- Colonel Adolphus Busch
Sorry if this seems rude, but it sounds like someone crying 'sour grapes.' Do a little research and you can easily find news group discussions about the profiteer who wanted to charge people to use the name Linux. The parent post is correct about your opportunity to fork a better implementation. It's a good thing my moderation points elapsed. I would have been looking for a crybaby option ;-)
Remember those good ol' days?
Contrary to what is said in the post, NUMA support has been in Linux for quite a while already. The recent patches accepted by Linus merely add NUMA awareness to the scheduler, which, while certainly being a prerequisite for Linux being used on production NUMA boxen, is not at all required for NUMA support in general.
quidquid latine dictum sit altum videtur.
SGI has also integrated NUMA technology into the linux kernel to support their new Altix servers. How do these two efforts relate? Is SGI's code generic enough that it could also be considered for inclusion in the mainstream kernel? Or is it specific to SGI's NUMA architecture? Is IBM's code generic enough that it would work on an Altix? What functional characteristics distinguish the two?
--Lawrence Lessig for Congress!
Proproetary companies are not despised. Most proprietary software is. Hardware companies that refuse to release driver level documentation are. Companies are not (unless everything that they do falls into the above catagories).
Thus: Proprietry hardware companies that give open documentation are not 'despised'. Companies that write GPL code are not 'despised'. They are valuable members of the linux comunity.
Because if it's GPL, and good, it gets in. It's not any more a handout than if you wrote some code, and submited it. There is nothing special about corporate ownership. If you have a problem with corporate submissions, you can modifiy the kernel, to remove their work. It's not a problem for me, or for the vast majority of the linux using comunity. I think that you seem to percive (or hold) an anti-corporate stance that is not representative of the comunity at large.