Slashdot Mirror
Linux Gains Support for NUMA
soosterh writes "CNet has an article about a NUMA patch from IBM. It says that the improvement adds some support in Linux for nonuniform memory access, or NUMA, a design for higher-end servers with many processors. Linus Torvalds, the original creator of the operating system and still its top authority, accepted the update this month into version 2.5, the current test version of the software."
I thought that I'd seen some other NUMA stuff in previous runs of 'make menuconfig'-- Can anyone explain what's already there and what this patch adds?
get 0wned. irc.w30wnzj00.com
Seriously, this is something that will close one of the last remaining gaps between Linux and Solaris. Not that it will do much good for 99% of users out there, but if you need this, you *really* need it.
And, of course, also support for the Hammer architecture, which is (smaller scale) NUMA. Each processor in an x86-64 system has its own memory bus, so time to access memory depends on whether the memory is directly connected to a given processor, or whether another processor needs to mediate, the definition of NUMA.
I've had this sig for three days.
Someone correct me if I'm wrong.
Not only is this beneficial for large computers, but also on smaller SMP systems with hyperthreading. On CPU's with hyperthreading,
it's often faster for a process to reside on the same CPU but not necessary the same 'virtual' CPU when accessing memory.
And alot of 8way+ systems are NUMA whether or not they are advertised as such.
Oh, for cryin' out loud. Dude, there's this thing called Google. Try it out some time.
That said, I'll give you a hint: non-uniform memory access. If you've got a computer that uses different banks of memory as a single physical address space, then that computer has a NUMA architecture.
If you want to maintain cache coherency across a NUMA system, you have to employ some tricks. These tricks are sufficiently complex to warrant their own name: ccNUMA.
I write in my journal
You misunderstand the way Open Source works. Linux actually has perfect democracy - *anybody* can take charge. Anybody - even you - could fork the kernel any time you wanted to. Of course, until you show why your stream is better than the main stream, you'll be pretty lonely on your fork.
The only reason that Linus is still the controlling authority on the Linux kernel is that he is doing it pretty well. And he isn't without advice from others - there are hundreds of people only too willing to favour him with their advice.
Every functioning organisation needs a chief executive - someone who makes the final decision. Even when you have executive committees - and, informally, most Open Source projects do - someone still has to make the final decision, to jusdge what the consensus actually is.
Anybody can "call an election" on an Open Source project any time, by proposing a fork. That is much closer to perfect democracy, it seems to me, than one where you only choosw the Chief Exec every few years.
As it is, Linux is a roaring success with Linus in charge. It ain't broke. don't fix it.
Consciousness is an illusion caused by an excess of self consciousness.
the MIPS/Itanium systems the parent refers to are (I assume) the SGI Origin and Altix multiprocessor servers, both 64bit, the first MIPS/IRIX, the second Itanium/Linux:
Origin
Altix
"we demand rigidly defined areas of doubt and uncertainty!"
Thank you for explaining who Linus Torvalds is.
Linus Torvalds, the original creator of the operating system and still its top authority.
You know Slashdot is going "mainstream" when people have to explain who Linus is.
----- rL
they are copying Linux related news from CNET.
You underestimate the intelligence of Linus, he initially refused the NUMA patch as it affected performance on non NUMA systems. So the author of the patch got some help with it and the patch now works without hindering non NUMA systems.
Surely this is quality control at its finest? every project has its project manager. Everyone is free to write their own applications without Linus controlling them, he just looks after his project which happens to be the kernel. Without strict quality control the kernel would be a right old dogs dinner by now.
"More recently, the NUMA scheduler patch has been reworked (by Martin Bligh, Erich Focht, Michael Hohnbaum, and others) around a simple observation: most of the NUMA problems can be solved by simply restricting the current scheduler's balancing code to processors within a single node. If the rebalancer - which moves processes across CPUs in order to keep them all busy - only balances inside a node, the worst processor imbalances will be addressed without moving processes into a foreign-node slow zone. A simple (three-line) patch which did nothing but add the within-node restriction yielded most of the benefits of the full NUMA scheduler; indeed, it performed better on some benchmarks. Real-world loads, however, will require a scheduler which can distribute processes evenly across nodes. Occasionally it is necessary, even, to move processes to a slower node; a lot of CPU time on a lightly-loaded node will give better performance than waiting in the run queue on a heavily-loaded node. So a bit of complexity had to be added back into the new scheduler to complete the job."
Extracted from:
http://lwn.net/Articles/20741/
However, the main way you might be able to add RAM over and above the MB limit is via some kind of PCI card with DIMMS on it. I'm not sure how that would work over PCI (even 66MHz/64bit) or how it would work at a lower level, but it might get by some limits. The limits OP was asking about may be of the order of trying to get over 1GB of RAM for some simulation code. Of course if you need over 1GB of RAM, buy a system which supports it.
In any event, from what people are saying, the NUMA patch is a change to the scheduler, to ensure that processes run on the CPU nearest the RAM bank storing the data. I don't think it addresses trying to add RAM from other sources (either disk or hypothetical PCI card)
Contrary to what is said in the post, NUMA support has been in Linux for quite a while already. The recent patches accepted by Linus merely add NUMA awareness to the scheduler, which, while certainly being a prerequisite for Linux being used on production NUMA boxen, is not at all required for NUMA support in general.
quidquid latine dictum sit altum videtur.