Linux Gains Support for NUMA
soosterh writes "CNet has an article about a NUMA patch from IBM. It says that the improvement adds some support in Linux for nonuniform memory access, or NUMA, a design for higher-end servers with many processors. Linus Torvalds, the original creator of the operating system and still its top authority, accepted the update this month into version 2.5, the current test version of the software."
I thought that I'd seen some other NUMA stuff in previous runs of 'make menuconfig'-- Can anyone explain what's already there and what this patch adds?
get 0wned. irc.w30wnzj00.com
Will this help with both 32 bit and 64 bit desktop platforms? It seems that in the future there won't be much distiction between the current server machines and desktops...
Seriously, this is something that will close one of the last remaining gaps between Linux and Solaris. Not that it will do much good for 99% of users out there, but if you need this, you *really* need it.
And, of course, also support for the Hammer architecture, which is (smaller scale) NUMA. Each processor in an x86-64 system has its own memory bus, so time to access memory depends on whether the memory is directly connected to a given processor, or whether another processor needs to mediate, the definition of NUMA.
I've had this sig for three days.
--OK, got a noob ram question then. Does this NUMA allow for upgrading total RAM beyond the original specs? Any sort of add-ons? I ask this from noticing it's used because of the physical distances it can access (among others).
thanks in advance to yon knowledgeable ones
Someone correct me if I'm wrong.
This article says the code was submitted by Martin Bligh, not Dirk Pitt.
Clearly it's a typo. I'll have to e-mail Clive about this.Rot-13 my address to e-mail me.
"So I hurry back to little earth / For another life another birth"
Not only is this beneficial for large computers, but also on smaller SMP systems with hyperthreading. On CPU's with hyperthreading,
it's often faster for a process to reside on the same CPU but not necessary the same 'virtual' CPU when accessing memory.
And alot of 8way+ systems are NUMA whether or not they are advertised as such.
Oh, for cryin' out loud. Dude, there's this thing called Google. Try it out some time.
That said, I'll give you a hint: non-uniform memory access. If you've got a computer that uses different banks of memory as a single physical address space, then that computer has a NUMA architecture.
If you want to maintain cache coherency across a NUMA system, you have to employ some tricks. These tricks are sufficiently complex to warrant their own name: ccNUMA.
I write in my journal
For a great adventure story, read some of Clive's books.
~S
Oh wiat. You mean _that_ NUMA.
[Please type your sig here.]
As long as you get whatever the heck you need to get done on whatever O/S or platform, everything after that is fun and games.
~S
OpenBSD doesn't have a democracy. It has a benevolent (heh) dictatorship. This is a Good Thing to a certain extent, because it keeps a project on-track. It gives it some level of direction, as opposed to letting things bloat... and bloat... and bloat... You can't compare open source to countries, because it's a lot easier to fork source and make your own branch than it is to revolt and declare independence. That said, sometimes the dictator can be a stubborn little bitch, and there's not much anyone can do. But that's why Linux users have the Alan Cox branches, and OpenBSD users have MicroBSD and other "unofficial" patched versions. It's no big deal.
I got a sig so you would remember me.
You misunderstand the way Open Source works. Linux actually has perfect democracy - *anybody* can take charge. Anybody - even you - could fork the kernel any time you wanted to. Of course, until you show why your stream is better than the main stream, you'll be pretty lonely on your fork.
The only reason that Linus is still the controlling authority on the Linux kernel is that he is doing it pretty well. And he isn't without advice from others - there are hundreds of people only too willing to favour him with their advice.
Every functioning organisation needs a chief executive - someone who makes the final decision. Even when you have executive committees - and, informally, most Open Source projects do - someone still has to make the final decision, to jusdge what the consensus actually is.
Anybody can "call an election" on an Open Source project any time, by proposing a fork. That is much closer to perfect democracy, it seems to me, than one where you only choosw the Chief Exec every few years.
As it is, Linux is a roaring success with Linus in charge. It ain't broke. don't fix it.
Consciousness is an illusion caused by an excess of self consciousness.
the MIPS/Itanium systems the parent refers to are (I assume) the SGI Origin and Altix multiprocessor servers, both 64bit, the first MIPS/IRIX, the second Itanium/Linux:
Origin
Altix
"we demand rigidly defined areas of doubt and uncertainty!"
Thank you for explaining who Linus Torvalds is.
Unfortunately it's not JUST showing someone you're better, it's "marketing" too. That's pretty hard in the case of Linux, because you can't use the name "Linux" anywhere - it's trademarked by Linus. If i made some funkyass Linux fork and called it Finnix, it wouldn't get the press Linux would, and you'd effectively be ten years back in time, building up a name etc. With BSD it's a little easier, because "BSD" in and of itself isn't trademarked, though it's doubtful whether you'd be able to get away with calling your Soviet Russia fork "NyetBSD".
That said, i pretty much agree. Linux has so much stuff in it right now as a kernel (in the sense of running on a zillion architectures with a zillion features) i'd say Linus is pretty much pleasing everyone in the end. Most Alan Cox stuff gets rolled in, most commercial stuff does too. It's all good.
I got a sig so you would remember me.
You do not have to run Linus stock kernel.
Not two vendors ship the same kernel. So in the end it's up to the vendor you use to tweak your kernel. Redhats are heavily patched to suit (what they belive) is there users needs..
I think thats a good system.
the decision of linus to not accept the NUMA patches right away was correct. we now have a patch that doesn't affect non-NUMA archs...
i don't like style guides
Linus Torvalds, the original creator of the operating system and still its top authority.
You know Slashdot is going "mainstream" when people have to explain who Linus is.
----- rL
Wow, that guy called Linux changed his name to be just like that computer thing!
Not trying to be a cheerleader but apparently Linus originally planned 2.5 as the "Numa Support" version. There were certain features that Linus wanted that he didn't actually merge into mainline until AFTER feature freeze. He just queued them for inclusion.
Cheers
Stor
"Yeah well there's a lot of stuff that should be, but isn't"
SGI must have added NUMA support for their Itanium-based Altix-servers as well. On their web-page it says: "Enhanced Operating System for High-Productivity Computing" [aka Linux] with "High-performance NUMA support".
Anyone ever seen a patch for this?
- jarman
...can be found here.
Not really. If Linus were to flip his lid and totally lose the plot we'd all be running AC kernels in short order.
they are copying Linux related news from CNET.
You underestimate the intelligence of Linus, he initially refused the NUMA patch as it affected performance on non NUMA systems. So the author of the patch got some help with it and the patch now works without hindering non NUMA systems.
Surely this is quality control at its finest? every project has its project manager. Everyone is free to write their own applications without Linus controlling them, he just looks after his project which happens to be the kernel. Without strict quality control the kernel would be a right old dogs dinner by now.
Gee, when you have to -remind- us of Linus' claim /.er's mind...
to fame, it seems a sign that the History of Linux
must be fading from the modern
I really -doubt- that it has...
It's not a SPF - if you want to run a kernel without the NUMA, go right ahead. If you want to run a kernel with DRM support, go right ahead. It's opensource - that's the whole point.
Linus decides what goes into his tree. You decide what goes into your tree. If you don't have the time or skill to build your own tree, pick someone you trust, and use thiers.
"More recently, the NUMA scheduler patch has been reworked (by Martin Bligh, Erich Focht, Michael Hohnbaum, and others) around a simple observation: most of the NUMA problems can be solved by simply restricting the current scheduler's balancing code to processors within a single node. If the rebalancer - which moves processes across CPUs in order to keep them all busy - only balances inside a node, the worst processor imbalances will be addressed without moving processes into a foreign-node slow zone. A simple (three-line) patch which did nothing but add the within-node restriction yielded most of the benefits of the full NUMA scheduler; indeed, it performed better on some benchmarks. Real-world loads, however, will require a scheduler which can distribute processes evenly across nodes. Occasionally it is necessary, even, to move processes to a slower node; a lot of CPU time on a lightly-loaded node will give better performance than waiting in the run queue on a heavily-loaded node. So a bit of complexity had to be added back into the new scheduler to complete the job."
Extracted from:
http://lwn.net/Articles/20741/
You know Slashdot is going "mainstream" when people have to explain who Linus is.
Or when they have to even use his last name.
My journal has hot
I was waiting for someone to jump all over this...
T
---- It puts the lotion on its skin or else it gets the hose again. It does this whenever it's told.
"Non-Uniform Memory Architecture/Access." Not sure what that implies, except that sometimes you need to have data that is "far away" in a different memory (CPU group) node and use it like it was local.
Like being able to ignore the speed diffrerences (although they affect perormance).
Or when they claim that Linus created the OS GNU/Linux.
We are everywhere.
Sort some snow? Eat ice cream too fast? At any rate, you've experienced BRAIN FREEZE.
Why bother.
IBM not replacing AIX with Linxu? Yea rigth we really believe that type of FUD...:)
Don't Tread on OpenSource
Richard Stallman has sent an angry e-mail demanding a name change to GNUMA.
You can only drink 30 or 40 glasses of beer a day, no matter how rich you are.
-- Colonel Adolphus Busch
Sorry if this seems rude, but it sounds like someone crying 'sour grapes.' Do a little research and you can easily find news group discussions about the profiteer who wanted to charge people to use the name Linux. The parent post is correct about your opportunity to fork a better implementation. It's a good thing my moderation points elapsed. I would have been looking for a crybaby option ;-)
--
There is no hatred more pure and true than that expressed by children.
On that note, does one also need a hyperthreading mobo? I've searched, but after reading several linux kernel archives that google pointed me to, I'm still not sure whether my lowly 1.8 GHz P4 can hyperthread.
make world, not war
Contrary to what is said in the post, NUMA support has been in Linux for quite a while already. The recent patches accepted by Linus merely add NUMA awareness to the scheduler, which, while certainly being a prerequisite for Linux being used on production NUMA boxen, is not at all required for NUMA support in general.
quidquid latine dictum sit altum videtur.
It will be interesting how the Linus to the "next Linus" transition occurs... May be some vote perhaps... or would Linus appoint the next sucessor? :)
Since when were Dirk Pitt and Al Giordino kernel hackers? I thought they were marine engineers?
I wonder if this isn't some of the technology they snarfed from Sequent? I worked there in the late '80s and they were one of the top Unix technology shops around (or at least we thought so of ourselves at the time).
SGI has also integrated NUMA technology into the linux kernel to support their new Altix servers. How do these two efforts relate? Is SGI's code generic enough that it could also be considered for inclusion in the mainstream kernel? Or is it specific to SGI's NUMA architecture? Is IBM's code generic enough that it would work on an Altix? What functional characteristics distinguish the two?
--Lawrence Lessig for Congress!
...now I can put Linux on that 256-node NUMA cluster sitting in my spare bedroom...
Seriously, though, this is one of the strengths of the GPL and is proof that the Linux kernel can only advance in time as it sucks up more and more features that will never go away (I hope 'refactoring' is in the developers' vocabularies!).
Healthcare article at Kuro5hin
Proproetary companies are not despised. Most proprietary software is. Hardware companies that refuse to release driver level documentation are. Companies are not (unless everything that they do falls into the above catagories).
Thus: Proprietry hardware companies that give open documentation are not 'despised'. Companies that write GPL code are not 'despised'. They are valuable members of the linux comunity.
Because if it's GPL, and good, it gets in. It's not any more a handout than if you wrote some code, and submited it. There is nothing special about corporate ownership. If you have a problem with corporate submissions, you can modifiy the kernel, to remove their work. It's not a problem for me, or for the vast majority of the linux using comunity. I think that you seem to percive (or hold) an anti-corporate stance that is not representative of the comunity at large.
Whereas everyone in Asshatlandia is wise and good and would never be mean and bad like us bad and mean Americans.
[Set Cain on fire and steal his lute.]
I want to see this in a cluster. If the kernel can understand NUMA then it should be able to handle a cluster over a network, such as gigabit ethernet, IMO. Craylink is only what 800Mbps/full duplex? But, dammit Jim, I'm a Sys Admin, not a programmer.
If it were possible to scale a cluster of PCs easily and keep a single concurrent system image there would be no limits to what we're capable of doing. Encode a video in 30 minutes or less, play the latest games with 2 year old hardware, etc.
Make it so.
For one, this was submitted before the feature freeze, which was the requirement that came down from Linus, not "merged by the feature freeze".
It just got cleaned up since then.
Secondly, the reason Linus accepted this was not that it came from IBM, it's that it was split up into small readable pieces, well documented, and obviously had no effect WHATSOEVER on standard machines.
It's pathetic that you think Linus has so little personal integrity that he'd take bribes to put patches into the kernel. And who said this code was untested?
Martin J. Bligh.