Slashdot Mirror
2.5.65 On 32-way NUMA-Q with Preempt Enabled
_iris writes "I think the subject speaks for itself. Here is the link to the story on KernelTrap." In case you have a spare 32-processor machine munching grass in the back 40.
← Back to Stories (view on slashdot.org)
Now this is cool. I know that SGI can scale the Altix to 64 CPU's running 2.4 with their own additions in an SSI. However not sure about. 2.5. Its nice to see it in the main kernel anyway and the only way is up
Rus
Cheap UK and US VPS
Here is the complete article, the fscking lameness filter made it quite a struggle to get it posted here. Anyway:
;)
;)
;)
Zwane Mwaikambo announced today on the lkml that he's successfully boot the 2.5.65 development kernel on a 32-way NUMA-Q server with -preempt enabled. Speaking to Robert Love [interview], the kernel preemption maintainer, he began his announcement saying, "Robert, I suppose you can add another notch on your erm.. bedpost(?) and congratulations to all the kernel developers!" NUMA awareness in the scheduler was added into the development kernel in late January [story].
William Lee Irwin III [interview] explained the significance of this achievement:
"This has had a hard time historically. I'm really glad NUMA-Q's are now immune (in the sense of correctness) to this config; previously it was believed that preemption points in printk(linux_banner) would take out the machine early in boot if preemption was enabled. Congratulations rml! If you're booting without issues on these things, you are a _very_ long way toward being race-free. This is incredibly good news, both for the preemption support, and for the general stability of the i386 bootstrap."
Read on for the full thread.
From: Zwane Mwaikambo
Subject: 2.5.65-preempt booting on 32way NUMAQ
Date: Sun, 6 Apr 2003 06:48:33 -0400 (EDT)
Robert i suppose you can add another notch on your erm.. bedpost(?) and congratulations to all the kernel developers! It survived some local networking stress tests, but there is more fun stuff like tty layer to completely obliterate
(Hardware courtesy of OSDL)
Running configuration
32 Processors, PIII 500
32G RAM
Patches required:
2.5.65 (only because isp1020 decided to get huffy)
Purge assign_irq_vector panic - Zwane Mwaikambo
[boot messages]
From: Robert Love
Subject: Re: 2.5.65-preempt booting on 32way NUMAQ
Date: 06 Apr 2003 14:28:42 -0400
On Sun, 2003-04-06 at 06:48, Zwane Mwaikambo wrote:
> Robert i suppose you can add another notch on our erm.. bedpost(?)
> and congratulations to all the kernel developers! It survived some
> local networking stress tests, but there is more fun stuff like tty
> layer to completely obliterate
Excellent, Zwane.
Congratulations! Good work.
Robert Love
From: William Lee Irwin III
Subject: Re: 2.5.65-preempt booting on 32way NUMAQ
Date: Sun, 6 Apr 2003 04:23:40 -0700
On Sun, Apr 06, 2003 at 06:48:33AM -0400, Zwane Mwaikambo wrote:
> Robert i suppose you can add another notch on our erm.. bedpost(?)
> and congratulations to all the kernel developers! It survived some
> local networking stress tests, but there is more fun stuff like tty
> layer to completely obliterate
Wow!
This has had a hard time historically. I'm really glad NUMA-Q's are now immune (in the sense of correctness) to this config; previously it was
believed that preemption points in printk(linux_banner) would take out the machine early in boot if preemption was enabled.
Congratulations rml!
If you're booting without issues on these things, you are a _very_ long way toward being race-free. This is incredibly good news, both for the preemption support, and for the general stability of the i386 bootstrap.
All that's really left is driver and non-i386 arch coverage if I'm right.
-- wli
From: Zwane Mwaikambo
Subject: Re: 2.5.65-preempt booting on 32way NUMAQ
Date: Sun, 6 Apr 2003 07:25:09 -0400 (EDT)
On Sun, 6 Apr 2003, William Lee Irwin III wrote:
> This has had a hard time historically. I'm really glad NUMA-Q's are now
> immune (in the sense of correctness) to this config; previously it was
> believed that preemption points in printk(linux_banner) would take out
> the machine early in boot if preemption was enabled.
Which kernel version was that from
Preemt means preemtive in kernel space, you are talking about userspace. kerneltrap has an interview IIRC with Robert Love where the ins and outs are explained, if not, try google.
The Linux kernel usually isn't. That leads to high latency when the kernel does something which takes a long time. With the preemptive kernel patch, the kernel can no longer make certain assumptions, which means it has to synchronize some things which were guaranteed to complete without interruption before. The usual problems arise: Deadlocks, missing synchronization, etc.
http://www.kerneltrap.org/node.php?id=628
Have fun!
Daxy's Networking Blog
Yup, Linux, so up to date it's just beginning to suport hardware that hasn't been built for 2 years
The point is that 2.5.65 booted with preemtion patches on a 32 processor machine
That is preemtion of kernel threads. If there is a deadlock or race condition it would be more likely to show up un a beast like that than in your average dual athlon. So this is really not about supporting 32 processors (which is old news) but about the quality of the work that has gone into kernel preemtion
I have no idea if any other OSes out there support preemtion of kernel threads running on multiple cpus. Anyone care to enlighten me?
- We are the slashdot. Resistance is futile. Prepare to be moderated -
I have no idea if any other OSes out there support preemtion of kernel threads running on multiple cpus. Anyone care to enlighten me?
Solaris. I believe Mach does as well. There are probably others that aren't as well known.
IBM's x440 shipping 16-way NUMA/SMP system will benefit from this significantly...this machine will also soon be going to 32-way capability...so for those of you thinking there's no hardware out there today to take advantage of this...guess again.
IBM x440
It refers to preemptable work on BSD, but here is a good general description of kernel preemption.
But a NUMA is not purely a SMP machine either, its a little of both. The Sequent NUMA-Q is a series of Quad-proccessor systems (Quads) linked via a high speed bus. Each Quad has its own memory pool, but on a virtual level its also one big memory pool. Hell, I was ready to be certified on Dynix/ptx and I don't fully understand it beyond knowing its not Parrallel computing, and ints not SMP. Its NUMA (Non-uniform memory architecture!).
And its not really dead. Sequent was bought out by IBM, who stopped producing the Intel based systems, but is continuing to produce NUMA systems based on PowerPC systems. (or at least were; there is some embarassment here in that DB2 doesn't run right on NUMAs, so they have to benchmark with Oracle :^)
I also suspect the NUMA technology will be very important for upcoming SMP on a chip systems, if you have 4 CPU's on a chip, how do you combine them into a MP machine?
You are in a maze of twisted little posts, all alike.