Indeed - the memory management units are in hardware on each side of the interconnects, meaning that it is transparent to the software. Not totally, because the software (kernel + userspace utils) makes the decision as to where on the machine it places memory on malloc()
So yes, there is only a single copy of the Operating System running, but that doesn't mean it's SMP. SMP code will run without recompilation, but won't be as efficient or perform anywhere near as well as hand optimized code. But they definitely aren't SMP - they're a cross between a cluster and an SMP machine.
I highly recommend reading John Mashey's comp.arch posting to comp.arch which explains the Origin, and what became the Altix. Origin was essentially Altix with MIPS CPUs, not Itanium.
I worked on a 64 CPU MIPS Origin 3000 for a year in a research role.
OS/2 Warp 4 and before used HPFS, which was quite similar to NTFS, I believe. Most of that code probably belongs to Microsoft. JFS was part of one of the final releases of the OS/2 server product - afaik never the client.
I should elaborate a bit - it's the shared bus that kills the scalability of SMP, which traditionally Windows has been limited to (by choice - NUMA isn't a big market). I have some figures to back this too - SGI's last large-scale, bus-based machine was the "Power Challenge R10000". If you look at the scalability of memory bandwidth:
1 CPU - 175 MB/s
2 CPU - 350 MB/s
3 CPU - 460 MB/s
4 CPU - 540 MB/s
5 CPU - 590 MB/s
6 CPU - 590 MB/s
As you can see, the bus to memory gets saturated as the number of CPUs accessing memory rises. Hence, NUMA. Whether it's Windows or UNIX, we're talking the same hardware. Just happens that all these boxes are UNIX because most researchers have always used UNIX.
Incidentally, if you can get the percentage of cache hits relatively high then you can cover up for some of the memory bottlenecks (Apple have done this in the past with their G4 hardware in particular). But typically, cache hits aren't commercial softwares strongpoint - scientific code fares much better here.
(Numbers lifted from chapter 2 of my honours thesis, which quoted them from some university website which appears to have taken the numbers for that particular machine configuration down since)
I have done a fair amount of work on a 32 way SGI Origin (NUMA IRIX) box, which are the predecessor to the now famed SGI Altix Linux boxes.
One common misconception is that these are 64 way SMP boxes. They aren't. See, they're really 4 CPU bricks (C-bricks) with low latency, high bandwidth interconnects and smart memory management units to do memory references on remote boxes.
Sun Fire 15k, HP SuperDome, DEC/Compaq/HP AlphaServer GS, and IBM p690 all work like this too.
So much of the brains is in hardware. The rest is kernel changes to be bright about memory placement (physically put the memory close to the CPU that the process is running on), and provide some user space APIs to give the kernel hints (overrides, really) about where it should place the memory. Given this, there's no reason why Windows couldn't do this (remember, UNIX was written for DEC PDP-11, not SGI Origin/Altix). Whether it does or not, I don't know.
I heard OS/2 has a lot of assembly code in it too, and a lot of the code is Microsoft's and other third parties. After all, it was the result of a collaborative agreement with Microsoft.
While we're on that point - In all seriousness, has IBM ever been involved in a partnership development of an OS that has been successful (disclaim Linux - they don't control it). The only two cases I can think of were Project Monterey and OS/2 - both were heralded as the next big thing, and neither delivered.
So the question is - is partnering with IBM a good thing?
Yep, while I was at ANU, we had an HP/Compaq guy full time with us to help maintain a ~500 CPU Alpha cluster we had. HP were keen to show us Itanium, so they loaned us one. Changing between the release version of the Intel compiler at the time to the beta of the next version yielded a 30% increase in performance for no code change at all.
The HP guys were saying that they were expecting these machines to get faster with age, because the compilers were making code run faster with each release. So yes, Itanium performance is largely dependant on the compiler.
I've read that most of the performance gain from amd64 comes from the extra general purpose registers it has available to it in 64 bit mode. Hence, I'd like to see the evidence backing your claim that amd64 is faster than anything Intel on a 32 bit OS.
Xeon is generally one generation behind on chipsets from the P4 for some reason. I guess they need to add features required by Xeon customers, and that takes time. Maybe??
I've found Java apps to be an exception. The amd64 JVM from Sun is much slower and less reliable than the 32 bit one. So the amd64 on my desk runs a fair bit slower than the dual xeon we have in production.
YMMV - but the next Java app server I purchase in a couple of months will run a 32 bit copy of SLES regardless of which CPU it has. It'll probably be Xeon because I quite like Dell's 24x7x4hr support, and they don't to AMD.
YaST is slow, but ok. YOU (YaST Online Update) is a joke compared to RHN (Red Hat Network). I'm a current SLES9 subscriber, and a previous RHEL customer, so I've used both.
I have a long list of YOU hates. The most frequent and irritating is that we get emails on *every* patch SuSE ship (including for SLES8 only stuff), and for things I don't have installed. They just send a bulk email to all their subscribers. RHN send an email with the package, and the *list* of servers we have that will be / need to be updated.
Having said that, SLES has its advantages. Support for Jakarta Tomcat in the base distribution (not a $1k/yr Red Hat App server optional extra) is high on that list...
Converting RPMs to Deb using alien isn't 100% a solution. It doesn't always get things like post-install scripts right. Try alien'ing a Sun JDK RPM, which runs said scripts on install.
But you're right - *most* do work. Not all.
In other news, ever tried installing Oracle on Debian. It's doable, but not much fun. It's documented and supported on RHEL/SLES.
In all seriousness, when you need RHEL/SLES you'll know it, and it makes AUD$550/yr look like good value when you're in that position. Currently I've got two SLES boxes, and a Debian box in production at work. The SLES boxes run applications I don't want to admin for 5 years, and I'm happy to pay for the privelige of not doing so.
I've got the similar keyboard from Kenesis Ergo, which comes with "hardware" support for switching between dvorak and qwerty, and sculpted like the keyboard above. Cost a fortune, but I love it and would happily buy another.
But wouldn't buy the dvorak one again. It's twice the price as the qwerty only, and I never really put the effort into learning dvorak anyway. It's always been in that "yeah, one day" bucket.
Slashdot Mirror
Indeed - the memory management units are in hardware on each side of the interconnects, meaning that it is transparent to the software. Not totally, because the software (kernel + userspace utils) makes the decision as to where on the machine it places memory on malloc()
So yes, there is only a single copy of the Operating System running, but that doesn't mean it's SMP. SMP code will run without recompilation, but won't be as efficient or perform anywhere near as well as hand optimized code. But they definitely aren't SMP - they're a cross between a cluster and an SMP machine.
I highly recommend reading John Mashey's comp.arch posting to comp.arch which explains the Origin, and what became the Altix. Origin was essentially Altix with MIPS CPUs, not Itanium.
I worked on a 64 CPU MIPS Origin 3000 for a year in a research role.
OS/2 Warp 4 and before used HPFS, which was quite similar to NTFS, I believe. Most of that code probably belongs to Microsoft. JFS was part of one of the final releases of the OS/2 server product - afaik never the client.
I should elaborate a bit - it's the shared bus that kills the scalability of SMP, which traditionally Windows has been limited to (by choice - NUMA isn't a big market). I have some figures to back this too - SGI's last large-scale, bus-based machine was the "Power Challenge R10000". If you look at the scalability of memory bandwidth:
As you can see, the bus to memory gets saturated as the number of CPUs accessing memory rises. Hence, NUMA. Whether it's Windows or UNIX, we're talking the same hardware. Just happens that all these boxes are UNIX because most researchers have always used UNIX.
Incidentally, if you can get the percentage of cache hits relatively high then you can cover up for some of the memory bottlenecks (Apple have done this in the past with their G4 hardware in particular). But typically, cache hits aren't commercial softwares strongpoint - scientific code fares much better here.
(Numbers lifted from chapter 2 of my honours thesis, which quoted them from some university website which appears to have taken the numbers for that particular machine configuration down since)
I have done a fair amount of work on a 32 way SGI Origin (NUMA IRIX) box, which are the predecessor to the now famed SGI Altix Linux boxes.
One common misconception is that these are 64 way SMP boxes. They aren't. See, they're really 4 CPU bricks (C-bricks) with low latency, high bandwidth interconnects and smart memory management units to do memory references on remote boxes.
Sun Fire 15k, HP SuperDome, DEC/Compaq/HP AlphaServer GS, and IBM p690 all work like this too.
So much of the brains is in hardware. The rest is kernel changes to be bright about memory placement (physically put the memory close to the CPU that the process is running on), and provide some user space APIs to give the kernel hints (overrides, really) about where it should place the memory. Given this, there's no reason why Windows couldn't do this (remember, UNIX was written for DEC PDP-11, not SGI Origin/Altix). Whether it does or not, I don't know.
I heard OS/2 has a lot of assembly code in it too, and a lot of the code is Microsoft's and other third parties. After all, it was the result of a collaborative agreement with Microsoft.
While we're on that point - In all seriousness, has IBM ever been involved in a partnership development of an OS that has been successful (disclaim Linux - they don't control it). The only two cases I can think of were Project Monterey and OS/2 - both were heralded as the next big thing, and neither delivered.
So the question is - is partnering with IBM a good thing?
Yep, while I was at ANU, we had an HP/Compaq guy full time with us to help maintain a ~500 CPU Alpha cluster we had. HP were keen to show us Itanium, so they loaned us one. Changing between the release version of the Intel compiler at the time to the beta of the next version yielded a 30% increase in performance for no code change at all.
The HP guys were saying that they were expecting these machines to get faster with age, because the compilers were making code run faster with each release. So yes, Itanium performance is largely dependant on the compiler.
I've read that most of the performance gain from amd64 comes from the extra general purpose registers it has available to it in 64 bit mode. Hence, I'd like to see the evidence backing your claim that amd64 is faster than anything Intel on a 32 bit OS. Xeon is generally one generation behind on chipsets from the P4 for some reason. I guess they need to add features required by Xeon customers, and that takes time. Maybe?? I've found Java apps to be an exception. The amd64 JVM from Sun is much slower and less reliable than the 32 bit one. So the amd64 on my desk runs a fair bit slower than the dual xeon we have in production. YMMV - but the next Java app server I purchase in a couple of months will run a 32 bit copy of SLES regardless of which CPU it has. It'll probably be Xeon because I quite like Dell's 24x7x4hr support, and they don't to AMD.
YaST is slow, but ok. YOU (YaST Online Update) is a joke compared to RHN (Red Hat Network). I'm a current SLES9 subscriber, and a previous RHEL customer, so I've used both. I have a long list of YOU hates. The most frequent and irritating is that we get emails on *every* patch SuSE ship (including for SLES8 only stuff), and for things I don't have installed. They just send a bulk email to all their subscribers. RHN send an email with the package, and the *list* of servers we have that will be / need to be updated. Having said that, SLES has its advantages. Support for Jakarta Tomcat in the base distribution (not a $1k/yr Red Hat App server optional extra) is high on that list...
Converting RPMs to Deb using alien isn't 100% a solution. It doesn't always get things like post-install scripts right. Try alien'ing a Sun JDK RPM, which runs said scripts on install. But you're right - *most* do work. Not all. In other news, ever tried installing Oracle on Debian. It's doable, but not much fun. It's documented and supported on RHEL/SLES. In all seriousness, when you need RHEL/SLES you'll know it, and it makes AUD$550/yr look like good value when you're in that position. Currently I've got two SLES boxes, and a Debian box in production at work. The SLES boxes run applications I don't want to admin for 5 years, and I'm happy to pay for the privelige of not doing so.
I've got the similar keyboard from Kenesis Ergo, which comes with "hardware" support for switching between dvorak and qwerty, and sculpted like the keyboard above. Cost a fortune, but I love it and would happily buy another. But wouldn't buy the dvorak one again. It's twice the price as the qwerty only, and I never really put the effort into learning dvorak anyway. It's always been in that "yeah, one day" bucket.