To me, the Infiniband is the most interesting part. I'm very interested to see how the Infiniband scales and to see when they actually get the cluster working. I'm concerned about this because:
1) I'm not aware of any Infiniband cluster that big. I'm not a big follower of IB, but I'd be surprised if there were any other clusters running MPI over IB even 1/2 that large.
2) A Mellanox developer was asking basic questions about OSX driver development on the Darwin device drivers mailing list as recently as a few weeks ago. This leads me to believe that the MacOS X IB driver may not yet be ready for prime-time. Or may not even exist..
Yeah, I already do that. It gets me the openfirmware prompt that I mentioned above. But when I boot OS-X, the OS prints its messages on the VGA screen, not on the serial port.
What I want is for OS-X to put all its messages on the serial port.
OK, so the article tells us how to enable logins from a serial port. That's great, but that's really only 1/2 the battle.
For various reasons (mainly driver development in a cramped office), I like to run serial CONSOLES. This means I want to see the kernel messages on the serial port, not on a VGA monitor. This allows me to log all kernel messages, even messages from a machine which crashed (hence syslog is not running). Otherwise, its easy for important messages to scroll off screen and be lost when the system crashes.
When running linux, getting a serial console is as simple as passing some parameters to the kernel (console=ttyS0). Similar options exist for FreeBSD, Tru64, Solaris, etc. All of them will use a serial port for a console.
With OS-X, I've been able to enable some extra verbosity on the serial port, and I'm able to get an openfirmware prompt on the serial port, but I can't figure out how to make the serial port the actual system console. I know it must be possible, because the X serves are supposed to be able to do it.
Does anybody know how to do this on a "normal" g4 with a serial port (g4port)??
From my experience with a previous release of Apple IP over Firewire, it seems slow, and it has very high overhead. A dual 800MHz G4 host which can transmit at well over 1.4 Gb/sec (using ethernet-over-Myrinet) maxes its CPU out at 200Mb/sec, or less with IP over Firewire. GigE performance maxes out at something over 700Mb/sec. Both tests were run against a 15" 867MHz Powerbook directly connected (no switches in either case)
As other posters pointed out, GigE switch hardware is much more expensive. But if you're just interested in point-to-point data transfers without using target mode, GigE is much faster.
My 2.53 GHz P4 is a hair quieter than my Dual G4 800MHz Mac, and much, much quieter than some athlons we have at work. Both have the same hard drive (Seagate ST380021A). But the real benefit is that the P4 doesn't have a lame 400W vaccum-cleaner sounding gamer power supply like the athlons. It beats the hell out of the alpha 21264 I was using as a workstatin in terms of noise.
The P4 has a D845EBG2 motherboard, ECC ram, a PCPower&Cooling 275W silencer PS, and 2 PC Power&Cooling silencer fans, and a pc power&cooling med tower case.
Sure, its only 275 watts, but its my desktop, not a deparmental NFS server. I run X, surf the web and send email. I don't overlock, and I've only got 1 disk drive. Its worked flawlessly so far.
kedar at asacomputers.com built it for me.. I'm sure ASA will sell you one too;)
My wife and I bought a Sony 24" TV. Top of the line for a small TV. Nearly 2x what the cheaper manufacturers similary sized TVs cost. All kinds of features, wonderful picture. Happy story right? Wrong:
We got it home, watched TV for an hour. Then, BZAAAP!. The TV makes an incredibly loud zapping sound, and acts like we power-cycled it. We thought it was maybe a one time thing. But it happened twice the very next day.
So we took it back for a refund, and we're still using my wife's 10 year old 19" Zenith from college. And I'll probably never buy a Sony product again, since the extra price does not seem to pay for any quality.
If your company objects to the GPL, and you are selling embedded hardware, it seems rather silly to be using Linux. Have you considered using a BSD? Since the BSD license does not require you to give back any code, this might be ideal for you.
If you're running on non x86 hardware, I think NetBSD would be the best bet, since they run on nearly everything.
Don't drag FreeBSD's good name through the mud because OSX is a dog. The Darwin kernel has very little to do with FreeBSD.
The Darwin kernel is, in fact, a shotgun wedding of ancient, circa 1992 Mach with a (very few) patches taken from FreeBSD. The core of the OS is ancient, dusty, cruft Mach. If anything, MacOS's closest relative (besides Nextstep) is DEC Tru64 UNIX. It is notable that DEC did it right: Tru64 is as fast as a bat out of hell, and Darwin is a dog. Download the Darwin sources and take a look for yourself.
BTW, its obvious the author did not try very hard to disable the gui. One can simply log in on console with the '>console' login, no passwd, to exit the gui and get a text-mode getty on the monitor. In 10.1, this is the best way to get a stack trace when the machine crashes.
Thje Technote fails to mention the best thing about 10.2 -- the kernel is compiled to support ktrace(1). In 10.1, the kernel was not compiled to support ktrace.
For you linux people out there -- ktrace is a little like truss or strace, but it relies on tracepoints in the kernel, rather than/proc. It shows you everything the kernel is doing on your process' behalf, even things which may not show up as a system call (like signal posting). And following forks actually works.
I doubt these guys wrote the drivers entirely by themselves. This would require some very low level stuff, and lots of help from apple (because low level stuff is still being tweaked, etc.)...
Nothing was "tweaked." The driver is available
as a binary or source kernel module. No kernel patches or special kernels are required.
While I'm not a huge fan of IOKit (just the opposite, in fact), I will say that Apple has fairly decent high-level APIs for pinning user's memory (the only really tricky thing an OS bypass driver needs to do) because its based upon BSD and Mach.. The MD driver code to pin and unpin a user's buffer in OS-X is 80 lines of code including comments. (and it is roughly similar in FreeBSD and Tru64). Linux, on the other hand, requires 300 lines to do the same job & does involve low-level access to the OS (low-level enough that new kernel versions require driver updates).
At least for micro benchmarks, the unix performance really, really sucks. See
http://lists.apple.com/archives/darwin-kernel/2001/Dec/2.html
Toward the bottom there's lmbench output indicating that Darwin is roughly an order of magnitude slower than Linux or NetBSD on the same hardware for things like syscalls, context switch, file creation, etc.
There is a packet filter (as well as a bpf):
~>uname -a
Darwin g4 1.4 Darwin Kernel Version 1.4: Sun Sep 9 15:39:59 PDT 2001; root:xnu/xnu-201.obj~1/RELEASE_PPC Power Macintosh powerpc
~>sudo kmodstat | grep Firewall
59 0 0x15053000 0x4000 0x3000 com.apple.nke.IPFirewall (1.1)
~>sudo ipfw list
65535 allow ip from any to any
The first boot on real hardware to single
user mode happened about 2 weeks ago. See
http://docs.FreeBSD.org/cgi/getmsg.cgi?fetch=3921+ 0+archive/2001/freebsd-ia64/20011007.freebsd-ia64
The IA64 port is booting multi-user now, and has been for quite some time.
You're forgetting that the BSDs have many flame wars of their own.
The advantage of Linux in in regard to these sorts of flamewars is that in Linus, you have a divine-right monarch. A final arbiter, if you will. While its true that the BSDs have core teams, those core teams don't engender the sort of respect (hero worship?) that Linus appears to have. After a controversial decision is reached by core, there will be a lot of biching & moaning from the developers. The loosing parties sometimes flee to other BSDs or even Linux. Sometimes large numbers of developers start talking about overthrowing core. About every 4 years or so, BSDs split because of flame-wars like this. When is the last time the Linux kernel split?
If the BSDs had a king like Linus to resolve disputes, things would be quite a bit different now. There would be one BSD and it would probably have more marketshare than Linux.
Slashdot Mirror
Sure, but the most important to me is the ability to address large amounts of memory in a single process.
I wonder when apple will release enough of the specs so that a real 64-bit OS like NetBSD or Linux can run on it.
Heh. I gave up programming in assembly years ago.
Anyway, if the OS only provides a 32-bit address space, I don't see how the programming language is relavent.
OK, so the answer is "No". Its not possible to write a 64-bit app. Just like x86/PAE. Sigh.
Does Panther make the G5 a truly 64-bit platform?
Ie, does it allow applications to address more than 32 bits (4GB) of RAM? What's sizeof(void *)?
We should all call him at dinner time and let him know what we think about his ruling.
To me, the Infiniband is the most interesting part. I'm very interested to see how the Infiniband scales and to see when they actually get the cluster working. I'm concerned about this because:
1) I'm not aware of any Infiniband cluster that big. I'm not a big follower of IB, but I'd be surprised if there were any other clusters running MPI over IB even 1/2 that large.
2) A Mellanox developer was asking basic questions about OSX driver development on the Darwin device drivers mailing list as recently as a few weeks ago. This leads me to believe that the MacOS X IB driver may not yet be ready for prime-time. Or may not even exist..
Yeah, I already do that. It gets me the openfirmware prompt that I mentioned above. But when I boot OS-X, the OS prints its messages on the VGA screen, not on the serial port.
What I want is for OS-X to put all its messages on the serial port.
OK, so the article tells us how to enable logins from a serial port. That's great, but that's really only 1/2 the battle.
For various reasons (mainly driver development in a cramped office), I like to run serial CONSOLES. This means I want to see the kernel messages on the serial port, not on a VGA monitor. This allows me to log all kernel messages, even messages from a machine which crashed (hence syslog is not running). Otherwise, its easy for important messages to scroll off screen and be lost when the system crashes.
When running linux, getting a serial console is as simple as passing some parameters to the kernel (console=ttyS0). Similar options exist for FreeBSD, Tru64, Solaris, etc. All of them will use a serial port for a console.
With OS-X, I've been able to enable some extra verbosity on the serial port, and I'm able to get
an openfirmware prompt on the serial port, but I can't figure out how to make the serial port the actual system console. I know it must be possible, because the X serves are supposed to be able to do it.
Does anybody know how to do this on a "normal" g4 with a serial port (g4port)??
Yeah, it only happened 10 years ago..
First 64-bit CPU: DEC Alpha, 1993
First 64-bit OS: DEC OSF/1, 1993
From my experience with a previous release of Apple IP over Firewire, it seems slow, and it has very high overhead. A dual 800MHz G4 host which can transmit at well over 1.4 Gb/sec (using ethernet-over-Myrinet) maxes its CPU out at
200Mb/sec, or less with IP over Firewire. GigE performance maxes out at something over 700Mb/sec. Both tests were run against a 15" 867MHz Powerbook directly connected (no switches in either case)
As other posters pointed out, GigE switch hardware is much more expensive. But if you're just interested in point-to-point data transfers without using target mode, GigE is much faster.
I figure that must be how his wife broke the old light ;)
My 2.53 GHz P4 is a hair quieter than my Dual G4 800MHz Mac, and much, much quieter than some athlons we have at work. Both have the same hard drive (Seagate ST380021A). But the real benefit is that the P4 doesn't have a lame 400W vaccum-cleaner sounding gamer power supply like the athlons. It beats the hell out of the alpha 21264 I was using as a workstatin in terms of noise.
;)
The P4 has a D845EBG2 motherboard, ECC ram, a PCPower&Cooling 275W silencer PS, and 2 PC Power&Cooling silencer fans, and a pc power&cooling med tower case.
Sure, its only 275 watts, but its my desktop, not a deparmental NFS server. I run X, surf the web and send email. I don't overlock, and I've only got 1 disk drive. Its worked flawlessly so far.
kedar at asacomputers.com built it for me.. I'm sure ASA will sell you one too
That's what I thought too. Until recently.
My wife and I bought a Sony 24" TV. Top of the line for a small TV. Nearly 2x what the cheaper manufacturers similary sized TVs cost. All kinds of features, wonderful picture. Happy story right? Wrong:
We got it home, watched TV for an hour. Then, BZAAAP!. The TV makes an incredibly loud zapping sound, and acts like we power-cycled it. We thought it was maybe a one time thing. But it happened twice the very next day.
So we took it back for a refund, and we're still using my wife's 10 year old 19" Zenith from college. And I'll probably never buy a Sony product again, since the extra price does not seem to pay for any quality.
Sigh.
If your company objects to the GPL, and you are selling embedded hardware, it seems rather silly to be using Linux. Have you considered using a BSD? Since the BSD license does not require you to give back any code, this might be ideal for you.
If you're running on non x86 hardware, I think NetBSD would be the best bet, since they run on nearly everything.
Don't drag FreeBSD's good name through the mud because OSX is a dog. The Darwin kernel has very little to do with FreeBSD.
The Darwin kernel is, in fact, a shotgun wedding of ancient, circa 1992 Mach with a (very few) patches taken from FreeBSD. The core of the OS is ancient, dusty, cruft Mach. If anything, MacOS's closest relative (besides Nextstep) is DEC Tru64 UNIX. It is notable that DEC did it right: Tru64 is as fast as a bat out of hell, and Darwin is a dog. Download the Darwin sources and take a look for yourself.
BTW, its obvious the author did not try very hard to disable the gui. One can simply log in on console
with the '>console' login, no passwd, to exit the gui and get a text-mode getty on the monitor. In 10.1,
this is the best way to get a stack trace when the machine crashes.
Thje Technote fails to mention the best thing about 10.2 -- the kernel is compiled to support ktrace(1). In 10.1, the kernel was not compiled to support ktrace.
/proc. It shows you everything
For you linux people out there -- ktrace is a little like truss or strace, but it relies on tracepoints in the kernel, rather than
the kernel is doing on your process' behalf, even things which may not show up as a system call (like signal posting). And following forks actually works.
I doubt these guys wrote the drivers entirely by themselves. This would require some very low level stuff, and lots of help from apple (because low level stuff is still being tweaked, etc.)...
Nothing was "tweaked." The driver is available as a binary or source kernel module. No kernel patches or special kernels are required.
While I'm not a huge fan of IOKit (just the opposite, in fact), I will say that Apple has fairly decent high-level APIs for pinning user's memory (the only really tricky thing an OS bypass driver needs to do) because its based upon BSD and Mach.. The MD driver code to pin and unpin a user's buffer in OS-X is 80 lines of code including comments. (and it is roughly similar in FreeBSD and Tru64). Linux, on the other hand, requires 300 lines to do the same job & does involve low-level access to the OS (low-level enough that new kernel versions require driver updates).
At least for micro benchmarks, the unix performance really, really sucks. See1 /Dec/2.html
http://lists.apple.com/archives/darwin-kernel/200
Toward the bottom there's lmbench output indicating that Darwin is roughly an order of magnitude slower than Linux or NetBSD on the same hardware for things like syscalls, context switch, file creation, etc.
There is a packet filter (as well as a bpf):
~>uname -a
Darwin g4 1.4 Darwin Kernel Version 1.4: Sun Sep 9 15:39:59 PDT 2001; root:xnu/xnu-201.obj~1/RELEASE_PPC Power Macintosh powerpc
~>sudo kmodstat | grep Firewall
59 0 0x15053000 0x4000 0x3000 com.apple.nke.IPFirewall (1.1)
~>sudo ipfw list
65535 allow ip from any to any
~>sudo tcpdump -i en0
tcpdump: listening on en0
13:15:13.707607 g4.22 > thunder.1357: P 2129343678:2129343722(44) ack 994474742 win 33304 (DF) [tos 0x10]
The first boot on real hardware to single+ 0+archive/2001/freebsd-ia64/20011007.freebsd-ia64
user mode happened about 2 weeks ago. See
http://docs.FreeBSD.org/cgi/getmsg.cgi?fetch=3921
The IA64 port is booting multi-user now, and has been for quite some time.
You're forgetting that the BSDs have many flame wars of their own.
The advantage of Linux in in regard to these sorts of flamewars is that in Linus, you have a divine-right monarch. A final arbiter, if you will. While its true that the BSDs have core teams, those core teams don't engender the sort of respect (hero worship?) that Linus appears to have. After a controversial decision is reached by core, there will be a lot of biching & moaning from the developers. The loosing parties sometimes flee to other BSDs or even Linux. Sometimes large numbers of developers start talking about overthrowing core. About every 4 years or so, BSDs split because of flame-wars like this. When is the last time the Linux kernel split?
If the BSDs had a king like Linus to resolve disputes, things would be quite a bit different now. There would be one BSD and it would probably have more marketshare than Linux.
Trust me, you don't want a core team.