If you get a Myrinet cluster and run IP over it I think it uses the GM kernel driver which does exactly DMA remote access. The NIC has to be smart enough to handle this of course.
Cplant style clusters do this as well. They also provide an API called Portals which revolves around RDMA. Portals, incidentally, is being used in the Lustre cluster filesystem and is implemented in kernel space for that project. It can use TCP/IP I believe but its not real RDMA.
*sigh* some day all NICs will be smart enough to not interrupt the CPU to do data delivery. That would rock:). But what of security?
I bought Baldur's Gate... There are a few "overly-sexy" characters in the game which seemingly are there for men's eyes only. The game itself is outstanding. I played it all the way through with my girlfriend [its cooperative action RPG... great fun]. I could see she was visibly upset at the jiggling boobies in the game. It totally took away from her experience.
My point is GOOD GAMES DON'T NEED to throw sex in our face to make us play. I mean really... who played Zork or Space Quest? Was there a ton of sex in those games? Did a lot of people play them? I *know* we can do better than this.
Video games were a gateway for me into computer science and programming as a child... by appealing only to men you add to the problem of pushing the feminine percentages down in this field.
I have to admit that this is because I find somewhat nerdy girls attractive:)... but that's besides the point. I think the field is missing out on contributions of the female gender and until we stop buying this rubbish the publishers will give it to us cuz it sells. Welcome to Capitalism:{
Inheriting everything from "Object" isn't good enough either... which is why Java 1.5 will have templates as well.
Often times you want the compiler to be able to do some type checking at compile time to make sure you can even call certain functions on certain objects.
OOP is ok... but it can't solve all problems easily and refactoring of old OOP designs is a huge pain in the ass sometimes. This is why there are technologies like Aspect Oriented Programming in existence. AspectJ is a good example of this and I believe it is available with the popular eclipse IDE.
Just so you know KGI is the Kernel Graphics Interface project that was to be the underlying layer of GGI in Linux.
I used to mess around with this back when Linus was saying how much it was a bad idea. The neat thing about ggi applications is if you compile them once they will run within X as well as at the console without a recompile. At least that used to be a goal... Admittedly I stopped following this project a long time ago but I am glad that it appears to be moving forward elsewhere:).
The reason is that the Darwin kernel runs in one address space (even both the Mach and BSD personalities run the same address space).
Actually the fact that they are running BSD + Mach [the XNU components] in the same addreess space is a speed optimization to reduce context switching for every message from BSD to Mach. This isn't a cause of slowness but a reduction. Can you imagine how bad it would have been if they hadn't done this? This is documented on Apple's website in Kernel programming so I am not just making it up. Its in the section where they clarify that XNU itself is not a proper microkernel.
Apple didn't just pick up the Mach kernel and used it, they improved it a lot and one of the things they did was to rip out all that message passing stuff, while still retaining the modular design of the kernel.
The message passing stuff still exists and is still useful [while it may not be used in the kernel space... I haven't verified this yet.] Walking through the source of CFMessagePort stuff shows you that it calls mach_msg a lot actually. It's also super fast due to out-of-line memory transfers etc.
I think there might be a grain of truth in the fact that KDE has very hard time winning the desktop. Gnome has the huge advantage of licensing (LGPL vs. GPL). It doesn't matter how much smoother or better the technology underlying KDE or KDE applications is.
Are you kidding? If you read the LGPL anyone can take any "copy" of the software that is LGPLd and relicense it as GPL as they deem fit... there will still be an LGPL version out there, that you cannot prevent, but it may be used just the same as any GPL software as far as I care:).
KDE people also have the weird habit of producing their own versions of various pieces of software. Surely a conservative decisionmaker will choose a desktop-agnostic Mozilla or OpenOffice over the KDE-specific versions. KDE applications might do better by just dropping the K from their names, thus competing on their own terms (snappines and other virtues associated with Qt).
Well Mozilla is really huge... Konqueror is pretty tiny in comparison. Konqueror also has the ability to use all the good KDE technology like KIOslaves [kind of like libferris in functionality] transparently... KDE is not just a desktop its a complete framework with which to develop applications. A fairly rich one at that. Why would anyone running KDE not want all KDE apps in this light?
I thought pthreads were going to be based on KSE's [Kernel Scheduling Entities and yes it is possible to see something less than a kernel process as a schedulable context and to build one's threads and processes upon it.. though I will admit to not being very familiar with the architecture of KSE's yet.:)] and not be full processes. In fact I am 100% positive that FreeBSD 5.0 is going for M:N mapping of threads to processes meaning that they are not planning to do the 1:1 that linux had for years up until NPTL came out.
Linux used to use clone to get a new LWP but 2.6 should have some much newer, better stuff.
Yeah but... They don't really support all linux. You can buy a Mac with an Nvidia card and load linux on it and those binary only drivers will *not* work.
Actually that means its not fault tolerant...:) You are the unstable one who ejects things without unmounting them.
The kernel does panic if you try to mount a CD that wasn't fixated though... or at least it did back in 4.6. That is the same kind of fault intolerance as the floppy eject thing I bet.
It might be... the message I posted about currently getting < 100Mbps performance from firewire ethernet was exactly a test on a cluster of firewire. [Albeit a cluster of 2 nodes but it was running an MPI job].
I just tried it here and its slightly worse than fast ethernet. Probably due to its prerelease status... No doubt its cool stuff though the performance is very erratic.
Mach usually requires another special process to do TCP/IP message forwarding... my understanding is Mac OS X doesn't currently have this...
You are correct about the design of Mach being such that it can operate on a network transparently. The caveat is you need all the appropriate pieces to do so.
Latency is nowhere near as important as bandwidth. If you can successfully overlap communication with computation in say an MPI application you achieve much better wall clock times for the runs of big jobs.
I have no idea why people are so obsessed with latency in clusters... perhaps the apps are coded with all blocking communication semantics and cannot do the overlap I described above. Basically if you can stream data to the processors and keep them busy... that's all you need... latency may or may not be important in that situation.
I actually develop a commercial MPI implementation and we have seen real evidence of my claims above. If you are going to SuperComputing 2002 in Baltimore Maryland next week stop by the MPI Software Technology Inc. booth [I won't actually be there... but some of my demos will be]... ask some questions and try to talk to one of our technical guys... they may be better able to explain this since I am not a senior engineer.:)
[This should be taken not as a representative statement on behalf of the company I work for but as my own humble interpretation of the real issues].
Lots of crossover cables eh? That would indeed make for point to point connections but imagine having eth0-eth7 to manage in your startup scripts... What would/etc/hosts look like? can these nics all get the same IP?
Sounds like a fundamental design that should be thrown away so you can do what you really want to do. I think most good software engineers see when things are going to "dead-end" and should abandon the losing design in favor of a better one.
Did BSD-Lites have this partition issue? It ran a Unix server on top of Mach too I believe. To force people to run a 64bit architecture to run Hurd seems like a bit of a non-liberating liberation:)
Slashdot Mirror
If you get a Myrinet cluster and run IP over it I think it uses the GM kernel driver which does exactly DMA remote access. The NIC has to be smart enough to handle this of course.
:). But what of security?
Cplant style clusters do this as well. They also provide an API called Portals which revolves around RDMA. Portals, incidentally, is being used in the Lustre cluster filesystem and is implemented in kernel space for that project. It can use TCP/IP I believe but its not real RDMA.
*sigh* some day all NICs will be smart enough to not interrupt the CPU to do data delivery. That would rock
Don't know myself.
I bought Baldur's Gate... There are a few "overly-sexy" characters in the game which seemingly are there for men's eyes only. The game itself is outstanding. I played it all the way through with my girlfriend [its cooperative action RPG ... great fun]. I could see she was visibly upset at the jiggling boobies in the game. It totally took away from her experience.
:)... but that's besides the point. I think the field is missing out on contributions of the female gender and until we stop buying this rubbish the publishers will give it to us cuz it sells. Welcome to Capitalism :{
My point is GOOD GAMES DON'T NEED to throw sex in our face to make us play. I mean really... who played Zork or Space Quest? Was there a ton of sex in those games? Did a lot of people play them? I *know* we can do better than this.
Video games were a gateway for me into computer science and programming as a child... by appealing only to men you add to the problem of pushing the feminine percentages down in this field.
I have to admit that this is because I find somewhat nerdy girls attractive
HEHEH :)
Suddenly slashdot is run by John C. Dvorak it seems. :)
Inheriting everything from "Object" isn't good enough either... which is why Java 1.5 will have templates as well.
Often times you want the compiler to be able to do some type checking at compile time to make sure you can even call certain functions on certain objects.
OOP is ok... but it can't solve all problems easily and refactoring of old OOP designs is a huge pain in the ass sometimes. This is why there are technologies like Aspect Oriented Programming in existence. AspectJ is a good example of this and I believe it is available with the popular eclipse IDE.
KGI FreeBSD project
:).
Just so you know KGI is the Kernel Graphics Interface project that was to be the underlying layer of GGI in Linux.
I used to mess around with this back when Linus was saying how much it was a bad idea. The neat thing about ggi applications is if you compile them once they will run within X as well as at the console without a recompile. At least that used to be a goal... Admittedly I stopped following this project a long time ago but I am glad that it appears to be moving forward elsewhere
The reason is that the Darwin kernel runs in one address space (even both the Mach and BSD personalities run the same address space).
Actually the fact that they are running BSD + Mach [the XNU components] in the same addreess space is a speed optimization to reduce context switching for every message from BSD to Mach. This isn't a cause of slowness but a reduction. Can you imagine how bad it would have been if they hadn't done this? This is documented on Apple's website in Kernel programming so I am not just making it up. Its in the section where they clarify that XNU itself is not a proper microkernel.
Apple didn't just pick up the Mach kernel and used it, they improved it a lot and one of the things they did was to rip out all that message passing stuff, while still retaining the modular design of the kernel.
The message passing stuff still exists and is still useful [while it may not be used in the kernel space... I haven't verified this yet.] Walking through the source of CFMessagePort stuff shows you that it calls mach_msg a lot actually. It's also super fast due to out-of-line memory transfers etc.
I think there might be a grain of truth in the fact that KDE has very hard time winning the desktop. Gnome has the huge advantage of licensing (LGPL vs. GPL). It doesn't matter how much smoother or better the technology underlying KDE or KDE applications is.
:).
Are you kidding? If you read the LGPL anyone can take any "copy" of the software that is LGPLd and relicense it as GPL as they deem fit... there will still be an LGPL version out there, that you cannot prevent, but it may be used just the same as any GPL software as far as I care
KDE people also have the weird habit of producing their own versions of various pieces of software. Surely a conservative decisionmaker will choose a desktop-agnostic Mozilla or OpenOffice over the KDE-specific versions. KDE applications might do better by just dropping the K from their names, thus competing on their own terms (snappines and other virtues associated with Qt).
Well Mozilla is really huge... Konqueror is pretty tiny in comparison. Konqueror also has the ability to use all the good KDE technology like KIOslaves [kind of like libferris in functionality] transparently... KDE is not just a desktop its a complete framework with which to develop applications. A fairly rich one at that. Why would anyone running KDE not want all KDE apps in this light?
Pretty sure Walter has listed in his FAQ that he will have templates or something like them as well
I thought pthreads were going to be based on KSE's [Kernel Scheduling Entities and yes it is possible to see something less than a kernel process as a schedulable context and to build one's threads and processes upon it.. though I will admit to not being very familiar with the architecture of KSE's yet. :)] and not be full processes. In fact I am 100% positive that FreeBSD 5.0 is going for M:N mapping of threads to processes meaning that they are not planning to do the 1:1 that linux had for years up until NPTL came out.
Linux used to use clone to get a new LWP but 2.6 should have some much newer, better stuff.
Yeah but... They don't really support all linux. You can buy a Mac with an Nvidia card and load linux on it and those binary only drivers will *not* work.
Actually that means its not fault tolerant... :) You are the unstable one who ejects things without unmounting them.
The kernel does panic if you try to mount a CD that wasn't fixated though... or at least it did back in 4.6. That is the same kind of fault intolerance as the floppy eject thing I bet.
Like ones where we can say "Go search on Google".
It might be... the message I posted about currently getting < 100Mbps performance from firewire ethernet was exactly a test on a cluster of firewire. [Albeit a cluster of 2 nodes but it was running an MPI job].
Dave
I just tried it here and its slightly worse than fast ethernet. Probably due to its prerelease status... No doubt its cool stuff though the performance is very erratic.
Senator's aimlessly to wire broadband....
Mach usually requires another special process to do TCP/IP message forwarding... my understanding is Mac OS X doesn't currently have this...
You are correct about the design of Mach being such that it can operate on a network transparently. The caveat is you need all the appropriate pieces to do so.
Its because the demo program uses altivec... you can run that demo standalone [one machine] but you must have a G4.
They effectively couldn't compare unless someone wants to write the SSE1 or SSE2 equivalent.
Absolutely wrong,[well... not absolutely :)]
... that's all you need... latency may or may not be important in that situation.
:)
Latency is nowhere near as important as bandwidth. If you can successfully overlap communication with computation in say an MPI application you achieve much better wall clock times for the runs of big jobs.
I have no idea why people are so obsessed with latency in clusters... perhaps the apps are coded with all blocking communication semantics and cannot do the overlap I described above. Basically if you can stream data to the processors and keep them busy
I actually develop a commercial MPI implementation and we have seen real evidence of my claims above. If you are going to SuperComputing 2002 in Baltimore Maryland next week stop by the MPI Software Technology Inc. booth [I won't actually be there... but some of my demos will be]... ask some questions and try to talk to one of our technical guys... they may be better able to explain this since I am not a senior engineer.
[This should be taken not as a representative statement on behalf of the company I work for but as my own humble interpretation of the real issues].
Lots of crossover cables eh? That would indeed make for point to point connections but imagine having eth0-eth7 to manage in your startup scripts... What would /etc/hosts look like? can these nics all get the same IP?
they were practicing safe sex
Am I supposed to give more of a shit now that you have posted this story 3 times?
Sounds like a fundamental design that should be thrown away so you can do what you really want to do. I think most good software engineers see when things are going to "dead-end" and should abandon the losing design in favor of a better one.
:)
Did BSD-Lites have this partition issue? It ran a Unix server on top of Mach too I believe. To force people to run a 64bit architecture to run Hurd seems like a bit of a non-liberating liberation
This shouldn't be a surprise... its the same situation they had in 1998.
Type-o's suck ass don't they? :)