I'd be interested from a large memory point of view, something like a fast swapfile. Another idea would be to use such a device for machines that NFS mount their root directory; use cheap slow ram to act like the disk drive they don't have, in sizes that the memory bus can't address, and which generate little heat and *no* noise. Coupled with an air-cooled cpu and cool graphics card, this *sounds* really nice.
It is very nice to have a reasonable discourse here on/.! A pleasant surprise which reminds me of the days when my user id was about average.
I don't have a problem with you, your writing style, presentation, or general viewpoint. My main quarrel with your post was about your comment on it being "clear" that the other fellow wasn't American. I'm beginning to develop a chip on my shoulder about "American unity", which increasingly looks more like political steamrolling and public brainwashing. So I'm a bit sensitive about defining what it is to be American. This is also why I was stressing being aware of biases, especially in processing what we see in the media.
None of what I've written was meant to be a personal attack (not to mention I don't know you;-). By the way, where I've used the word "funny" in previous posts, I should have said "ironic". For what it is worth, I too am impressed with the writings and works of Jefferson, Madison, Franklin, et al.. They created the most pragmatic and robust Constitution I'm aware of.
Something interesting I learned about the Cuban Missle crisis: it was ended by negotiation, not "America standing firm." We agreed to pull our missles out of Turkey in exchange for the Soviets pulling back from Cuba. These negotiation were done secretly, and America got to propound the myth that by standing firm they prevailed. I've heard GWB use this myth in his speeches about terrorism.
My opinions include:
1) this whole situation sucks, and there aren't any good options
2) military force is not wholly unreasonable, and thank goodness Colin Powell is available as a moderate (I never thought I'd be calling Powell a "dove", but I'm coming closer...)
3) The Taliban are as stupid as our own stubborn leadership, as well as those of Israel and the Palestinians
4) With some luck, we won't create to many more people that really hate the US.
5) I have no idea what to do for the people we've already pissed off, but haven't attacked us (yet). I'm thinking about missle defense, being butts in China, pulling out of negotiations on international treaties that took many years to develop, not to mention the usual complaints about our economic colonizaiton of the world, but who haven't attacked us (yet).
I've been afraid the GWB would start a war ever since he rolled into action (or inaction, depending on the issue) with his foreign policy. Well, here we are. And Americans are still being told bullshit about why we were attacked: "because we're the best and brightest beacon for freedom and opportunity". I believe this is complete crap. I think we were attacked by people boiling over with hate, encouraged by devils with money. These people didn't hate freedom and opportunity, they hated the USA for its actions.
I believe that number 4) is the most important for counteracting terrorist acts against the US. Overall, it doesn't appear that our opinions are too far apart.
It's also funny to see the quote "The price of liberty is eternal vigilance." This makes me think of Israel, not the United States.
Your comment about it being "clear" that the parent post was not American is extremely unjust. I don't agree with you, and I'm American. In fact, I agree more with the parent post than with yours. And next time, before you start quoting dead white Europeans as experts on what it means to be American, consider that they were more influenced by European cultures than by American "culture". If the author of the parent post is *not* American, it is conceiveable that he or she is more culturally and intellectually similar to Franklin and Jefferson than we are.
Finally, your dichotemy "Pick a side, be with the free world and pro-civilization, or be against it" is false. I assert that this is just GW Bush pushing his stupidity on all of America. And just who *is* the free world? Switzerland has a real democracy, but we sure are not a real democracy. Who told the US they were "best and brightest beacon for freedom"? I think it was the US (well, GWB in particular, this time). It is difficult to nail down what it means to be "pro-civilization". In fact, I expect the US has far to few social programs to really be considered "pro-civilization". Think of our pathetic parental leave policies, vacation policies, health policies. Unelss, of course, you wish to define "pro-civilization" as "pro-wealthy-civilization". Allowing companies to give only two weeks of vacation per year is down right inhumane (and hence anti-social and not civilized).
And how do you know that the Taliban is aiding and harboring terrorists? I agree that, given the evidence in Western media, this is a natural conclusion. Your opinion comes down to a matter of trust -- do you trust American Hollywood, English Hollywood, French Hollywood,...? None of us Americans posting here really know anything about who is where doing what in Afghanistan (we'd be in FBI custody if we did;-).
I'm not suggesting that we should avoid having opinions. I'm suggesting we need to be very careful about what we think is true. We need to be very careful to understand what is motivating our decisions and actions. And if we want to impress the rest of the world, it's time we showed more sophistication than is evidenced by the remark "I am assuming this author is not American, and it is clear why."
It's funny to see you quote "Those that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety" when our Congress is outflanking the whole population. They're passing stupid legislation faster than we can become aware of it.
I think what you are pointing out is when the U.S. gets busy in other parts of the world, it pisses some people off. When it *cooperates* with the rest of the world, it doesn't attract to much hatred.
I agree with your assesment that pulling out in response to terrorist acts encourages terrorism. My conclusion, though, is that we should be more careful about "exerting influence" without broad invitation -- with the expected result that we won't be asked, in one form or another, to go home.
On the other hand, I believe Canada has the largest number of peacekeepers throughout the world. This does deserve an admittedly unrelated pat on the back.
As far as the compilers go, the DEC compilers and fancy math libs are available for linux (thanks for everything, Maddog!). Many of the optimizing features of the Tru64 version of these compilers are available on the linux version.
It seems like there is still a small performance delta in my experimental results (on our own Alphas, not on their cluster!), in favor of Tru64. But I can't be sure about this, and the delta wasn't large.
Thus my conclusion is that 1) they liked something about Tru64 besides the compilers, and/or 2) Compaq liked having their name on the OS running the cluster, and gave them a good deal.
Somebody else talked about support from Compaq. I expect the good folks at PSC know nearly as much about these machines as the Compaq engineers do. Furthermore, linux is a supported OS for these machines taken singly, though I didn't see "750-member ES40 cluster" in the support options...;-). I expect that if you buy 750 Alphaservers from Compaq, you won't have to worry about support. =-)
-Paul Komarek
It just occured to me that a P4, with its super-long pipeline, would be even more sensitive to timing and synchronization problems in this sort of setting. How many stages does the P4 have, over 20? That's 20 cycles after receiving the "go!" signal before you get an answer out the business end of the P4.
I'd think this fixed-overhead incurred for each synchronization would have quite an effect for computational loads that required a lot of synchronization. Thus a cpu with a shorter (dare I say "reasonable"?;-) pipeline would have an advantage on such loads. For instance, I'd like to nominate the EV68 Alpha.;-)
It would be great if someone who knew what they were talking about corrected me, since I really don't know what I'm talking about. It would be really great if this person wasn't as lazy as me, and could compare actual pipeline lengths instead of dredging up old and unreliable memories.
-Paul Komarek
I suspect the speed of light problem had more to do with synchronization than with time-in-flight. That is, the difference of time-in-flight between computers is what mattered for maximum performance. For instance, it will take 10 times as long for a signal to travel 10 meters than it will for a signal to travel 1 meter.
I think it takes about 40 nanoseconds for light to travel 1 meter. A 750MHz cpu does one cycle every 4/3 of a nanosecond. In 40 nanoseconds, a 750MHz cpu has gone through 30 cycles. Because these EV68 chips are really beautiful superscalar processors, a lot of instructions are consumed and retired during 30 cycles.
If the time-in-flight between nodes is highly asymmetric, I expect it would be difficult to reasonably schedule work across all the nodes. With some machines 1 meter away, and others 33 meters, the nearest machines could get a signal 1000 cycles before furthest machines did.
Even without all the computation, you can see that the nearest machines could get 33 times more work done than the furthest, when waiting for synchronization signals. What the computations were meant to show was that a 750MHz cpu, and in particular an EV68 Alpha, can do meaningful work during this time.
Please take note that this kind of analysis is not in my field, and furthermore this post is the first time I've thought about it (though I am familiar with these chips, as I administer and use them daily). So please go easy on the flaming corrections!
Unfortunately, they didn't ask our permission. Including share holders like me. They ask me to vote for boardmembers, as if I ever cared. But when they decide to make a massive strategic change, they just do what they feel like. And selling the technology to Intel feels like twisting the knife. Mike Capellas, do you hear me? You JERK! I feel a little better, now, somehow.
At least they've still got the iPAQ. Who knows what its future is, though. Maybe we'll get lucky, and Intel will bail on IA64 and just make Alphas. I can dream, can't I?
At least Transmeta has shown some signs of marketing prowess. DEC and Compaq screwed up marketing the Alphas. At any rate, we've bought 7 alphas. =-)
And I don't think Compaq sold the Alpha line because they couldn't make it work financially -- I think they sold it because they wanted to make the whole company look attractive to HP. Thank goodness we still have the PowerPC from IBM. That will hold for the next 10 years, at which time Intel might finally deliver a good IA64 implementation. We've been waiting, what, 4 years already? -- with no end in sight, either.
I didn't actually catch the model number of the machine, but I believe their using ES-40 (how do you pluralise a model number, without changing the model number?). We have an ES-40 in our lab, and I've spoke with the PSC guys about cheap (i.e. not from Compaq) memory for an ES-40. I got the impression that all of the nodes on their new big machine were ES-40s. But maybe it was a different "new big machine".
Anyway, we run linux on our ES-40 Model II, and it works great. The machine is a dream to administer (I last thought about it several weeks ago =-). However, we're not doing any parallel processing on ours, and aren't likely to anytime soon. We bought it for expandability. The ES-40 Model II will hold 32 1GB PC100 ECC wierd-physical-form-factor dimms, and up to 4 cpu cards. We like to think of the cpu multiplicity as "more machines with no extra administration". The memory is great, too -- we've got folks running 15 GB processes. Maybe the program didn't need to use that much memory; but the astronomers don't have to get fancy with their source code, and hence are saving a lot of time and effort.
You haven't lived (maybe died) until you've waited for a 4GB process to finish dumping core. Thank heavens I haven't been involved in any 15GB "accidents". =-)
I will say that our Tru64 machine (a Microway dual EV67, similar to a Compaq DS-20) has held up surprisingly well under hellish loads: load averages above 5 for a week are not rare, and I was able to recover from a load avg of 40 during an administrative accident. It will be interesting to see if linux does as well for us. However, I've got a lot of gripes about Tru64, and I'd prefer running linux even if we have to ask the users to play nicer.
Some of what you say makes sense, but mostly you sound like an advert. I'll pick on this line:
"When you pay for the cost of commercial UNIX systems, you are paying for the assurance that 1) you aren't going to have stupid design flaws like the one the 2.4 kernel has in its inability to use virtual memory efficiently and 2) All of your nice new custom hardware is going to be supported"
Having administered a Tru64 4.0 and 5.0 box, I can't agree with your statements about "what you pay for" when buying commercial UNIX systems. We had to upgrade because Tru64 4.0E did not support more than 8 SCSI devices on a single chain. Why on earth did we have to pay $1000 to be able to support an old SCSI standard?
We would have moved to linux, except that we have a half-terabyte of ADVFS-formatted data -- i.e. our data is "held hostage" by a proprietary file system format. If all goes well, we'll soon have 700GB of linux-readable space with which we'll rescue our data and then reformat the original array.
Oh, and let's not forget the time (before I was the admin, thank goodness) the machine was crashed by facilities to stop it from relaying spam -- turns out Tru64 ships (or shipped) with an open mail relay. linux has flaws, but at least you get the flaws for free!;-) Oh, then there are the Tru64 network drivers for our old tulip based card . The card doesn't support full duplex, but Tru64 tried to make it support full duplex!
Now there's the reboot cycle it got into, which corrupted the filesystem. However, the disk check ran without errors, and there's nothing unusual in the logs. Tru64 has some great features, none of which we need. We're only using it because we have to. We only paid to upgrade it because we had to.
My experience with optimizing research code for a particular platform: it will never happen, unless you're starting from scratch and expect your code to have a short lifetime. We've got libraries that are several years old, written to be portable across various unix and Windows platforms, running on MIPS, Alpha, x86, SPARC and PA-RISC. These libraries aren't optimized for any particular platform, and nobody has time to mess with platform-specific optimizations.
I've never tried optimizing for SSE, but someone in lab did once. He reported higher performance when doing his computations element-at-a-time than vector-at-a-time. His conclusion, for his particular application, was that memory latency was killing SSE. He was better off doing lots of work on a few numbers, than he was doing fancy stuff to a lot of numbers with SSE. On the other hand, some people have had some luck with SSE optimized FFTs, or so I've heard.
At 2GHz, I'll bet that you're better off doing element computations than vector computations because of the radical difference in memory-versus-processor performance, if the P4's L1 take more than a cycle to feed the registers. Otherwise, do whatever fits in L1 and can be prefeteched -- like elementwise computations on long vectors. Anyone have any real or anecdotal evidence to refute or support this theory?
In the end I think that platform-specific optimizations are a waste of time for research code. I seem to remember some people eventually including hooks in BLAS or LAPACK to allow the user to specify cache sizes; and FFTW does some runtime experiments for optimization. But my guess is that, overall, SSE, 3DNOW, and even AlitVec are irrelevant to most computer researchers. I'll bet their highly relevant to most embedded engineers or many robotics researchs, i.e. people targetting specific hardware.
My experience doesn't suggest that the P4 does twice as much per cycle. I'm seeing P4s do a fair bit less than the P3 per cycle, and the P3, P2, and PPro cores didn't seem *that* much faster per clock than the original Pentiums. My gut tells me that the P4 doesn't do any more than the original Pentiums per clock cycle, and the only thing they have going for them is Intel's ability to manufacture them at high clock speeds.
If you really want a cpu that does a lot in a single cycle, look at the IBM POWER series. IIRC, on the floating point side, a 2xx MHz POWER III is darn not too far from an Alpha 21264 at 733 MHz. And now there are 1.1GHz and 1.@GHz POWER IV chips, in the new IBM p690 machines. I don't know how they compare to the POWER III per cycle, though, because the POWER IV opens a whole new (good) can of worms.
Sharing a computer between multiple users simultanesouly doesn't work well when every program pretends it's God's gift to the computer. This is one reason that linux has some great cost/benefit (shouldn't that be benefit/cost?) ratios over many Windows installations -- it assumes a multiuser, shared environment, in which it is impolite to be inefficient.
No matter which linux kernel they used, it's not a final build either (says someone trying each new 2.4.x linux kernel, hoping to find one living with until 2.6.x).
On the other hand, there's an argument to be made that many people don't need 1.2GHz on their desktop or laptop. When a 400MHz K6-II went down on me, I switched back to an old 200MHz K6 (supposed to be 233MHz, but my crappy Azza motherboard doesn't have the right multipliers).
I was able to run all my normal stuff (I'm a Math/CS grad student) mostly painlessly, including Gnucash and Mozilla. Mozilla was a little slow, but surprisingly it wasn't that bad. I suppose someone running the latest MS Office or MS Dev might have more problems, from what I'm told.
er, the point is that you have to get close to buildings and the ground when landing. To wit I badly quote the authoritative moveie Airplane: "We're going to come in low. But that's something you've gotta do, when you land." =-)
You made just realize something -- I'd much rather have a hydrogen tanker or hydrogen pipeline rupture, than have an oil tanker or oil pipline rupture. Think of the millions, possibly billions, of dollars saved by avoiding the many costly clean up operations we've seen in the last 20 years.
I've been told by a lawyer that many were technically/scientifically inclined in college, but bad in math; hence they studied law. Do you find this is common?
And while it is easy to blame a hatred of math on crappy teachers, it is more difficult assess who to blame for a lack of math ability. Unless you acted above reproach in elementary and high school,... =-) That isn't to say you can't blame your old teachers *also*.
It's really a shame how much people hate math because of crappy teachers. It is just as bad that nobody connects *math* with *thinking* until they've finished a calculus sequence. Math *is* fun, once you leave the numbers out of it and start thinking.
Slashdot Mirror
I'd be interested from a large memory point of view, something like a fast swapfile. Another idea would be to use such a device for machines that NFS mount their root directory; use cheap slow ram to act like the disk drive they don't have, in sizes that the memory bus can't address, and which generate little heat and *no* noise. Coupled with an air-cooled cpu and cool graphics card, this *sounds* really nice.
-Paul Komarek
That was beautiful, from so many angles.
-Paul Komarek
Here here! Nicely put on all accounts.
-Paul Komarek
It is very nice to have a reasonable discourse here on /.! A pleasant surprise which reminds me of the days when my user id was about average.
;-). By the way, where I've used the word "funny" in previous posts, I should have said "ironic". For what it is worth, I too am impressed with the writings and works of Jefferson, Madison, Franklin, et al.. They created the most pragmatic and robust Constitution I'm aware of.
I don't have a problem with you, your writing style, presentation, or general viewpoint. My main quarrel with your post was about your comment on it being "clear" that the other fellow wasn't American. I'm beginning to develop a chip on my shoulder about "American unity", which increasingly looks more like political steamrolling and public brainwashing. So I'm a bit sensitive about defining what it is to be American. This is also why I was stressing being aware of biases, especially in processing what we see in the media.
None of what I've written was meant to be a personal attack (not to mention I don't know you
Something interesting I learned about the Cuban Missle crisis: it was ended by negotiation, not "America standing firm." We agreed to pull our missles out of Turkey in exchange for the Soviets pulling back from Cuba. These negotiation were done secretly, and America got to propound the myth that by standing firm they prevailed. I've heard GWB use this myth in his speeches about terrorism.
My opinions include:
1) this whole situation sucks, and there aren't any good options
2) military force is not wholly unreasonable, and thank goodness Colin Powell is available as a moderate (I never thought I'd be calling Powell a "dove", but I'm coming closer...)
3) The Taliban are as stupid as our own stubborn leadership, as well as those of Israel and the Palestinians
4) With some luck, we won't create to many more people that really hate the US.
5) I have no idea what to do for the people we've already pissed off, but haven't attacked us (yet). I'm thinking about missle defense, being butts in China, pulling out of negotiations on international treaties that took many years to develop, not to mention the usual complaints about our economic colonizaiton of the world, but who haven't attacked us (yet).
I've been afraid the GWB would start a war ever since he rolled into action (or inaction, depending on the issue) with his foreign policy. Well, here we are. And Americans are still being told bullshit about why we were attacked: "because we're the best and brightest beacon for freedom and opportunity". I believe this is complete crap. I think we were attacked by people boiling over with hate, encouraged by devils with money. These people didn't hate freedom and opportunity, they hated the USA for its actions.
I believe that number 4) is the most important for counteracting terrorist acts against the US. Overall, it doesn't appear that our opinions are too far apart.
-Paul Komarek
It's also funny to see the quote "The price of liberty is eternal vigilance." This makes me think of Israel, not the United States.
...? None of us Americans posting here really know anything about who is where doing what in Afghanistan (we'd be in FBI custody if we did ;-).
Your comment about it being "clear" that the parent post was not American is extremely unjust. I don't agree with you, and I'm American. In fact, I agree more with the parent post than with yours. And next time, before you start quoting dead white Europeans as experts on what it means to be American, consider that they were more influenced by European cultures than by American "culture". If the author of the parent post is *not* American, it is conceiveable that he or she is more culturally and intellectually similar to Franklin and Jefferson than we are.
Finally, your dichotemy "Pick a side, be with the free world and pro-civilization, or be against it" is false. I assert that this is just GW Bush pushing his stupidity on all of America. And just who *is* the free world? Switzerland has a real democracy, but we sure are not a real democracy. Who told the US they were "best and brightest beacon for freedom"? I think it was the US (well, GWB in particular, this time). It is difficult to nail down what it means to be "pro-civilization". In fact, I expect the US has far to few social programs to really be considered "pro-civilization". Think of our pathetic parental leave policies, vacation policies, health policies. Unelss, of course, you wish to define "pro-civilization" as "pro-wealthy-civilization". Allowing companies to give only two weeks of vacation per year is down right inhumane (and hence anti-social and not civilized).
And how do you know that the Taliban is aiding and harboring terrorists? I agree that, given the evidence in Western media, this is a natural conclusion. Your opinion comes down to a matter of trust -- do you trust American Hollywood, English Hollywood, French Hollywood,
I'm not suggesting that we should avoid having opinions. I'm suggesting we need to be very careful about what we think is true. We need to be very careful to understand what is motivating our decisions and actions. And if we want to impress the rest of the world, it's time we showed more sophistication than is evidenced by the remark "I am assuming this author is not American, and it is clear why."
-Paul Komarek
It's funny to see you quote "Those that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety" when our Congress is outflanking the whole population. They're passing stupid legislation faster than we can become aware of it.
-Paul Komarek
I think what you are pointing out is when the U.S. gets busy in other parts of the world, it pisses some people off. When it *cooperates* with the rest of the world, it doesn't attract to much hatred.
I agree with your assesment that pulling out in response to terrorist acts encourages terrorism. My conclusion, though, is that we should be more careful about "exerting influence" without broad invitation -- with the expected result that we won't be asked, in one form or another, to go home.
-Paul Komarek
On the other hand, I believe Canada has the largest number of peacekeepers throughout the world. This does deserve an admittedly unrelated pat on the back.
-Paul Komarek
As far as the compilers go, the DEC compilers and fancy math libs are available for linux (thanks for everything, Maddog!). Many of the optimizing features of the Tru64 version of these compilers are available on the linux version.
;-). I expect that if you buy 750 Alphaservers from Compaq, you won't have to worry about support. =-)
It seems like there is still a small performance delta in my experimental results (on our own Alphas, not on their cluster!), in favor of Tru64. But I can't be sure about this, and the delta wasn't large.
Thus my conclusion is that 1) they liked something about Tru64 besides the compilers, and/or 2) Compaq liked having their name on the OS running the cluster, and gave them a good deal.
Somebody else talked about support from Compaq. I expect the good folks at PSC know nearly as much about these machines as the Compaq engineers do. Furthermore, linux is a supported OS for these machines taken singly, though I didn't see "750-member ES40 cluster" in the support options...
-Paul Komarek
It just occured to me that a P4, with its super-long pipeline, would be even more sensitive to timing and synchronization problems in this sort of setting. How many stages does the P4 have, over 20? That's 20 cycles after receiving the "go!" signal before you get an answer out the business end of the P4.
;-) pipeline would have an advantage on such loads. For instance, I'd like to nominate the EV68 Alpha. ;-)
I'd think this fixed-overhead incurred for each synchronization would have quite an effect for computational loads that required a lot of synchronization. Thus a cpu with a shorter (dare I say "reasonable"?
It would be great if someone who knew what they were talking about corrected me, since I really don't know what I'm talking about. It would be really great if this person wasn't as lazy as me, and could compare actual pipeline lengths instead of dredging up old and unreliable memories.
-Paul Komarek
I suspect the speed of light problem had more to do with synchronization than with time-in-flight. That is, the difference of time-in-flight between computers is what mattered for maximum performance. For instance, it will take 10 times as long for a signal to travel 10 meters than it will for a signal to travel 1 meter.
I think it takes about 40 nanoseconds for light to travel 1 meter. A 750MHz cpu does one cycle every 4/3 of a nanosecond. In 40 nanoseconds, a 750MHz cpu has gone through 30 cycles. Because these EV68 chips are really beautiful superscalar processors, a lot of instructions are consumed and retired during 30 cycles.
If the time-in-flight between nodes is highly asymmetric, I expect it would be difficult to reasonably schedule work across all the nodes. With some machines 1 meter away, and others 33 meters, the nearest machines could get a signal 1000 cycles before furthest machines did.
Even without all the computation, you can see that the nearest machines could get 33 times more work done than the furthest, when waiting for synchronization signals. What the computations were meant to show was that a 750MHz cpu, and in particular an EV68 Alpha, can do meaningful work during this time.
Please take note that this kind of analysis is not in my field, and furthermore this post is the first time I've thought about it (though I am familiar with these chips, as I administer and use them daily). So please go easy on the flaming corrections!
-Paul Komarek
Unfortunately, they didn't ask our permission. Including share holders like me. They ask me to vote for boardmembers, as if I ever cared. But when they decide to make a massive strategic change, they just do what they feel like. And selling the technology to Intel feels like twisting the knife. Mike Capellas, do you hear me? You JERK! I feel a little better, now, somehow.
At least they've still got the iPAQ. Who knows what its future is, though. Maybe we'll get lucky, and Intel will bail on IA64 and just make Alphas. I can dream, can't I?
-Paul Komarek
At least Transmeta has shown some signs of marketing prowess. DEC and Compaq screwed up marketing the Alphas. At any rate, we've bought 7 alphas. =-)
And I don't think Compaq sold the Alpha line because they couldn't make it work financially -- I think they sold it because they wanted to make the whole company look attractive to HP. Thank goodness we still have the PowerPC from IBM. That will hold for the next 10 years, at which time Intel might finally deliver a good IA64 implementation. We've been waiting, what, 4 years already? -- with no end in sight, either.
-Paul Komarek
I didn't actually catch the model number of the machine, but I believe their using ES-40 (how do you pluralise a model number, without changing the model number?). We have an ES-40 in our lab, and I've spoke with the PSC guys about cheap (i.e. not from Compaq) memory for an ES-40. I got the impression that all of the nodes on their new big machine were ES-40s. But maybe it was a different "new big machine".
Anyway, we run linux on our ES-40 Model II, and it works great. The machine is a dream to administer (I last thought about it several weeks ago =-). However, we're not doing any parallel processing on ours, and aren't likely to anytime soon. We bought it for expandability. The ES-40 Model II will hold 32 1GB PC100 ECC wierd-physical-form-factor dimms, and up to 4 cpu cards. We like to think of the cpu multiplicity as "more machines with no extra administration". The memory is great, too -- we've got folks running 15 GB processes. Maybe the program didn't need to use that much memory; but the astronomers don't have to get fancy with their source code, and hence are saving a lot of time and effort.
You haven't lived (maybe died) until you've waited for a 4GB process to finish dumping core. Thank heavens I haven't been involved in any 15GB "accidents". =-)
I will say that our Tru64 machine (a Microway dual EV67, similar to a Compaq DS-20) has held up surprisingly well under hellish loads: load averages above 5 for a week are not rare, and I was able to recover from a load avg of 40 during an administrative accident. It will be interesting to see if linux does as well for us. However, I've got a lot of gripes about Tru64, and I'd prefer running linux even if we have to ask the users to play nicer.
-Paul Komarek
Some of what you say makes sense, but mostly you sound like an advert. I'll pick on this line:
;-) Oh, then there are the Tru64 network drivers for our old tulip based card . The card doesn't support full duplex, but Tru64 tried to make it support full duplex!
"When you pay for the cost of commercial UNIX systems, you are paying for the assurance that 1) you aren't going to have stupid design flaws like the one the 2.4 kernel has in its inability to use virtual memory efficiently and 2) All of your nice new custom hardware is going to be supported"
Having administered a Tru64 4.0 and 5.0 box, I can't agree with your statements about "what you pay for" when buying commercial UNIX systems. We had to upgrade because Tru64 4.0E did not support more than 8 SCSI devices on a single chain. Why on earth did we have to pay $1000 to be able to support an old SCSI standard?
We would have moved to linux, except that we have a half-terabyte of ADVFS-formatted data -- i.e. our data is "held hostage" by a proprietary file system format. If all goes well, we'll soon have 700GB of linux-readable space with which we'll rescue our data and then reformat the original array.
Oh, and let's not forget the time (before I was the admin, thank goodness) the machine was crashed by facilities to stop it from relaying spam -- turns out Tru64 ships (or shipped) with an open mail relay. linux has flaws, but at least you get the flaws for free!
Now there's the reboot cycle it got into, which corrupted the filesystem. However, the disk check ran without errors, and there's nothing unusual in the logs. Tru64 has some great features, none of which we need. We're only using it because we have to. We only paid to upgrade it because we had to.
-Paul Komarek
My experience with optimizing research code for a particular platform: it will never happen, unless you're starting from scratch and expect your code to have a short lifetime. We've got libraries that are several years old, written to be portable across various unix and Windows platforms, running on MIPS, Alpha, x86, SPARC and PA-RISC. These libraries aren't optimized for any particular platform, and nobody has time to mess with platform-specific optimizations.
I've never tried optimizing for SSE, but someone in lab did once. He reported higher performance when doing his computations element-at-a-time than vector-at-a-time. His conclusion, for his particular application, was that memory latency was killing SSE. He was better off doing lots of work on a few numbers, than he was doing fancy stuff to a lot of numbers with SSE. On the other hand, some people have had some luck with SSE optimized FFTs, or so I've heard.
At 2GHz, I'll bet that you're better off doing element computations than vector computations because of the radical difference in memory-versus-processor performance, if the P4's L1 take more than a cycle to feed the registers. Otherwise, do whatever fits in L1 and can be prefeteched -- like elementwise computations on long vectors. Anyone have any real or anecdotal evidence to refute or support this theory?
In the end I think that platform-specific optimizations are a waste of time for research code. I seem to remember some people eventually including hooks in BLAS or LAPACK to allow the user to specify cache sizes; and FFTW does some runtime experiments for optimization. But my guess is that, overall, SSE, 3DNOW, and even AlitVec are irrelevant to most computer researchers. I'll bet their highly relevant to most embedded engineers or many robotics researchs, i.e. people targetting specific hardware.
-Paul Komarek
My experience doesn't suggest that the P4 does twice as much per cycle. I'm seeing P4s do a fair bit less than the P3 per cycle, and the P3, P2, and PPro cores didn't seem *that* much faster per clock than the original Pentiums. My gut tells me that the P4 doesn't do any more than the original Pentiums per clock cycle, and the only thing they have going for them is Intel's ability to manufacture them at high clock speeds.
If you really want a cpu that does a lot in a single cycle, look at the IBM POWER series. IIRC, on the floating point side, a 2xx MHz POWER III is darn not too far from an Alpha 21264 at 733 MHz. And now there are 1.1GHz and 1.@GHz POWER IV chips, in the new IBM p690 machines. I don't know how they compare to the POWER III per cycle, though, because the POWER IV opens a whole new (good) can of worms.
-Paul Komarek
Sharing a computer between multiple users simultanesouly doesn't work well when every program pretends it's God's gift to the computer. This is one reason that linux has some great cost/benefit (shouldn't that be benefit/cost?) ratios over many Windows installations -- it assumes a multiuser, shared environment, in which it is impolite to be inefficient.
-Paul Komarek
No matter which linux kernel they used, it's not a final build either (says someone trying each new 2.4.x linux kernel, hoping to find one living with until 2.6.x).
-Paul Komarek
On the other hand, there's an argument to be made that many people don't need 1.2GHz on their desktop or laptop. When a 400MHz K6-II went down on me, I switched back to an old 200MHz K6 (supposed to be 233MHz, but my crappy Azza motherboard doesn't have the right multipliers).
I was able to run all my normal stuff (I'm a Math/CS grad student) mostly painlessly, including Gnucash and Mozilla. Mozilla was a little slow, but surprisingly it wasn't that bad. I suppose someone running the latest MS Office or MS Dev might have more problems, from what I'm told.
-Paul Komarek
er, the point is that you have to get close to buildings and the ground when landing. To wit I badly quote the authoritative moveie Airplane: "We're going to come in low. But that's something you've gotta do, when you land." =-)
-Paul Komarek
You made just realize something -- I'd much rather have a hydrogen tanker or hydrogen pipeline rupture, than have an oil tanker or oil pipline rupture. Think of the millions, possibly billions, of dollars saved by avoiding the many costly clean up operations we've seen in the last 20 years.
-Paul Komarek
I've been told by a lawyer that many were technically/scientifically inclined in college, but bad in math; hence they studied law. Do you find this is common?
... =-) That isn't to say you can't blame your old teachers *also*.
And while it is easy to blame a hatred of math on crappy teachers, it is more difficult assess who to blame for a lack of math ability. Unless you acted above reproach in elementary and high school,
It's really a shame how much people hate math because of crappy teachers. It is just as bad that nobody connects *math* with *thinking* until they've finished a calculus sequence. Math *is* fun, once you leave the numbers out of it and start thinking.
-Paul Komarek
How does the Portland Group compilers fare in the current compiler market?
-Paul Komarek
It's called "Mathematics" (I'm a math grad ;-). Unfortunately, math is so botched before college it's a wonder anyone studies it at all.
-Paul Komarek