They're not really scared of clustering for reliability reasons. You can do big machines that are far more reliable than the cluster made of cheap components. In fact, it's being done all the time.
First of all, the Google setup isn't the type of cluster that is being compared to Cray supercomputers, and it's also a simple form, with little communication between nodes. Secondly, in a HPC cluster, the reliability of each node becomes far more important, since a failure for a node can mean hours or even days of having to performm calculations again. If you're unlucky, and you have a project where each node needs a lot of data from the other nodes, and a node goes down, you've stalled the entire project for quite a while. If a CPU dies in a supercomputer, process state etc can be migrated over to other CPU's.
I've been using Blender to and from since 1996, when it was only available for Irix. Even looking at this latest release, it's not that much that has changed UI wise, it's still an enormous kludge. It always ends up with me giving up and going back to Maya so I can get things done, with a proper 1 hand on mouse, 1 hand on keyboard UI that works superb.
No, they are buying them for use with tasks that are well-suited for clusters, or because they have no alternative. As a footnote: NSC in Linköping has a 200-node cluster, each node dual 2.2GHz Xeons, 2GB RAM, and SCI interconnect. They also h ave a SGI 3800 with 128 CPU's and 128GB RAM.
For some of the stuff I do, such as building airflow simulations, heat distribution etc, the 3800 outperformed the cluster, despite the CPU's being so much weaker. Simply because of the Shared Memory architecture.
Actually, Infiniband, Quadrics, Myrinet etc are all down to 15-60 _micro_seconds for useful payload sizes, not milliseconds. Gigabit and 10GbE are far slower though, and you'd have to write a software stack for handling remote DMA(Which Infiniband handles in hardware). And it's still not even near achieving what a true shared memory system can achieve.
Likelihood of it costing only 1/3? Pretty small. Good interconnect fabric is costly, as are the nodes themselves if you intend to do tasks that requires lots of communication, since if a node fails, it will cause stalls etc as the part it was working on needs to be computed again due to that data being needed for the computation to go on.
Realistically, for the tasks we're talking about, you might get 1/5th-1/4th of the performance, for maybe 50-60% of the price.
There's one thing you touch upon in passing: The protocol stack. Many of the problems in speed when it comes to clusters is that people still use TCP(And it doesn't help that a bunch of idiots are trying to get people to use TCP/IP even at subnet level with Infiniband... Talk about crippling Infiniband by doing that..), with alll the performance hits that entails.
The problem with communications-heavy tasks is that you easily saturate a cluster, even if you use Myrinet, Quadrics or Infiniband. You also hit a point of diminishing returns.
That cluster with twice as many CPU's might just perform at 1/3(And that's optimistic) of the speed that the supercomputer does the job at when it comes to such tasks.
One, Altivec only supports single-precision floats. Big no-no in quite a few scientific areas. Also, to be blunt: GCC sucks. If you want something decent, at least use IBM's compiler.
Cheaper? Not likely, you'd have to buy the high-speed interconnect to make it worthwhile. And the Opterons perform fairly poorly in larger clusters, since they have the NUMA latency penalties locally on each node. Checking the Top500 list, a cluster of 256 Opteron 246 using Infiniband will perform worse than a cluster of 256 Xeon 2.8GHz using Infiniband. The scariest example is that a cluster of 256 P4's@3GHz using Gigabit Ethernet outperforms the Opteron cluster.....
Important to note is that the Linpack test doesn't stress the interconnect that much. The more a task stresses the interconnect, the more the Opteron cluster will be penalized. There's one exception though, and that's the Cray Octiga Bay systems.... And if you go that route, it costs _at_ _least_ as much as an Altix system
But, as I mention in a post of mine, the price you've put as rough average for Sweden is way off. If I had to put an average, I'd put it closer to around $4.5-$5
A: Way too old. It hasn't cost that much on average for several years now.
B: Only sampled from some particular hotels etc, that aren't internet cafés as such.
C: Made up.
Matrix, a large internet café in Stockholm charges around $4/hour for non-members. They also have some packages where it becomes cheaper. Dragons Lair charges around $2/hour. Nexuz about 2-4, depending on time etc.
I've currently got ADSL2+, 24Mb/s down and 1Mb/s up. 4 persons are sharing the connection, and even though we've only had it for about 3 weeks, we've already saturated it a number of times. The fastest single download I've clocked so far was 22Mb/s(Downloading a Gentoo ISO from the Sunet servers).
Other things we've noticed is that we are having less troubles now with things such as voice-chatting via Skype, downloading while playing games, streaming video(BBC and National Geographics via PC-TV), streaming audio(some radio stations also run their broadcasts via the net. Works great, since radio reception sucks in this apartment) and similar things.
Blender has a few features, but the renderer is still a bit behind quality-wise, even though it's fast. The reason I don't use it seriously(and why I've never stuck with it, despite trying it out to and from since 1996) is because it's such a kludge to work with. And, now that it's Open Source, I expect it to develop more in the direction of "Oh, shiny!" cool technologies, while the UI will still remain annoying.
700MB in 12 minutes? Yeah, sounds about normal for a 10-12Mb/s connection. The fastest speed I've reached so far with my ADSL is 2 700MB movies in around 16 minutes each, both downloading simultaneously, together with some smaller stuff.
With Maya you choose if you want to work with triangles or with mathematical curves. As for efficiency, I've used PoV-Ray, and I still consider it mostly a toy/hobby. If I need something done on a deadline, I use the commercial apps, because they are far more efficient where it really counts.
Obviously you've never used Maya. In Maya, you can do _exactly_ what you can do in PoV-Ray when it comes to modelling. That's why MEL is there. And, even when you work with a program such as Maya, you need storyboarding, planning etc to get an idea of what the final result should be. What Maya does is that it gives you a superb user interface in addition to the coding possibilities.
First of all, read the subject line. I said Sweden, not Japan. Sweden also has a fairly low population density. The difference is that the state-owned company Skanova(An offshoot of Telia, former Televerket) has a mandate on them to make ADSL available in almost the entire country. Sweden is slightly larger than the state of California in the USA, with a much smaller population, and broadband is highly available here in Sweden anyway.
Second, my ISP right now offers 3 channels webcasted, with decent resolution. They also have an agreement with SF, a large movie distributor, for online movie rentals. They also offer VoIP.
A third point you missed is that maybe you're sharing the connection inside the family. Having 5 computers simultaneously using the connection, downloading, playing games, watching TV etc, you need a higher bandwidth. As for the 50:1 contention, I've not exactly seen most Swedish providers anywhere near that. Otoh, we do have stronger consumer protection laws in some respects, as well as stronger consumer demands.
So far, the only problem I've had with this connection was that I only got around 10-11Mb/s in the beginning. The reason was that the telecom grid in the Stockholm area is almost entirely new, while BBB's stuff was calibrated for older grids, which needed higher signal strength, thus the S/N ratio was increased. Took a total of 1½ day to fix that.
As an aside, in the Greater Stockholm metropolitan area, the telecom grid is now entirely packet-switched ATM.
When I see US people talk about the cost of their broadband, I'm always shocked. Sure, we have some providers that are sub-par, but right now, I've got Bredbandsbolaget ADSL2+, offering up to 24Mb/s downstream and 1Mb/s upstream(I've reached around 22Mb/s down at most right now), no bandwidth cap, and I can host a non-profit, non-commercial server, and it costs around 45-48/month.
And lots of US people like to cry about Pearl Harbour(Which, in light of evidence such as letters and US military archives, wasn't a surprise attack) and the World Trade Center, conveniently neglecting Hiroshima, Nagasaki, the camps lots of asian people were put in during WW2, the genocide campaigns against the native americans etc etc.
Slashdot Mirror
They're not really scared of clustering for reliability reasons. You can do big machines that are far more reliable than the cluster made of cheap components. In fact, it's being done all the time.
First of all, the Google setup isn't the type of cluster that is being compared to Cray supercomputers, and it's also a simple form, with little communication between nodes. Secondly, in a HPC cluster, the reliability of each node becomes far more important, since a failure for a node can mean hours or even days of having to performm calculations again. If you're unlucky, and you have a project where each node needs a lot of data from the other nodes, and a node goes down, you've stalled the entire project for quite a while. If a CPU dies in a supercomputer, process state etc can be migrated over to other CPU's.
I've been using Blender to and from since 1996, when it was only available for Irix. Even looking at this latest release, it's not that much that has changed UI wise, it's still an enormous kludge. It always ends up with me giving up and going back to Maya so I can get things done, with a proper 1 hand on mouse, 1 hand on keyboard UI that works superb.
The GeForce 6800 series can actually handle 64-bit floating point calculations.
No, they are buying them for use with tasks that are well-suited for clusters, or because they have no alternative. As a footnote: NSC in Linköping has a 200-node cluster, each node dual 2.2GHz Xeons, 2GB RAM, and SCI interconnect. They also h ave a SGI 3800 with 128 CPU's and 128GB RAM.
For some of the stuff I do, such as building airflow simulations, heat distribution etc, the 3800 outperformed the cluster, despite the CPU's being so much weaker. Simply because of the Shared Memory architecture.
Actually, Infiniband, Quadrics, Myrinet etc are all down to 15-60 _micro_seconds for useful payload sizes, not milliseconds. Gigabit and 10GbE are far slower though, and you'd have to write a software stack for handling remote DMA(Which Infiniband handles in hardware). And it's still not even near achieving what a true shared memory system can achieve.
Likelihood of it costing only 1/3? Pretty small. Good interconnect fabric is costly, as are the nodes themselves if you intend to do tasks that requires lots of communication, since if a node fails, it will cause stalls etc as the part it was working on needs to be computed again due to that data being needed for the computation to go on.
Realistically, for the tasks we're talking about, you might get 1/5th-1/4th of the performance, for maybe 50-60% of the price.
There's one thing you touch upon in passing:
The protocol stack. Many of the problems in speed when it comes to clusters is that people still use TCP(And it doesn't help that a bunch of idiots are trying to get people to use TCP/IP even at subnet level with Infiniband... Talk about crippling Infiniband by doing that..), with alll the performance hits that entails.
The problem with communications-heavy tasks is that you easily saturate a cluster, even if you use Myrinet, Quadrics or Infiniband. You also hit a point of diminishing returns.
That cluster with twice as many CPU's might just perform at 1/3(And that's optimistic) of the speed that the supercomputer does the job at when it comes to such tasks.
One, Altivec only supports single-precision floats. Big no-no in quite a few scientific areas. Also, to be blunt: GCC sucks. If you want something decent, at least use IBM's compiler.
Agent Green:
Cheaper? Not likely, you'd have to buy the high-speed interconnect to make it worthwhile. And the Opterons perform fairly poorly in larger clusters, since they have the NUMA latency penalties locally on each node. Checking the Top500 list, a cluster of 256 Opteron 246 using Infiniband will perform worse than a cluster of 256 Xeon 2.8GHz using Infiniband. The scariest example is that a cluster of 256 P4's@3GHz using Gigabit Ethernet outperforms the Opteron cluster.....
Important to note is that the Linpack test doesn't stress the interconnect that much. The more a task stresses the interconnect, the more the Opteron cluster will be penalized. There's one exception though, and that's the Cray Octiga Bay systems.... And if you go that route, it costs _at_ _least_ as much as an Altix system
But, as I mention in a post of mine, the price you've put as rough average for Sweden is way off. If I had to put an average, I'd put it closer to around $4.5-$5
They are one of the following:
A: Way too old. It hasn't cost that much on average for several years now.
B: Only sampled from some particular hotels etc, that aren't internet cafés as such.
C: Made up.
Matrix, a large internet café in Stockholm charges around $4/hour for non-members. They also have some packages where it becomes cheaper. Dragons Lair charges around $2/hour. Nexuz about 2-4, depending on time etc.
Same thing with other places around Sweden.
I've currently got ADSL2+, 24Mb/s down and 1Mb/s up. 4 persons are sharing the connection, and even though we've only had it for about 3 weeks, we've already saturated it a number of times. The fastest single download I've clocked so far was 22Mb/s(Downloading a Gentoo ISO from the Sunet servers).
Other things we've noticed is that we are having less troubles now with things such as voice-chatting via Skype, downloading while playing games, streaming video(BBC and National Geographics via PC-TV), streaming audio(some radio stations also run their broadcasts via the net. Works great, since radio reception sucks in this apartment) and similar things.
The original ADSL specification handles speeds of up to 8Mb/s downstream and 800kB/s.
Blender has a few features, but the renderer is still a bit behind quality-wise, even though it's fast. The reason I don't use it seriously(and why I've never stuck with it, despite trying it out to and from since 1996) is because it's such a kludge to work with. And, now that it's Open Source, I expect it to develop more in the direction of "Oh, shiny!" cool technologies, while the UI will still remain annoying.
700MB in 12 minutes? Yeah, sounds about normal for a 10-12Mb/s connection. The fastest speed I've reached so far with my ADSL is 2 700MB movies in around 16 minutes each, both downloading simultaneously, together with some smaller stuff.
With Maya you choose if you want to work with triangles or with mathematical curves. As for efficiency, I've used PoV-Ray, and I still consider it mostly a toy/hobby. If I need something done on a deadline, I use the commercial apps, because they are far more efficient where it really counts.
Obviously you've never used Maya. In Maya, you can do _exactly_ what you can do in PoV-Ray when it comes to modelling. That's why MEL is there. And, even when you work with a program such as Maya, you need storyboarding, planning etc to get an idea of what the final result should be. What Maya does is that it gives you a superb user interface in addition to the coding possibilities.
Actually, in Sweden 15 years is the age of consent, period. As long as both partners are over 15 and consenting, it's ok.
As I stated in the subject line: Sweden
First of all, read the subject line. I said Sweden, not Japan. Sweden also has a fairly low population density. The difference is that the state-owned company Skanova(An offshoot of Telia, former Televerket) has a mandate on them to make ADSL available in almost the entire country. Sweden is slightly larger than the state of California in the USA, with a much smaller population, and broadband is highly available here in Sweden anyway.
Second, my ISP right now offers 3 channels webcasted, with decent resolution. They also have an agreement with SF, a large movie distributor, for online movie rentals. They also offer VoIP.
A third point you missed is that maybe you're sharing the connection inside the family. Having 5 computers simultaneously using the connection, downloading, playing games, watching TV etc, you need a higher bandwidth. As for the 50:1 contention, I've not exactly seen most Swedish providers anywhere near that. Otoh, we do have stronger consumer protection laws in some respects, as well as stronger consumer demands.
So far, the only problem I've had with this connection was that I only got around 10-11Mb/s in the beginning. The reason was that the telecom grid in the Stockholm area is almost entirely new, while BBB's stuff was calibrated for older grids, which needed higher signal strength, thus the S/N ratio was increased. Took a total of 1½ day to fix that.
As an aside, in the Greater Stockholm metropolitan area, the telecom grid is now entirely packet-switched ATM.
When I see US people talk about the cost of their broadband, I'm always shocked. Sure, we have some providers that are sub-par, but right now, I've got Bredbandsbolaget ADSL2+, offering up to 24Mb/s downstream and 1Mb/s upstream(I've reached around 22Mb/s down at most right now), no bandwidth cap, and I can host a non-profit, non-commercial server, and it costs around 45-48/month.
And lots of US people like to cry about Pearl Harbour(Which, in light of evidence such as letters and US military archives, wasn't a surprise attack) and the World Trade Center, conveniently neglecting Hiroshima, Nagasaki, the camps lots of asian people were put in during WW2, the genocide campaigns against the native americans etc etc.
The larger the system is, the more it matters.
Wrong. It's a very real distinction. The more nodes you have, the more important it becomes to pay attention to it.