Slashdot Mirror

"the machine may be ineligible because of its specialized hardware"
What specialized hardware? I would really like to read a more technical article about this machine. I would guess that the Japanese focused on vector processing like they did in the design of the Earth-Simulator.

The best supporting evidence I have for this conclusion is the comparison of Japan's last two supercomputers:
Sun Fire X64 Cluster
Earth-Simulator

Sun Fire has 10,368 processors with a Rmax(GFlops) of 38,180.
Earth-Simulator has 5,120 processors with a Rmax(GFlops) of 35,860.
That's 49% less processors with 94% the processor power*.

Here's the original article link:
http://www.businessweek.com/globalbiz/content/jul2 006/gb20060726_150659.htm?chan=topStories_ssi_5/

*Only comparing one aspect of performance.

"the machine may be ineligible because of its specialized hardware"
What specialized hardware? I would really like to read a more technical article about this machine. I would guess that the Japanese focused on vector processing like they did in the design of the Earth-Simulator.

The best supporting evidence I have for this conclusion is the comparison of Japan's last two supercomputers:
Sun Fire X64 Cluster
Earth-Simulator

Sun Fire has 10,368 processors with a Rmax(GFlops) of 38,180.
Earth-Simulator has 5,120 processors with a Rmax(GFlops) of 35,860.
That's 49% less processors with 94% the processor power*.

Here's the original article link:
http://www.businessweek.com/globalbiz/content/jul2 006/gb20060726_150659.htm?chan=topStories_ssi_5/

*Only comparing one aspect of performance.

I know its apples (not literally) and oranges but IBM's Power5 processor already has quad core and will have an 8-core sometime next year(Power6).

The p5-550Q has 2 chips, quad core each for 8 cores with each core running at 1.65Ghz (PPC RISC) 64KB of L1, 8MB of L2(shared) and 144MB of L3(shared).The biggest Power5 based system I've seen is the p5-595 with 64 processors, each running 2.3Ghz with 64KB of L1, 64MB of L2(shared) and 1152MB of L3(shared). Keep in mind, this is 1 server.

All the IBM BlueGene systems are running the Power5 processors and they are most of the http://www.top500.org/lists/2006/06

All I have to say to Intel (and AMD for that matter) is, if you build it, they will buy it (unless it's called the Itanium)

Both Intel's x86[-64] offerings and AMD's x86[-64] offerings beat itanic in TDP, floating point performance per watt, FP perf per dollar

It really depends on your problem. If you have a code that is incredibly communications-intensive, you're likely to use a very large amount of your interrupts sending and receiving messages (if you're using MPI) rather than doing computations. There are codes out there that are essentially useless over GigE (there is no performance gain realized when increasing your used resources; it does not scale) and the very _minimum_ becomes a high-end interconnect like Myrinet; therefore, your cost of implementation argument does not hold true in a number of cases. In the case of Linpack/HPL, which is used for the Top500 benchmark, http://www.top500.org/about/linpack, the algorithm works in such a manner that the limitations of GigE are not as much a problem. Linpack is simply not representative of all of the different codes out there and in many cases, the results it provides may not provide any indication whatsoever of how well your code will run on a particular computer.

Top 500 has lists going back to 1993; back then the fastest supercomputer listed showed 60 Gigaflops. This is probably a little faster, but on the same order of magnitude, of a high-end desktop today (I am just estimating, as I cannot find general linpack scores in under 3 minutes). So perhaps give or take a 15-year span; that is, an average computer in the present is as fast as the fastest computer in the world of 15 years back.

The change in installation date indicates that the system was upgraded to include more processors and memory since the last time the system was on the list. See similar changes to installation date for the number 1 BlueGene/L here, here and here. Compare Purple from November 2005 to the system from June 2006. Why they don't have them linked together as the same system is beyond me.

The change in installation date indicates that the system was upgraded to include more processors and memory since the last time the system was on the list. See similar changes to installation date for the number 1 BlueGene/L here, here and here. Compare Purple from November 2005 to the system from June 2006. Why they don't have them linked together as the same system is beyond me.

The change in installation date indicates that the system was upgraded to include more processors and memory since the last time the system was on the list. See similar changes to installation date for the number 1 BlueGene/L here, here and here. Compare Purple from November 2005 to the system from June 2006. Why they don't have them linked together as the same system is beyond me.

The change in installation date indicates that the system was upgraded to include more processors and memory since the last time the system was on the list. See similar changes to installation date for the number 1 BlueGene/L here, here and here. Compare Purple from November 2005 to the system from June 2006. Why they don't have them linked together as the same system is beyond me.

The change in installation date indicates that the system was upgraded to include more processors and memory since the last time the system was on the list. See similar changes to installation date for the number 1 BlueGene/L here, here and here. Compare Purple from November 2005 to the system from June 2006. Why they don't have them linked together as the same system is beyond me.

Real supercomputers do not use any type of networking between nodes, they have a shared memory bus.
Don't say that, all it does is show how little you know.

There are 2 common types of interfaces between nodes on a supercomputer: shared memory and message passing.

Shared memory is where all the nodes can access memory over some sort of network. In order for communication to happen all 2 nodes need to do is read and write to the same location in memory. There is little talk about the network protocol used at this level because for the most part it is an emulation of layer 2 of the OSI (as if all you are doing is ordering the hardware around).

Message passing would best be described in terms of layer 7. Communication occurs between 2 nodes via messages that are sent back and forth (hence the name). The most common message passing scheme is MPI. In MPI, there is a concept of a sender and a reciever. The reciever calls MPI_Recv and the sender calls MPI_Send and a message is sent from send to recv. You could almost think of this as an HTTP communication; the server is listening, the client sends information, the server sends back, except in MPI the reciever must be calling MPI_Recv and waiting for a send from a specific sender and the sender must call MPI_Send to send the information to the reciever (there really [well, sorta] isn't a concept of a timeout). In my experience, this makes MPI (I use MPICH2) difficult to debug, if A calls send to B and B calls send to A at the same time, your program blows up (often with very little useful information).

On the cluster I do my work on, the implementation of MPI sends TCP/IP packets over ethernet (much like 256 of this top500 list). The libraries could be written to do the work over Myrinet or any other network.

For future reference please learn some factual information before you go spouting bull. If you follow this link, and choose interconnect family, you would find that most of the supercomputers in the top500 list are using some standard network interconnect.

Indeed, straight from the horses, uhh, website:

Gflop/s is a rate of execution, billions of floating point operations per second. (Emphasis mine)

Also see #21 COLSA uses a cluster of Apple Xserves http://www.top500.org/system/7741

Just to follow up, you can get OS information here: http://www.top500.org/stats/27/osfam/ (by family)

OS (# systems) (Percent)
Linux 367 73.40%
Windows 2 0.40%
Unix 98 19.60%
BSD 4 0.80%
Mixed 24 4.80%
Mac OS 5 1.00%
Totals 500 100%

Alternately there's a more refined breakdown listing them by Operating System type and version. Oddly, "Linux" is listed both as an operating system family and as a distinct flavor/distro ... I can only assume that the systems using "Linux" as the particular operating system are using a custom-made distro, instead of one of the commercial ones (which are listed separately on the detailed chart). Unless they just failed to report one in particular.

As for the Windows-based systems, there were one each for Windows 2003 Server and Windows Compute Cluster Server 2003.

Just to follow up, you can get OS information here: http://www.top500.org/stats/27/osfam/ (by family)

OS (# systems) (Percent)
Linux 367 73.40%
Windows 2 0.40%
Unix 98 19.60%
BSD 4 0.80%
Mixed 24 4.80%
Mac OS 5 1.00%
Totals 500 100%

Alternately there's a more refined breakdown listing them by Operating System type and version. Oddly, "Linux" is listed both as an operating system family and as a distinct flavor/distro ... I can only assume that the systems using "Linux" as the particular operating system are using a custom-made distro, instead of one of the commercial ones (which are listed separately on the detailed chart). Unless they just failed to report one in particular.

As for the Windows-based systems, there were one each for Windows 2003 Server and Windows Compute Cluster Server 2003.

Actually, as I posted above, there is a new system in the US. It's currently the #3 system.
http://www.top500.org/system/8128

"Does it run Linux(R)?"
Umm.. Yes as do most supercomputers these days.
http://www.top500.org/stats/26/osfam/
When it comes to supercomputers Linux is the OS of choice these days. Even Mac OS/X has five times the market share of Windows when you talk about the top 500 supercomputers. You see when it comes to doing real work Windows is just a hobbyist play thing. The big boys run Linux.

Generally speaking, clusters who want high performance used something like Myrnet instead of ethernet. It's like the difference between consumer, prosumer, and professional products you see in, oh, every industry across the board.

• Myrinet is the second most-used interconnect in the TOP500 at 14% (70 out of 500) followed by HyperPlex at 6% (31).
• Gigabit ethernet is by far the most used interconnect at 50% (249).

In the TOP500, it looks like ethernet is not yet an "outsider." Perhaps in the "top 100."

Generally speaking, clusters who want high performance used something like Myrnet instead of ethernet. It's like the difference between consumer, prosumer, and professional products you see in, oh, every industry across the board.

• Myrinet is the second most-used interconnect in the TOP500 at 14% (70 out of 500) followed by HyperPlex at 6% (31).
• Gigabit ethernet is by far the most used interconnect at 50% (249).

In the TOP500, it looks like ethernet is not yet an "outsider." Perhaps in the "top 100."

Forgive me if I'm incorrect here, but the 7-core cell processor that is shipping in the PS3 has a thoeretical max performance of 218 GFLOPS according to Wikipedia: (http://en.wikipedia.org/wiki/History_of_video_gam e_consoles_(seventh_generation))

Now the 500th fastest computer in the world has a speed of 1645 GFLOPS (http://www.top500.org/lists/2005/11/basic/5)

So essentially, all I would have to do is purchase eight PS3 systems for 5000 bucks, beowulf cluster, and I'll have one of the 500 fastest computers in the world.... And that's not even counting the GPU. Not to mention the fact that the hardware is built for Linux and requires NO hacking.

Who the /hell/ said the PS3 was expensive?

Windows has a long way to go before it can compete on serious servers. And Linux on these small devices really is an odd fit for it - but I guess necessary due to the expensive licenses of alternative OS's.

In case anyone was wondering how Columbia stacks against their rig, check out:

http://www.top500.org/

Here's the November 2005 list:

http://www.top500.org/lists/2005/11/TOP10_Nov2005. pdf

It shows Columbia with:

51.87 Rmax (teraflops/second).. It also states that it moved from #3 ranking to #4.

In case anyone was wondering how Columbia stacks against their rig, check out:

http://www.top500.org/

Here's the November 2005 list:

http://www.top500.org/lists/2005/11/TOP10_Nov2005. pdf

It shows Columbia with:

51.87 Rmax (teraflops/second).. It also states that it moved from #3 ranking to #4.

Exactly. As another example, there is a little known computer based on a 700 MHz PowerPC chip, the PowerPC 440, which seems to perform quite well ...

Exactly. As another example, there is a little known computer based on a 700 MHz PowerPC chip, the PowerPC 440, which seems to perform quite well ...

Exactly. As another example, there is a little known computer based on a 700 MHz PowerPC chip, the PowerPC 440, which seems to perform quite well ...

According to http://www.top500.org/, ASC Purple is number 3. Number 1 is BlueGene/L and number 2 is BlueGene W.

(I don't want to be a fanboi but) this kind of shows that while people say "Linux is not ready for the desktop" it is ready for certain other tasks that you need computers for:
http://www.top500.org/lists/2005/11/l/Operating_Sy stem_Family.

(I don't want to be a fanboi but) this kind of shows that while people say "Linux is not ready for the desktop" it is ready for certain other tasks that you need computers for:
http://www.top500.org/lists/2005/11/l/Operating_Sy stem_Family.

Especially this one:

http://www.top500.org/lists/2005/11/l/Operating_Sy stem

Shows linux vs the rest. Note on perf that linux+CNK/linux have 72.9% of total performance.

My understanding unix rocks with clusters and similar , linux and windows far less.

This list says you should apt-get upgrade your understanding.

Well, from the article, the main part of this is a Blue Gene system (57.3 Teraflops according to the article).... so looking at http://top500.org/lists/2005/11/TOP10_Nov2005.pdf that would put it in 4th place... if you add the Hitachi flogs in there.... 4th place.

Isn't that nr 6 in the world?
The list is here.

Yeah, except for the most powerful supercomputer clusters in the world: http://www.top500.org/

He said "desktop". Not "any computer that could concieviably run Linux".

Go AC Go!

Be honest and matter-of-fact about it. Tell them the truth and hope that they are smart enough to realize how this will help the company.

You can say impressive things without lying. For instance, you can say (if it happens to be true): "I trust Linux for my home computer and all my important files." That alone means alot. Or you can say "if I were asked to place a $1000 bet on a computer OS that would run without getting infected with viruses or crashing for a whole year (while connected to the net!) I would place the bet on Linux instead of Windows."

Or, you can point out other projects/companies. For instance, according to top500.org, in 2005, 390 of the top 500 super-computers were using Linux. That means that 78% of super-computers run Linux. For instance, the world's most powerful computer is IBM Blue Gene, and it uses Linux for its I/O nodes (more info here). Also, Google's gigantic, powerful, and distributed search engine runs using over 60,000 Linux machines (more info here, here, and on Google's Research page). The fact that big, complicated, and highly successful operations use Linux shows what it can do. In the case of Google, it shows that they trust it to deliver the security they need.

You can urge them to get a second opinion. For instance, tell them to look over Secunia's report on Windows XP compared to Ubuntu 5.10.

Ultimately, however, all you can do is provide them with an honest assessment of Linux' strengths and weaknesses, and point out in what ways the media reports are wrong. If they respect your opinion, then they'll make the right choice. If they refuse to listen to reason, then there is nothing you can do. People who are more interested in media sound-bites than expert discussion are essentially impossible to convince of anything they don't already believe. Don't waste your time, and don't buy company stock.

Be honest and matter-of-fact about it. Tell them the truth and hope that they are smart enough to realize how this will help the company.

You can say impressive things without lying. For instance, you can say (if it happens to be true): "I trust Linux for my home computer and all my important files." That alone means alot. Or you can say "if I were asked to place a $1000 bet on a computer OS that would run without getting infected with viruses or crashing for a whole year (while connected to the net!) I would place the bet on Linux instead of Windows."

Or, you can point out other projects/companies. For instance, according to top500.org, in 2005, 390 of the top 500 super-computers were using Linux. That means that 78% of super-computers run Linux. For instance, the world's most powerful computer is IBM Blue Gene, and it uses Linux for its I/O nodes (more info here). Also, Google's gigantic, powerful, and distributed search engine runs using over 60,000 Linux machines (more info here, here, and on Google's Research page). The fact that big, complicated, and highly successful operations use Linux shows what it can do. In the case of Google, it shows that they trust it to deliver the security they need.

You can urge them to get a second opinion. For instance, tell them to look over Secunia's report on Windows XP compared to Ubuntu 5.10.

Ultimately, however, all you can do is provide them with an honest assessment of Linux' strengths and weaknesses, and point out in what ways the media reports are wrong. If they respect your opinion, then they'll make the right choice. If they refuse to listen to reason, then there is nothing you can do. People who are more interested in media sound-bites than expert discussion are essentially impossible to convince of anything they don't already believe. Don't waste your time, and don't buy company stock.

I think that it is worth pointing out that is not quite true.

I think that it is worth pointing out that is not quite true.

I think that it is worth pointing out that is not quite true.

Why not take a look at the top500 list? They have a nice piechart by processor family. We can see that there are more AMD X86-64 systems than IA-64 systems. You also have to keep in mind that there is a large lag time from when a top500 system is planned and when it's actually purchased, installed, and makes it on the list. Opteron is a newer processor and adoption takes times. Expect to see a huge increase in the number of opteron systems on the next list.

XMilkProject said: "...and it's also fair to say that most of the 4% of OSX users are in very non-technical fields, and far less likely to be capable of the technical mischief"

On the contrary it's Windows that is not up the task of running in highly technical areas. The University of Virginia built the currently 20th fastest supper computer in the world out of essentially stock Mac Xserves see: http://www.top500.org/lists/2005/11/basic

How many Windows boxes are on the top 500 list, hint, zero because Windows is not a supper computer class operating system. The BSD based Darwin is a real Unix, it can run XII and in fact G5 powermacs are used in genetics research as a full fledged workstation, despite being a consumer "desktop" computer etc.

See for example: http://www.bio-itworld.com/products/041604_finch.h tml

http://www.macobserver.com/article/2004/01/13.4.sh tml

Microsoft does make a decent word processor I wrote this in word, that's about as "technical" as I'd like to get with M$ software.

Please do some research before you just spout off a troll M'K?

http://www.top500.org/site/297

As a sysadmin for JPL I ran a few major Linux clusters and standalone Linux servers. JPL is very segmented, there are divisions, and within divisons there are projects.

Many of the projects run their own infrastructure. Some on Linux, some on HPUX, some on Windows, and so on. Perhaps in the MER project they didn't do too much with Linux on the server, but that doesn't mean that JPL likes or dislikes Linux for servers. It's more representative of the admins in control of MER more than anything.

In otherwords this interview is not representative of JPL. Just like everything i say isn't representative of JPL as a whole. It's just our individual projects.

Or this one.

We don't run our main servers on Linux

Oh, really?

So explain this guy (www.top500.org).

Not quite what you were asking for, but...
I went to have lunch with a friend, that had been gone on travel for a couple weeks. She showed me around the building (ex-top-twenty supercomputer http://www.top500.org/lists/2005/11/basic), introduced me to a Network Specialist. He showed me the "off-limits" areas. I ended up getting an interview that day, and two weeks later I started. I've been here two years now.

the parent is a troll and an idiot, but you seem to be genuinely asking, so i'll take the time to answer.

GUI quality: The troll gives no indication of what or how he's measuring. it's difficult to deny that MS's GUIs are more polished, but there are numerous inconstancies. GUIs available on unix systems, including FreeBSD, tend to be more configurable. i'm inclined to agree that traditional X11-based GUIs are behind that of Windows, but that's a far cry from FreeBSD not having one, as the troll claims. also, OS X is widely agreed to be easier to use than Windows' and is unquestionably more technically advanced (we'll see what Vista brings).
Support: The troll's claims that Microsoft is "the world's most trusted software company" is simply laughable. major failures in security and stability in Microsoft products are legendary; their reputation for quality is thoroughly mediocre. they are, however, quite large and do stand behind their products (such as they are) for defined periods of time, which has a certain level of comfort associated with it. FreeBSD, on the other hand, has much higher initial quality and also has commercial support available from various sources. the open source nature of FreeBSD and the vibrant community existing around it also means particularly obscure problems are more addressable than they are in Windows, where you're left waiting for Microsoft to release a patch. again, there are trade offs to be made, but i think FreeBSD is a clear winner here.
Cost and convenience: It is undeniable that having the system pre-installed is a huge win for convenience. but the troll goes way off-track from there. first, XP is available pre-installed, but for how many architectures, maybe two (x86 and itanium)? FreeBSD is available on about a half dozen (NetBSD, incidentally, is available on dozens); this is particularly important in the sever and appliance realms, which are FreeBSD's primary target spaces. FreeBSD is available pre-installed at least on server equipment (i don't know of anyone who does workstations/laptops). the troll claims that XP is free, which is flatly false: the cost is bundled in the cost of the hardware. the troll is also implicitly defining terms like "every major manufacturer" to be only ones he cares about: get me an XP system from Sun or Apple, for example.
Stability/scalability:Again, the troll gives no measurements. at a minimum, XP has a reputation for being unreliable. in my experience at work, XP is a step down in stability and reliability from 2000, although both of these are still leaps ahead of any Microsoft system predating that (except probably DOS, which was highly stable by virtue of being so tremendously simple). DoS-style attacks which bring down the system remain common against XP and virtually unheard of against FreeBSD. FreeBSD is highly stable. the standard edition of XP also scales to 2 processors; special versions are available to get it up to higher number, but still pretty modest number of processors (i think it was 16, but i don't remember). i'm not sure specifically what SMP problems the troll is talking about (again, no specifics), but i've personally run FreeBSD on dual-processor SMB systems without issue and other BSDs on systems much, much larger than any Microsoft product has any hope of touching. for reference, note that BSD-based systems hold many places in the Top 500 supercomputer list, including several in the top 20; Windows can't hope to touch that level of performance.
Software availability: No, troll, not everyone uses it. but yes, it does have more software. for that reason, when i was Director of IT for our company, we continued to by Windows boxes; our accounting package wasn't available on any other platform. but this very much depends what you need. FreeBSD certainly runs a far cry more than vi. most things that'll run on other open-source systems like Linux,

Oh really? Perhaps you should check the latest Top500 list.

From the 11/2005 TOP500 list:

#6 - Cray XT3 - 38TF/s
#10 - Cray XT3 - 20.5TF/s
#14 - Cray XT3 - 17TF/s
#17 - Cray X1E - 15TF/s

Of course Linpack doesn't stress other important aspects of a supercomputer such as memory bandwidth and interconnect latency. The HPCC benchmark is a better measure of performance on real-world applications. Cray systems do very well on HPCC as well.

A year ago Apple computers made number 7 it the top list of supercomputers http://www.top500.org/lists/plists.php?Y=2004&M=11 the story of them losing ground when they were higher is http://www.macnn.com/articles/04/11/08/vt.falls.to .7th/

If I wanted to go with the easiest computer operating system to use, that is heavy on graphical user interface, I would pick a Mac. It said in the ranking list: "System X 1100 Dual 2.3 GHz Apple XServe/Mellanox Infiniband 4X/Cisco GigE / 2200 Self-made"

Top500 classifies supercomputers as clusters, constellations, and MPPs (Massively Parallel Processors). BG/L is an MPP. It is obviously not a cluster since a single compute node is not a general-purpose machine. BG/L is specifically intended for massively parallel applications.

The difference between clusters and constellations is the ratio between the number of nodes (that can either work independently or be clustered) and the number of processors in a node.

Go to the Sublist generator of Top500 to see what machines belong to which category.

See, e.g., definitions here