Slashdot Mirror

I think you mean the KLAT2 project and their Flat Neighborhood Networks.
Good stuff.

I think you mean the KLAT2 project and their Flat Neighborhood Networks.
Good stuff.

The Klatz-2 implemented this technology... Here's a story on it and a link to their wiring app...

Have you ever heard of the Flat Neighborhood Network topology? Just because you can't figure out how to design a network for a cluster supercomputer, it doesn't mean nobody can. Apparently, you also seem to think you know Cray better than Cray. Not only is Cray now marketing Linux server clusters, it's also planning on building a cluster supercomputer using 16000 AMD Opteron processors. Perhaps you should jump in and inform them that clusters have gone the way of the dodo. Either you're wrong, or many highly paid Cray employees and various PhD's are.

*applies Ockham's Razor*

Good dual amd boards come with gigabit ethernet. With prices as they are, the nodes can be put together for about $350,000. That would leave $150,000 for 512 ports of gigabit switches. Cisco gigabit 48 port switches run $5,000. Double that and add an additional nic to each box and use a flat neighborhood network (.pdf)

That should give each node about 200 MB/s aggregate bandwidth (the best gigabit ethernet runs at 800 Mb/s or 100 MB/s), easily exeeding what can be achieved with much more expensive solutions.

About the cost of a nice house.

Put into perspective, a cluster that could outperform Japan's earth simulator would cost 2 million in hardware costs. Outperforming Seti@home's 3,000,000 users would require $10,000,000.

I know where my lotto money is going :P

Since its optimization you are concerned with I have a few choices you will be interested in:

1. The Zen of Code Optimization by Michael Abrash.
2. Agner Fog's Assembly Resources
3. The Athlon Optimization Guide
4. Intel's IA32 Optimization Guide
5. The Aggregate Magic Algorithms

These sources will give you everything you need to know about code optimization for x86.

This is seriously disturbing!

When I go to http://www.aggregate.org/KLAT2/

I get a "404 Not Found" But you get to a real web site?

What's going on here?

I don't think there is anything special about my IP address... just tried it from two different netblocks and I can get these links just fine:

Genetic Algorithm CGI
Main Site

It's at a university - maybe they blackballed your subnet at their firewall because some loser tried cracking their systems from your site? I dunno. Maybe your browser is just busted, I'm using Mozilla 1.0 and it works fine for me.

I don't think there is anything special about my IP address... just tried it from two different netblocks and I can get these links just fine:

Genetic Algorithm CGI
Main Site

It's at a university - maybe they blackballed your subnet at their firewall because some loser tried cracking their systems from your site? I dunno. Maybe your browser is just busted, I'm using Mozilla 1.0 and it works fine for me.

I was always amused at how a unibus PDP-11 with 512K of main memory could beat the snot out of a 386 at real-world tasks. I/O is critical for so many applications...

However, don't write off clusters yet; have you looked at The AGGREGATE? The link points to Klat2 (Gort, klaatu barada nikto! Sorry) which is a very photogenic aggregate-based machine. The techniques these guys are developing may bring high I/O throughput into clustering at mainframe levels eventually.

In many clusters today like KLAT2 they only use hard drives for the root nodes, and the other 98% of nodes use 2GIG of ram.

This saves you at least $150 per slave node by not buying a hard drive, thousands for having to deal with less hard drive failures, and acess times are orders of magnitutes better.

Lets do the math. 512MB of PC133 on pricewatch today was $67. For 2GIG of ram that comes out to $268 per node. For a terabyte(2modules*$67*1000GB)=$134,000.

That blows my mind. A small research lab can now own a terabyte of PC133 for under $150,000. Man, do I feel old.

Ethernet has very high latensy at about ~10 milliseconds. Projects like PAPERS and the KLAT2 use the parallel port to connect compute nodes because of the much lower 1 ms latency.

Ok, no parallel port on Macs... but I wonder how do Firewire ports perform?

Ethernet has very high latensy at about ~10 milliseconds. Projects like PAPERS and the KLAT2 use the parallel port to connect compute nodes because of the much lower 1 ms latency.

Ok, no parallel port on Macs... but I wonder how do Firewire ports perform?

Nice in theory, but in practice Gigabit ethernet does not deliver Gb/sec - and in fact Ethernet's contention access mechanism (CSMA/CD) guarantees that you won't!. In practice you're better off with channel bonded 100Mb/sec Ethernet and a smart switch topology such as KLAT2's Flat Neighborhood Networks to minimize latency and maximize inter-node bandwidth.

KLAT2 FNN

Note to moderators: above link refers to Linux, AMD processors and Beowulf clusters! Do the right thing! ;-)

thats why you use a nifty cgi to help with the topology.
64 Nodes, $42000, 64.459 GFLOPS on 32bit ScaLAPACK

If you look the recent article about "A Supercomputer in Every Garage" and follow a link to the KLAT2 project it says that they used GA to work out their network topology...

I'm pretty sure that that counts as a practical use.

http://aggregate.org/CDR/ , the Cluster Design Rules tool

You specify some characteristics of your application, your site (power and space), and budget; it presents the best designs taken from a design space of millions.

When you are comparing costs for construction, don't forget the essentials.

16 Pizzas for student helpers @ $10
4 Cases of soda student helpers @ $7

Total: $188

For example, in Lineo's response we read:

MS:Networking support in embedded Linux simply is not as comprehensive and feature-rich as that in Windows. Linux does not offer..."

LN:Lineo: The Microsoft author should review the state of the current Linux 2.4 kernel. Linux grew up on the Internet. Its support for networking services is comprehensive. Due to the open source nature of Linux, elements that may not yet be supported can be added by any competent engineer. Just because Red Hat doesn't support a driver doesn't mean it isn't there and usable. This has more to do with Red Hat test bandwidth than Linux maturity. They are picking nits here."

Looks like Lineo is so onanistic about Linux that they're leaving out or conveniently forgetting that 802.11b is a standard that most handheld/palmtop systems - which both of these embedded OS's are targets of - will be a requirement for most users of such devices.

In the section about "Localization", Lineo responds in a fairly arrogant tone:

MS:Multilanguage availability in Linux is extremely limited for both OS localization and application development. Localizing the OS means rebuilding it in the desired language.

Lineo: The OS should never be localized in the embedded space. You localize the user interface, not the OS. Even desktop Linux versions don't localize the OS--it's not necessary.

A fairly arrogant supposition at best, if not downright disingenuous.

Even more "NFL-calibre" dodging and weaving in this next one:

MS:To find and fix security flaws, Linux relies heavily on the Open Source community and source access, assuming that the "many eyes" of the Open Source community can equal a more secure OS. Even vendors such as Red Hat cannot guarantee the reliability and security of their OS. Red Hat's Security and Exchange Commission (SEC) filing states that they may not be able to test their software effectively because independent software programmers, over whom Red Hat has no control, contribute to Open Source programs.

Lineo: SEC filings always contain very alarming risk statements, even when they are unlikely, in the interest of full disclosure to the investment community. The reality of Open Source software and the "many eyes" theory is realized by comparing the track record of security reports for Microsoft OSes and Linux. Microsoft's Windows XP End User License Agreement is a telling report of Microsoft security record.

If the Linux Caliphs didn't mean what they meant in the SEC filing, then why did they put that in there? This is the last-nail-in-the-coffin for Embedded Linux; if the Linuxians who penned the SEC filing info said that, then it's a EXTREMLY telling fact of how reliable they think Linux and E-Linux is.

Another "chalkboard" lesson that any NFL RB would take to heart:

MS:Linux looks to Java as a distributed applications development environment. Java provides only a set of least-common denominator services across different OSes. Java was originally designed to be interpreted, and only gained the ability to compile down to native code after a major retrofit. It fails to take advantage of the unique capabilities of the underlying OS.

Lineo: Translation: .Net is a proprietary solution targeted at only Windows platforms.

DODGE...FADE...FAKE...RUN...DODGE AGAIN...SPIN...BUMP-n-RUN!

Lineo doesn't even address the issue and throws some obscure reference to .Net in a vain attempt to smokescreen the issue. Why? Because MS hit it right-on-the-head. Also, the Lineo response is incredulous, due to the purported work on Linux/*NIX/OpenSource projects to provide services compatabilites to .Net, so that Linux can interoperate with .Net!

And finnaly, the "smoking gun" for OpenSource!

MS:An OEM is required to sort through a potential legal morass of licensing issues around the GPL if it wants to protect its intellectual property rights. This creates extra costs from both a development and legal perspective. An example of this risk can be taken from Nvidia. An Nvidia programmer, in the course of developing a driver for one of its products, used a portion of code from a freely available video driver. The developer failed to realize the code was licensed under the GPL and would therefore require Nvidia to release the source code for its entire driver. Because Nvidia did not want to release the source code to its commercial software, the company incurred substantial costs to develop a new driver that did not contain the GPL code.

Lineo: The open source licenses are often spoken of as a disadvantage but they are truly a strong advantage. What they offer you is a choice: Do you want to use existing, freely available code (and contribute the resultant product back to the community) to quickly produce your driver or application or do you want to program from the ground up--increasing the development cycle, but avoiding the derivative work clauses of the open source licenses? This choice is not offered by a closed operating system such as XP Embedded.

Lineo spins and spews their own brand of FUD here, and the example that MS gives is an actual event that happened. OpenSource and the GPL are great if you're going to tinker and play with an OS on your own system at home, but it's a disaster waiting to happen when you start developing it in the manner that eLinux is being developed.

Additionally, I have a few questions for the Linux/OpenSource mavens here:

1) Will the end-customer be required to do a "build-and-nmake" on their own PDA or other eLinux device when a new rev of the software is released?

2) Who will the end-customer go to for support; the manufacturer of the device, or to some arcane amalgumn of Linux/OpenSource coders who tell the end-user that they'll have to change such-and-such lines in the sourcecode and then re-compile & re-build the OS?

3) What will happen when the end-user wants to make use of 802.11b or the other new "WiFi" connectivity scheme; have drivers made by the manufacturer that are certified by the manufacturer of the OS, or wait until some guy who's spent 7 years on his B.S. degree wakes up from his recouperation from a 36-hour Quake3 Frag-fest at his best friend's appartment to finish the code for the driver...and leaves out a vital piece of init-code for the driver to talk to the device?

As I've always said: Linux *does* have it's place - in Academia, the Hobbyist's desk, a web-server that doesn't need direct interaction with the end-user, or a super-cool Beowolf cluster like the one mentioned here on /. a few days ago (I tried to find the article, but unable to find it) - go to U of Kentucky's KLAT2 cluster; however, Linux and eLinux will not be the OS-of-choice for the average home/business because of the obstacles it must overcome (3rd Party software, compatability issues, support issues)

Linux="Linus, I Now Understand Xenophobia"

Why are you buying a single box? Is it necessary? You can find a lot of information about distributed computing at the aggregate. You could use the money to build a computer as fast, and have computers to use for other things when the computer is obsolete (6 months). :)

We are releasing the technology into the Public Domain as soon as we can (i.e. when the hacked code has at least some clarity/documentation). So, yes, you can apply the concepts from KLAT2 to most any size cluster. See my my other comment for a little more info.
--

As for using the 3DNow! stuff, their K6-2's can have some real punch if they are willing to code for it... Check out our SWAR - SIMD Within A Register compiler technology for doing just that. Actually, the Ph.D. student doing most of the work on SWAR is AT Purdue.
--

As for using the 3DNow! stuff, their K6-2's can have some real punch if they are willing to code for it... Check out our SWAR - SIMD Within A Register compiler technology for doing just that. Actually, the Ph.D. student doing most of the work on SWAR is AT Purdue.
--

As for using the 3DNow! stuff, their K6-2's can have some real punch if they are willing to code for it... Check out our SWAR - SIMD Within A Register compiler technology for doing just that. Actually, the Ph.D. student doing most of the work on SWAR is AT Purdue.
--

As for using the 3DNow! stuff, their K6-2's can have some real punch if they are willing to code for it... Check out our SWAR - SIMD Within A Register compiler technology for doing just that. Actually, the Ph.D. student doing most of the work on SWAR is AT Purdue.
--

Our Athlon based KLAT2 Beowulf cluster at the University of Kentucky achieved over 64 GFLOPS on LINPACK for only $41K using 3DNow! instructions. The FreeBSD Cluster at Purdue doesn't even mention ANY benchmarks for performance. I'm a Purdue Alum, so I think this is great that they are getting slashdot coverage for an inexpensive cluster. However, when we submitted KLAT2's (Kentucky Linux Athlon Testbed 2) story to slashdot last month, which in many respects is much more "news for nerds", it got passed over. Ah well, thats the way of slashdot.
--
--

Our Athlon based KLAT2 Beowulf cluster at the University of Kentucky achieved over 64 GFLOPS on LINPACK for only $41K using 3DNow! instructions. The FreeBSD Cluster at Purdue doesn't even mention ANY benchmarks for performance. I'm a Purdue Alum, so I think this is great that they are getting slashdot coverage for an inexpensive cluster. However, when we submitted KLAT2's (Kentucky Linux Athlon Testbed 2) story to slashdot last month, which in many respects is much more "news for nerds", it got passed over. Ah well, thats the way of slashdot.
--
--