Slashdot Mirror
All 500 of the World's Top 500 Supercomputers Are Running Linux (zdnet.com)
Freshly Exhumed shares a report from ZDnet: Linux rules supercomputing. This day has been coming since 1998, when Linux first appeared on the TOP500 Supercomputer list. Today, it finally happened: All 500 of the world's fastest supercomputers are running Linux. The last two non-Linux systems, a pair of Chinese IBM POWER computers running AIX, dropped off the November 2017 TOP500 Supercomputer list. When the first TOP500 supercomputer list was compiled in June 1993, Linux was barely more than a toy. It hadn't even adopted Tux as its mascot yet. It didn't take long for Linux to start its march on supercomputing.
From when it first appeared on the TOP500 in 1998, Linux was on its way to the top. Before Linux took the lead, Unix was supercomputing's top operating system. Since 2003, the TOP500 was on its way to Linux domination. By 2004, Linux had taken the lead for good. This happened for two reasons: First, since most of the world's top supercomputers are research machines built for specialized tasks, each machine is a standalone project with unique characteristics and optimization requirements. To save costs, no one wants to develop a custom operating system for each of these systems. With Linux, however, research teams can easily modify and optimize Linux's open-source code to their one-off designs. The semiannual TOP500 Supercomputer List was released yesterday. It also shows that China now claims 202 systems within the TOP500, while the United States claims 143 systems.
From when it first appeared on the TOP500 in 1998, Linux was on its way to the top. Before Linux took the lead, Unix was supercomputing's top operating system. Since 2003, the TOP500 was on its way to Linux domination. By 2004, Linux had taken the lead for good. This happened for two reasons: First, since most of the world's top supercomputers are research machines built for specialized tasks, each machine is a standalone project with unique characteristics and optimization requirements. To save costs, no one wants to develop a custom operating system for each of these systems. With Linux, however, research teams can easily modify and optimize Linux's open-source code to their one-off designs. The semiannual TOP500 Supercomputer List was released yesterday. It also shows that China now claims 202 systems within the TOP500, while the United States claims 143 systems.
Linux was originally made in Finland.
When all you have is a hammer, every problem starts to look like a thumb.
five years ago, 3 of the top 500 did run windows, and in 2011 4 did.
Eh, no, not really. You're talking about a KDE 1.0, pre-Gnome desktop... I used it, but I wouldn't have inflicted it on anyone I needed to support. Five years later it was certainly reasonable, at least where the average non-technical user was concerned.
KDE was '96, GNOME '97.. in 1999 you'd already have KDE 2.0, didn't use that but I remember trying RHL 6.2 that came out in April 2000 which looks pretty much like a normal desktop to me. Remember that it was going head to head with Windows ME as the consumer desktop, using either was a major PITA. Granted, XP was a big step up but then you had Vista... you can make a lot of excuses for YotLD not happening but that Microsoft brought their A-game is not one of them.
The cornerstone for Microsoft's dominance is Office and Excel in particular, all those people who had to use Windows at work of course took what little knowledge and training they had and bought a Windows machine for home too. When Outlook kicked Lotus Notes to the curb they locked that market up good.
Live today, because you never know what tomorrow brings
Linux still isn't ready for any desktop it isn't installed on. It IS installed on lots of desktops in places like research labs, mine included. But if it's going to make it to anybody else's desk it needs some basic things fixed. I don't know if it's possible to do something as simple as configure a graphics driver in Ubuntu's GUI, but it's certainly not easy.
Everything else works perfectly fine, but none of the GUI systems seem to offer a user friendly way for command line averse users to fiddle with their system settings.
Both AMD and Nvidia have their config UI packaged to their binary blob driver on linux.
I'm not even sure what you are asking here. Do you truly have no idea how a GPL works?
Anyway, you have this exactly backwards. The reason Linux became popular during the parallel supercomputing "revolution" (and I say this as a modest expert, at least at that time) is because it IS an open source operating system, so you could hack the kernel, write your own kernel drivers, fix things like networking bugs or system balance issues, and handle memory at a very primitive level. You got then, and can easily get now, the complete source of the OS and all of its device drivers, although the latter has been a constant source of contention between hardware mfrs who think that a device driver that makes their hardware run is some sort of "trade secret" and the keepers of the Linux kernel. Over decades (at this point) the mfrs have largely given up and actively help with kernel drivers instead of insisting on binary-only distributions. This played a critical role in the development of early parallel supercomputers once Linux had its first kernel capable of symmetric multiprocessing with two (and rapidly more) CPUs or (later) cores, or both. That would be roughly kernel 2.0, although there were still serious issues with race conditions, (network) driver interrupts and lockups, memory management, and so on, through 2.0.4+ -- really they went on forever as the 2.0 kernel wasn't truly symmetric, handled interrupt locking "badly", and took a lot of revision and some new paradigms to smooth out and stabilize. Ah, those were the days...
Microsoft, on the other hand, made you sign away your firstborn child in order to get a copy of the OS source -- even as a research institution. If (say) your network drivers were slow, or locked up while multiprocessing, you were SOL. You COULDN'T fix it. You couldn't even find the bug. And it wasn't worth the effort -- even if you sacrificed a goat and got the source -- to learn to work with the source because it changed at MS's whim and all your work could go down the tubes at any moment and if you DID develop anything that ran on their system in some "custom" fashion, you ran into serious issues if you wanted to share it. You COULDN'T share your work with anybody else, not unless they had a surplus of goats or firstborn children too.
"Anybody" (with a need and decent programming chops) could join the linux kernel list and communicate directly with the main kernel developers and report bugs, contribute fixes or drivers, etc. There was a lot of healthy debate about what needed to be fixed, or improved, first, second, third etc, as well as just how to go about fixing them -- sometimes it required substantial redesign and had to wait for a major bump (and a lot of testing). You could of course hack/fix your own kernels or add your own device drivers, or fix broken drivers, or mess with internal "tuning", and I and many others did, but behind the public scenes the actual kernel developers -- the heart of linux, as it were -- made steady, inexorable progress.
By the year 2000, Linux had made serious inroads into not only the top 500, but there were literally uncounted small clusters that weren't fast enough (or weren't architected correctly) to crack the top 500, which relied on things like the Linpack benchmark to determine who to include. There were lots of folks who didn't USE linear algebra in their computations who built massively parallel compute farms with many different architectures and purposes who didn't even have the benchmark software installed (or give a shit) about their "ranking". Both PVM and MPI were fully ported onto Linux and most of their ongoing development was taking place on Linux boxes. Additional tools for management and job distribution and much more were developed -- on mostly Linux boxes, but yeah, there were still SGIs and Sun Microsystems clusters and much more out there. They suffered -- badly suffered, terminally badly suffered in pretty much all cases -- from being much, much more expensive than over the counter Intel or AMD box
Even when the experts all agree, they may well be mistaken. --- Bertrand Russell.
Bullshit.
Linux is used because it's far, far, FAR more flexible, less resource intensive and more efficient than Windows, while supporting and making good use of vastly larger amounts of RAM and CPUs.
If you baseline is one of the proprietary Unices, it's still more flexible, less archaic and more familiar to users while supporting a wider range of hardware while being infinitely cheaper.
> The Supercomputers level of OSS use is primarily a concern with science. It compiles on multiple platforms, and is well maintained on most of them. Windows and MacOS are only available for the x86-64, ARM, and PPC platforms, and even then, not all of them.
This makes no sense. Almost all supercomputers are x86-64 based (+/- GPUs).
> That only leaves FreeBSD as an option, and FreeBSD isn't as virtualization friendly, and drivers aren't readily available for GPU systems.
Lol. Supercomputers don't use virtualization.
> So it's quite literately the only logical choice, owing to that the other choices would have required engineering resources.
That's not true, supercomputers within the past 5 years on the top500 list have used Windows, AIX, BSD, Linux. It's just that Linux is better for the job than the others.
> That said, Linux does not belong in safety systems, and I hope it never ends up in car automotive systems, power plants, or spacecraft.
I hope nobody who thinks supercomputers use virtualization ever have their opinion on a computing matter taken by the designer of a safety critical system.
Linux is in safety critical systems already. But it depends on the level and capabilities you're talking about. Processing doppler radar data and sending it to ATC systems in a timely manner is one thing. Running tight control loops in automotive engine and control systems is completely different and just isn't appropriate for Linux.
> Everything else is fair game. These systems need real time operating systems that are highly threaded and can respond to events instantly, not be scheduled, or deferred due to eating all the swap space (one of Linux's worst default features, and what makes it woefully awful for web servers by default.)
You're mixing up all sorts of things here. Nothing responds to interrupts "instantly", what you want is guaranteed hard upper limits. It doesn't even have to be all that fast often times, it just has to be an upper limit so you can design the system to meet response time requirements. Linux can respond "immediately" to interrupts, by the way. It does not have to be "scheduled". Work can be done in interrupt context.
"Highly threaded" what? That's nothing to do with real time.
"Deferred due to eating all swap space" What is this meaningless drivel? Automotive and aircraft control systems don't use swap space. They don't even use virtual memory for god's sake lol.
> (one of Linux's worst default features, and what makes it woefully awful for web servers by default.)
Apparently better than all the others at that too. Windows, OSX, and BSD must *really* be shit if Linux is so bad yet it still beat them all there too.
It has nothing to do with not being able to handle the load. It has everything to do with costs. Linux is free. Windows isn't. Most of the tools for supercomputing were written for the linux platform. There are tools for windows but since it's a niche market there aren't nearly as many. And since in the super-computing world having a good desktop/GUI environment doesn't mean squat there is no real incentive to use windows outside of certain circumstances.
In all the time I used it I never encountered any serious problems other than the lack of tools and tracking down a couple numerical inconsistencies (issues with compiler differences). It ran fine. It handled loads fine. There's nothing wrong with it's MPI performance, nor any performance issues with it's infiniband stack. If you have a windows admin they can basically hop right on it with little difficulty, and if you're a windows shop then tying it together with AD and such is pretty simple. If you're going full windows, then it might make sense to consider it but there are very few organizations where that's the case.
But there's no getting around the costs.
~X~