Slashdot Mirror
In The Works: Windows For Supercomputers
Robert Accettura writes "According to ZDNet, Microsoft may be feeling threatened by Linux gaining ground in the High Performance Computing (HPC) arena. As a result, they have formed a HPC group to bring windows to these systems. It makes a mention of how clustered computing may be a target. I guess the only thing better than crashing 1 computer at a time is crashing an entire room full at once."
...certainly seems to want a finger in every pie.
There's mischief and malarkies but no queers or yids or darkies within this bastard's carnival, this vicious cabaret.
The point is that Microsoft can probably work very hard to castrate the gui windows out of Windows(tm) and end up with smallish kernel or micro-kernel architecture. They would then own the architecture and could bring in any interface technology they desire. And you could compile your non-gui code with VC++! Almost as useful as Linux on a Beowulf cluster only with large licensing fees.
Mainframe on the Internet? Windows, linux, OpenVMS, AS400, VAX.. doesn't matter, if you put it on the Internet, you're a moron and your entrails should be extracted slowely with rusty pliers through your eye sockets, end of story.
I would imagine that the hardware used in HPC would be very, very varied. Putting an OS on would probably require modification to the source code, especially with a system like windows, with everything integrated into the kernel. I can't see a closed source OS being used for an application where mainly custom programs are being used anyway, and performance is more important than anything else. If you have access to the source, unneeded features can be disabled (eg. sound, usb, mass storage devices, etc.) And besides, why waste processing power on a GUI anyway?
Also, by integrating everything into the kernel, stability is compromised. In your desktop computer, stablility is somewhat less important than when you have hundreds, or thousands of computers doing parallel calculations for a nuclear weapons simulation.
I see your point, though I personally disagree that it's the same problem as when you get an unstable Linux (you can fix a Linux, with windows, all you can do is wipe and reinstall).
What really bugs me though is that this NT5 kernel that everyone loves so much, has half a dozen services that should be in user space, and before I get flamed cause 'NT is a micro kernel' it isn't, it started out as one, but then they just shoved all and sundry into kernel space to improve performance. OK, it's not the DMM (Dangerous Memory Model, the plague of Win9x where every page of memory had a piece realmode addressable space in it, which is why anything could bring the system down with incredible ease), but still this isn't the kind of machine I'd want running a cluster. It's bad enough on servers.
Ask 8 slackers a question, get 10 awnsers (a citation, but I can't remember from who)
Friends don't help friends install M$ junk.
Actually, they have done this with XP Embedded. We have tried this in a project for a windows-controlled device, and you *can* build a rather small Windows XP that has your program as "shell" instead of the usual Explorer. Maybe not quite as small as Linux in text mode, but it will do.
The claims about Internet Exploder being inextricably connected to the OS were pure FUD for the antitrust suit.
C - the footgun of programming languages
I'm a Windows dood at times, but I can't see this working. The only way it might work is if the nodes are not traditional Windows installs, but rather are the core kernel and support only - ie: they have no GUI or any of the fluff and just enough to get on with doing what they have the do. The management or user land machines (or whatever the correct term is in HPC land) of course would have some GUI components. Mind you, I've wished to see a stripped down version of Windows without the GUI (or being able to start the GUI as an option ala STARTX) for ages. Maybe this is the beginning... Good luck is all I can say. They ahve a lot of work to do if it is ever to be credible.
Eh, Windows is really a brand, not necessarily a codebase. Who says they'll stick full win2003 boxes on each node? Maybe each node will have the bare min to run .net framework. The next ver of windows server will have a commandline only version. Maybe a single graphical server to interface with the nodes. There, now you have clue.
Rocket science is easy. Neurosurgery, now *that's* difficult.
Is it just me or does the notion of a GUI on high performance computers sound at bit pointless.
With grid computing and the availability of high-performance graphics accelerators, there is now the ability to visualize results in real-time (climate simulation, finite-element analysis). This requires a graphics API (OpenGL), and consequently a windowing system (X Windows system/Motif, MS-Windows).
One solution is to have the results networked across to a separate workstation. The problem with this, is that the network becomes the bottleneck. Say you want to run a discrete event simulation of an attribute of the earth at 1 mile resolution (25000 samples longitude x 12500 latitude), and you're getting fifteen events per second, and you want to visualise a pincushion model of this. So you want to transfer (25000x12500x3 32-bit floats) across the network/second. While this is only 150 million lines/second, that's something like 52.5 Gigabytes of data/second going across the network. So you need all sorts of fancy caches/buffer memory to handle this communication. Alternatively, you just add the graphics card onto the supercomputer.
Well, unless Bill's going to introduce a version of Windows that doesn't have a Windows interface, WTF is the point? How many Beowulf nodes have you seen even plugged into a KVM? Windows is a stupid choice for a headless compute node just as Linux is a stupid choice for a home desktop.
It sounds like Wolfpack, Microsoft's clustering system for NT (circa 1997) is back. I guess Microsoft thinks we've forgotten about the last time they tried to get into this market. It didn't go well for them.
Am I part of the core demographic for Swedish Fish?
Arent' there more recent examples?
What about 'faster' cluster systems? Are there some with fewer processors then 600 that runs similar hardware with a different OS?
Where's the link to this list?
How can we verify context without that being provided?
If you ignore the other uses of a tool, does that make the tool less useful, or you less useful?
6. The system registry, a bad idea no matter how many processors you run. the registry is suposta be pretty much gone in longhorn, so i would assume that this would be based off of longhorn code
" For example, with the standard Windows install"
Uhm.. *Which* standard Windows install? Xp pro? 2K sp35? 2K3 sp69?
"It is not possible to kill a process from the command line without getting a Resource Kit utility from Microsoft."
Not true. XP PRO ships with tasklist.exe and taskkill.exe.
The first lists your processes and the second kills them. The second is quite useful, too, as you can mass-exterminate processes by username or other filters. Entirelly useful if you want to delete all the spyware & other-useless-crap your computer boots up with.
What I really don't understand is why it would be necessary or smart to brand such a product as Windows at all.
That's easy. Sun, for example, sells workstations to its server customers on the premise that you can develop your app on small, cheap machines then when it's ready deploy it staight into your data centre without needing to change anything. Solaris is built from the ground up for this; fundamentally your code neither needs to know nor care that its threads are being scheduled on a uniprocessor U5 or a 106-way E15K. In reality of course you have to test for race conditions if you have any inter-thread dependencies, but if you don't, it's plain sailing.
That's Microsoft's game plan here. They are selling a solution not a server. They are selling the idea that your developers will be able to use their familiar Visual Studio tools on their cheap desktops (and Visual Studio these days is an incredibly productive environment) then run that code on the "Windows mainframe" in the basement. That strategy worked pretty well for every Unix vendor that sold both servers and workstations, there's no reason that it can't work for Microsoft.
Sure Microsoft could develop or buy a dedicated HPC OS, but why would they? If you can be bothered right now, it's easy enough to develop on Windows and deploy onto big Unix boxes. Microsoft wants to make it seamless, and they're doing an interestingly good job of it. Pretty soon, you'll be able to use the same tools to develop for PDAs, tablets, PCs, games consoles and mainframes. That's the vision.
I thought they told the EU that they couldn't even remove mediaplayer from windows, and they told the US DOJ that they couldn't remove IE?
One is just a motherboard, processor and hard drive sitting in and around a motherboard box. This is the database server for my website.
They run with virtually no maintenance, and only ever need to reboot if I do a kernel upgrade (rarely, on a server machine) or get a power failure. (I know....I'm an idiot for not having UPS's on my servers. Well, it's a home network....sue me.)
My Windows machine just got a fresh install of Windows XP on a brand new 120GB drive for a LAN party this past weekend. The install was done the Wednesday before. Three days old.
When I got to the LAN party, it wouldn't boot, as the entire registry was corrupted. One piece of it was actually completely missing.
After an hour and a half of screwing around, doing a recovery install of Windows from my CD, and generally wanting to take the Flak Gun from UT2004 to my system, I finally got it to the point where I could actually play something.
This is on hardware that runs Linux just as well as the rest of my machines.
And don't even get me started about what happens to XP when you install SP1.......
"City hall" in German is "Rathaus" Kinda explains a few things......
I have a feeling that the Microsoft approach to HPC will be significantly different then traditional systems. It is entirely possible that they could create a stripped down operating system, like something you would find in the embedded market, to create drone computers. These drone computers would not have a gui, not have any programs on them and do nothing but be a slave to some proprietary remote execution protocol. Then they would release a "Windows HPC server" which would administer all the drone computers with a GUI interface. They might even be able to get the drones to PXE boot from the server. To integrate with this product there would be HPC.NET with which you could write programs to harness the power of the grid/cluster. It might even be that the HPC system itself is distributed a .NET runtime. Microsoft's approach to HPC will not be what we know as clustering/grid computing today. It will be an integrated Microsoft proprietary system that will be simple to get into and hard to move away from.
The ES-7000 is a machine that is up to 32-way (at the time I was using them about two years ago).
Against our wishes our client purchased one instead of a small cluster. Now aside from the price difference, the stability sucked. We were only running IIS, Apache, SQL Server and Tomcat. The machine needed to be rebooted every day. Yes a million dollar machine running windows needed to be rebooted ever single day.
So my point in relating the story is that MS has a LONG way to go before they are able to really handle supercomputer sized machines. But I do wish them good luck, because they have smart people that will bring some good ideas to the table. On the other hand GO {Linux, FreeBSD, OpenBSD}
Windows is a stupid choice for a headless compute node...
...), I can think of no reason to have an OS that requires a video card (and drivers), and prompts the first time you boot without a pointing device connected, on a system that requires no interface or direct interaction.
...just as Linux is a stupid choice for a home desktop.
On that I will agree; unless there is a very specific reason to use Windows for a cluster (or server or
On this I don't agree. For you perhaps. For me even, in most cases: I run Linux (and FreeBSD) on my servers, and Windows 2000 and XP on my desktops (laptop is dual-boot XP/Fedora). However, there are plenty of good reasons to go with Linux (or BSD) on a desktop system.
I would agree that as a pre-install, or on a desktop for a user who doesn't know Linux (and will be angry that they can't run the latest Windows-based spyware-riddled game) it's not a great choice. But I wouldn't just generalize that "Linux is a stupid choice", because there are times where Linux is a good choice.
I've set more than a couple of desktop users up with Linux -- specifically, unsophisticated people who only need to check email and browse the web, and are using older machines (that I donated in most cases). And in all of these cases, the user really didn't notice any difference (and they constantly ask if the latest virus they heard about on the news affects them).
NGWave - Fast Sound Editor for Windows
Microsoft is addressing the trend of using COTS (commerical of the shelf) products to build large to terrascale computing systems. One of the most popular COTS systems is the x86 based platform because it is cheap, reliable and readily available. MS says: "Hey, we run on x86. Let's see if we can get into this market." And I say come on in.
Lots of people assume the big decision to running a multi-machine computer is choosing the OS, but I don't care what the OS is, once you have 2000 copies of it, it isn't that easy to manage. The current supercomputer market suffers from a lack of quality software.
Mainly this stems from some of the attitudes I've seen in this article (paraphrasing):
"We'll run UNIX because it is inherently remotely administratable": OK, and next kernel release for your UNIX, how do you upgrade 2000 copies of it?
"Ok, we'll run Linux and control our own kernel updates and make it run exactly how we want.": Ok, then once you've customized it beyond anything that anyone else in the world wants, how do you keep it updated to new releases? It isn't that easy anymore.
And... "We'll hire some software developers to hack together a solution, it shouldn't be that hard. Some Perl or Tcl/Tk and we're all set.": Psh, ya. That's what they all think.
Anyway, these trains of thought have lead to each major vendor (SGI, HP/Compaq, Cray, etc) having their own flavor of installers, managing agents, OSs/Kernels, etc, but nothing that really develops into a unified effort. Each one of their machines is fairly unique.
This is a place where Microsoft excells. Because they will basically say, "we don't care how anyone else is doing it, here is how we are going to do it". And if history says anything, they'll make it work, and (eventually) it will work well.
I've worked on many gigaflop to multi-terraflop machines and I can easily say that if someone could pull together the resources to develop a system where I can run and manage a 2000 node system as seamlessly as my desktop, they will get my money. "evil empire" or no.
Sorry final rant: Supercomputers are *hard*. Yes, you can hack together a cluster of linux machines in your basement or school and it works great but, along with lots of other things, success doesn't scale. Don't knock anyones attempts to help the community until you've spent 48 hours at a terminal trying to get a couple thousand nodes to hang together long enough to get actual work done.
I have 2 computer, one is a Dell Inspiron 5150 laptop which I installed Fedora Core 1 on (and is now somewhere around Fedora Core 2 since I've been using the unstable branch for updates). My other computer is a custom built. It has Windows XP on it. So far I've had that one for almost 4 years, every now and then being upgraded. So far I've had to reinstall Windows on it about 4 times since I got my 100 GB hard drive for it (which was around 6 months ago). On my laptop which is also 6 months old I've only reinstalled Fedora once because I wanted to test out Debian but decided I liked Fedora more so I switched back to it.
Recently my copy of XP has started its death phase (after experiencing it so many times I can sense when its close to dieing). I now just get random BSOD for no reason, often it happens when im not doing anything. My Laptop is running flawlessly (right now I'm on it, while ripping the 7th Evangelion DVD in the background).
I would never use my XP computer if it wasn't for games (its rather hard to play FPS and RTS using my laptop's pad mouse, I should get a USB mouse sometimes so it will be easier).
I've been using Linux since last september, and I'm loving everything about it.
M$ will never succeed in HPC because:
- There's no money in it. The HPC market is too small and too expensive for M$ to support. There's not enough thrill in the kill.
- You must customize the kernel. To add H/W CPU perf counter support, remove ACPI interrupts, shrink the kernel's memory footprint, remove/add un/necessary services, etc). You can't do this in a closed kernel like Windows. The M$ solution is to build a monster kernel. That's a no-go for HPC.
- 95% of the HPC community is extremely cost sensitive. They spend as little money on hardware as possible and even less on software. The M$ O/S *starts* at $300 per node, and all its tools are closed, commercial, and so, more expensive (compilers, debuggers, profilers, libraries, etc).
- Few or none of the myriad HPC libraries has ever been compiled on non-Unix O/Ses. Is M$ going to reengineer all of these for Windows? For FREE?
- No HPC support staff or users know Windows. Why should they learn it? Just to please M$?
- All of M$'s fielded HPC systems have been fully underwritten by M$ (Cornell, NCSA, UCSD). This won't happen when *others* have to foot the bill.
Randy
make it so that codes written for unix systems can be ported over essentially transparently - have the same libraries, the same types of compilers, etc. etc.
Surely you jest! Right now I am rewriting a production application that was written under Microsoft VC++ 4.0. Why? Because it will not compile under VC++ 6.0 which is what we have now. Fixing became enough work that I decided on a rewrite. Even if Microsoft was to provide libraries, compilers and tools sufficient to port your *nix applications over, they'd all be changed in a year anyway!