Slashdot Mirror
IBM Runs 41,000 Copies of Linux on Mainframe
An anonymous reader wrote in to send us a story from Bloomberg about IBM making mainframes act like hundreds of servers. The best part is the bit at the end where they mention testing it by running
41,000 copies of Linux. "
From the article, I got the impression that they wanted customers to use their existing mainframes (presumably data warehouses and such) as webservers. At least, that's what I got out of their claims at increasing speed by doing away with webserver-database latency.
Problem with that is, something that takes down one service takes down both of them. I realize mainframes are pretty damn reliable boxes, but if it goes down, do you want it to take your webserver with it?
(I'm assuming the security issues inherent in putting a webserver -- esp. a public one -- directly on one's data warehouse have been hashed out in the course of the VM development. Nonetheless, websites are flypaper for h4x0rz -- that's putting a lot of trust in software.)
Same thing holds for anyone using it to replace 41,000 (yeah, whatever) webservers. One machine fails, 41,000 web servers (and god knows how many sites) out of business. I suppose a redundant mainframe is sufficient insurance -- but how much more appealing is that than buying a comparable number of Suns, and having just a few backup boxes?
Seems like an interesting idea, and it certainly creates options. I don't know if it's the Sun-killer, though; and though it might convince existing users to not buy Sun, I don't know how many new buyers it would attract.
Then again, the closest I got to working on a mainframe was touring a server room with a bunch of AS/400's in it once, so don't think I'm the Delphic Oracle or anything. :)
phil
There are more given in the LinuxPlanet article (which is where I got the other links).
Seems to me that if I had a mainframe, or could get hold of one, this is an awesome virtual hosting environment.
Give me money, and I give you root access to your own, incredibly reliable, Linux box. If you trash it, it can be restored from backups in seconds. Incremental cost of adding another virtual host: almost nil. Until, of course, we get to 41,000. By then I should have enough money to buy a new mainframe. And so on.
--
E_NOSIG
True, they're not going to replace 390 with Linux anytime soon. It's a market expansion thing. Buy a single IBM or 5,000 PC's.. IBM sells more iron, their customers save money. IBM keeps the mainframe market healthy.
That would actually make a good sales slogan for Big Blue. Pan the camera over a virtual jungle of CAT5 and RS232 strewn on raised flooring, and up onto a cluttered wall of dilapidated 2U servers. One unit is smoking and sparking foreground right. Announcer: 'Would you rather have 10,000 chickens'. Screen goes black as the camera zooms out of the black background of the IBM logo on a shiny new mainframe. Announcer: 'Or one bull?' Wrap it up with the standard IBM music and blue bar quick scrolling to stop. 'IBM E-Business: A bull in your corner.'
.sig: Now legally binding!
...one copy for each distro.
*ducks*
Here's my copy of DeCSS. Where's yours?
censorship is a form of noise, which actively seeks to drown out content with silence - Crash Culligan
Hey they tried! But each NT server tied itself up trying to fight the others to be Domain Controllers!
So how do *you* fit 41,000 penguins in a room?
;-)
(sorry for the ad slogan infringement, but it seemed like the right thing to do
-Bugbbq
(to the tune of 99 bottles of beer/..... sorta)
41000 copies of Linux on the box
41000 copies of Linux
if one of those copies should happen to fail
wait in the dark, til the power comes on
(Repeat)
Finally, someone catches on to the good that big iron can do. Mainframes *DO* have a practical use in the modern computing environment. Ever time I hear someone mention the 'Old, vunerable mainframes', I cringe.
They have been doing what VMWare is doing, aka, running virtual machines, for nearly three decades. They know what they're doing.
I'd be interested to know what large ISP is looking to use them in this way. To my knowledge, this would be the first published use of the mainframes specifically to serve as a server-multiplexor (Is that a word?) in an ISP environment. This could be the 'next big thing' for these machines. Either that, or be yet another flash in the pan with alot of 'cool' factor..
(Fingers crossed)
-- I'm the root of all that's evil, but you can call me cookie..
I'm not involved in this work; I wish I was. However, I'm very familiar with VM/ESA and the low-level programming facilities that are being used to pull of this 41,000 Linux virtual machine cluster. I used to write assembly language programs that used IUCV and virtual CTCAs, and what they are doing is crystal clear.
First off, this setup is running under VM/ESA. This is NOT the same operating system as OS/390. Diehard VM'ers tend to view OS/390 about as fondly as Linux users view Windows. OS/390 is the huge, IBM-management-approved operating system with JCL, that evolved out of OS/360. VM is the back-room project that IBM management has tried to kill, over and over, but can't kill, because it's needed for OS/390 development, IBM developers demand it, and many customers demand it. OS/390 is what management wants to sell -- it's the "strategic" operating system. VM/CMS is what the IBM development teams use because it was designed, from the bottom up, by IBM's best software developers, specifically as a platform for software development. Really. I used it for 15 years. If you're developing or debugging IBM assembly code, it's just the best. VM was a skunkworks project, and a damn fine one. It's a shame that it isn't that well known.
The two operating systems should NOT be confused. Different operating systems. Entirely. OS/390 can run as a guest under VM/ESA, but not vice versa.
That said, VM has a HORRIBLE native TCP/IP implementation. It's a big program, written in Pascal, and it's a dog. In fact, it's about the weakest part of VM. It never got much attention, because mainframe networking has always been driven by SNA, VTAM, etc. and IBM development is traditionally done on a 3270 style terminal. All the tools, XEDIT, the mail system, etc, are all designed for 3270, block mode terminals. VM is lacking in TCP/IP support for the same reason that Unix systems are lacking in SNA support, because no one wanted it. This is changing.
The VM TCP/IP implementation is a standalone program. The TCP/IP program runs in its own virtual machine. When someone wants to connect to TCP/IP, they use a system call to establish a connection between their virtual machine and the TCP/IP virtual machine using a facility called IUCV -- Inter User Communication Vehicle.
IUCV is a very fast, block-oriented, secure, unspoofable point to point protocol for establishing data links between virtual machines. A programmer using IUCV starts by creating a link to the target, then sends blocked data by making a system call with the address/length of the data. The CP nucleus (their word for the kernel) copies the data into the system address space, synthesises and schedules an interrupt for the target virtual machine, and immediately reschedules the source virtual machine. The target virtual machine receives the interrupt, issues an IUCV receive system call, and CP copies the data into the target machine address space. This is all done completely asynchronously. It's extremely fast, and utterly secure. Zero-latency networking is a nice thing to have.
Which leads to something very cool. IP over IUCV.
I don't know exactly how they set their system up, but here are the basic tools that they have to work with:
1) TCP/IP to the outside world can be handled in at least two ways:
o Through a native Linux network device driver. In VM, physical peripheral devices are assigned to individual virtual machines. A virtual machine with a physical network interface attached to it simply uses it as an ordinary I/O device.
o Via a connection to a native TCP/IP virtual machine, using a special device driver that knows the native IP-via-IUCV protocol.
2) Connections between virtual Linux machines can be handled in a couple of ways:
o through a virtual (or real) CTCA (Channel to Channel adapter.) a CTCA is a high speed parallel interface used to connect mainframes together, point to point, very fast. If you use virtual CTCAs, you can move Linux images from one machine to another without having to ever reconfigure anything within the Linux images themselves simply by replacing the virtual CTCAs with an attached real CTCA and changing the directory entry of the virtual machine.
o Using an IUCV driver, one can interconnect all of the internal Linux images via virtual point-to-point lines. This is much faster then virtual CTCAs. The drawback being that you need to configure IUCV links within a virtual machine, so changing things around requires reconfiguration within the linux image itself, and IUCV is designed to work efficiently within a single system, not across multiple systems. It can be done, but it's a hack, and it's inefficient.
o Through an obsolete API called VMCF, which was superceded by IUCV.
The big innovation going on here is the realization that by running multiple Linux images on a single machine, or multiple Linux images on multiple machines, using mostly IUCV links, one can almost eliminate the network latency, because the data transfers are simply memory copies, and one can eliminate the network collision problem, and the network traffic problems. If you have 100 machines sitting on a fast ethernet, and you start getting a lot of inter-machine traffic, you are going to have collisions, and each machine has to waste a fair amount of time evaluating which packets are his. This removes the biggest bottleneck in large clusters of small machines, Also, an IUCV connection is guaranteed to never drop a packet, and always transmits packets in order, so TCP over IUCV proceeds smoothly and efficiently.
This gives you lots of scaling options for your virtual Linux network.
One more point.
There was an article that came out two days ago, but due to a slashdot bug never appeared on the main page, but proceeded directly to the "older news" catagory. In it, the author wrote:
An S/390 running a light load will not run as quickly as a fast PC server under a light load, according to Courtney. The difference between the two systems will not be apparent until the load is much larger.
"The PC will begin to degrade and will typically reach a point where it avalanches down in performance as its load limit is exceeded. The mainframe starts out at a lower performance level, from the standpoint of an individual program task, but degrades much more slowly and much more linearly as the load increases," he says.
Revisiting my previous comment in this thread, I remember, a while ago, reading in another article about a difference of opinion between some IBM programmers and the kernel maintainers. Supposedly, IBM was complaining that Linux performance went south when the number of running tasks became large, and proposed some scheduler changes, but the kernel developers didn't want to change it because the changes would have slowed the kernel down in the "normal" case of only a few active processes. Does anyone have a link to this or remember what I'm talking about?
Sounds like this article is describing the same known effect. However, by running multiple images of Linux under VM, one obtains a workaround for the problem. If a Linux virtual machine is overloaded, create a new virtual machine image, and offload one or more of the biggest processes to run on the new machine.
This is all very interesting stuff. Don't forget, the stuff we're just discovering now in the Linux world, is largely stuff that the IBMers, and especially the VMers have been working on and perfecting for about 30 years. I'd love to see a Linux kernel that can run 41,000 tasks, with a linear performance degradation curve. Until then, at least there is a way to run Linux on an operating system that has those characteristics.
And the fact that their operating system can run 41000+ simultaneous tasks without disintegrating, but ours can't, should eventually get under someone's skin and prompt efforts to make the Linux kernel scale better under heavy multitasking loads. Why should they have all the fun?
- John
Mainframes are usually tuned for block device I/O and are notorious for being crappy at TCP/IP
Spent some time working on IBM's TCP/IP stack. You're talking from the past. The mainframes stack used to be single threaded and very slow. As a workaround, IBM hacked it so that you could run several stacks on one image (this caused its own 'stack' of problems of course).
The Release 6 was a complete rewrite of the TCP/IP stack. They used it to set record industry benmarks when it reached the 1.5 Billion pages per day mark about two years ago. They gave us all nice denim shirts with the embroidered slogon "1.5 Billion served". I wouldn't call that crappy, would you?
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
They considered running NT servers, but they didn't have the $8,000,000 for the licenses and the 4 TB of disk space. :)