Slashdot Mirror
IBM Saves $250M Running Linux On Mainframes
coondoggie writes "Today IBM will announce it is consolidating nearly 4,000 small computer servers in six locations onto about 30 refrigerator-sized mainframes running Linux, saving $250 million in the process. The 4,000 replaced servers will be recycled by IBM Global Asset Recovery Services. The six data centers currently take up over 8 million square feet, or the size of nearly 140 football fields."
This proves Linux has a smaller carbon footprint then other OS's!
Because they're using all that Microsoft IP without paying for it....
(it's a joke)
Do you have ESP?
for AIX on those mainframes! After all, AIX has more Unix IP than Linux, isn't it?
If you keep throwing chairs, one day you'll break windows....
We (Bigattichouse's Vectorspace Database) went through their Linux certification (as well as Grid cert), and they were a pleasure to work with - providing expert advice and patience in every step of the process. Not exactly on topic, I guess, but I thought I'd share. They really seem to embrace the engineering and spirit of Linux.
meh
The article says that the data centers required for the 4000 "small computer servers" aggregate to about 8 million square feet. It takes IBM 2000 square feet to house a small computer? Also, saving $250 million suggests that it costs them something over $60K per "small computer" even ignoring the price of the new mainframes. Amazing.
My employer recently 'consolidated' their server farm too. We used to have a room with fifty aging Dell PowerEdge servers, each running independently, requiring massive support, cooling, and electricity.
Now we have ten VM servers running all the migrated services, PLUS a room with about fifty aging Dell PowerEdge servers, each running independently, requiring massive support, cooling, and electricity.
I never thought 'consolidation' would require so much more space, electricity, air conditioning, and upgrades to core switches and UPS units.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
When they came for the communists, I said "He's next door. Take him away. Goddam commies."
Lets see... $250M / 4000 = $62,500 per server being consolidated? I mean, I know floor space, buildings, racks, power, AC etc cost money... but that's still a *lot*. Anyone care to chime in on how close to normal that is?
The story here is about consolidation, virtualization, etc.
Linux is a small part of the technology involved here. z/OS is the real story here.
Am I part of the core demographic for Swedish Fish?
Part of that is because IBM will customize the machines to your heart's content. The sky and your budget are the only limits. They leave a good many of the loadout details (xGB/TB of RAM, DASD storage size, # of CPUs per card, # of CPU cards, even number of mainframes - they can be chained in parallel). You should look at the Z series hardware specs for the general details and look up what details you don't know.
If you're looking for benchmarks or comparisons to x86/x86-64 or other commodity architectures good luck - they are nearly impossible to find. This is due to the implementations being on entirely different scales. The best comparison you an find is the MIPS per CPU. You can find some slightly stale numbers here (BTW: an LPAR is something that's been around on mainframes for several decades - one LPAR can run up to several hundred x86 VMs concurrently).
US Democracy:The best person for the job (among These pre-selected choices...)
If you take hundreds of cabs and consolidate them down to 40 (with the associated consolidated storage) you are going to save millions. That has little to do with Linux. It is the modern mainframe that makes this kind of thing possible, which is why more people are moving to them. They must have a lot of servers spinning idle to get this done.
The reason why companies are in this pickle is because they thought more was better. They though "All we need to do is buy 4000 x86 servers and we can do tons of work." They didn't realize how HARD it is to get 4000 servers to operate in a cluster so you can take advantage of those individual systems as one body. So, they ended up with islands of computing power instead of a cluster. Naturally the mainframe consolidates these islands back to computing continents and you end up running the mainframes at near capacity all the time. Modern mainframes make this easy with dynamic CPU/RAM allocation, as well as dynamic storage. So you segment out the mainframe in to four or eight chunks. Chunk 1 is hot, chunk 3 and 5 are idle. Simply re-assign some of the CPUs from chunk 3 and 5 to 1 until the load goes down. You can take advantage of this in a big way if you segment your work load to match global demand. So chunk 1 might be data for the western USA, and chunk 7 might be EMEA. You can bounce resources between those segments much more easily. You can even script it. HP has an offering that does this automagically, I'm sure IBM has something similar.
Now, my personal opinion is why Linux? Some of the more advanced features like dynamic RAM, CPU, and IO allocation don't appear to be that solid to me. Perhaps IBM added these features to Linux or made them more robust? Maybe they run Linux inside an AIX virtualization container?
I had to maintain some software that was running on a aging 370 mainframe. The 370 was emulating a 360 which was emulating a 1401.
It was pension and payroll software and it was legally blessed.
It was such a frigging song and dance trying to get anything done that it was cheaper and faster for the company to emulate their butts off rather than trying to go through the management and the unions and the employees.
But I did learn about optimizing instruction fetches by scattering the compiled code around the circumference of a magnetic drum so that the drum would have rotated around beneath the read head in time for the next instruction.
Try and tell that to the young people of today, and they wont believe you, eh Obadiah?
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
These are machines that don't break, period. We're talking the types of machines that run the major banking systems of the world and the like. They simply do not go down. In this situation, if one of the 133 apps buggers up, it's only that VM that's shot. You just nuke it and restart it, the rest of the machine just keeps ticking along.
...si hoc legere nimium eruditionis habes...
Where I work we currently run two mainframes in a sysplex environment for all the core transactions. It's a very optimized environment and handles millions of financial transactions a day. In mid-2006, IBM started giving us zLinux engines to "try out" and they gave us all of the software we needed to make a go of it. Kind of like a playground drug dealer, they hoped that by giving us a bit for free we'd get hooked and become dedicated customers. The problem was, for the type of workload that typically runs on our servers (high CPU, moderate I/O) we were experiencing poor performance on the mainframe VMs. IBM sent all their engineers out to help make tweaks and tune all sorts of things. Despite all the tuning and tweaking that took place, we could never get a single engine to perform better than a $5,000 server. Keep in mind that a single engine was retailing for around $80,000 after discounts.
We did some calculations and determined that for the price of a zLinux engine we could buy an entire rack of high-end HP servers that would outperform the single engine by a factor of 200:1. Again, maybe it was just the workload we were doing, but even IBM couldn't figure it out and our server work profile isn't exactly uncommon. Granted you can cram a lot of guests onto a host system provided that none of the guests want to use more than 10% of their CPU at any given time, but that defeats the purpose. I could probably run a VMWare host with 100 guests and call it a success, provided they all sat idle.
It was kind of funny because the IBM engineers would shake their heads and admit that for our workload it just wasn't going to work out. Then the next week the sales guy would call and ask if we were ready to buy that third mainframe since he just read the engineer's report and our visit was obviously a smashing success.
I'm not knocking the whole Linux on the mainframe concept, I'm just sharing our experience and how the whole thing seemed to be like someone in IBM Marketing declared "we need to sell Linux on the mainframe" and the Dilberts were forced to sell a product that worked about as well as a chocolate fireguard. It was a very awkward experience and even the IBM engineers seemed like they were stuck in an uncomfortable position of supporting sales for a product that even moderately demanding customers wouldn't be able to run with.
Personally I consider Linux on the mainframe to be on par with running Linux on an iPhone. Sure you probably can, but does it actually do anything uniquely useful for the business? I have a hard time selling technology to the CIO on the grounds that because it's Linux it's a good business decision regardless of the context.
Not quite. They are engineered (as they have been for decades) for stability and were designed to handle that kind of load. Its CPU/RAM/storage are redundant, so that if something in the system goes down, new resources are allocated. Additionally, shops will have multiple mainframes just for that kind of redundancy. Its kind of like saying your car is a "single point of failure" - sure, it is, but they were engineered for the purpose of being reliable.
Because after OS/2 died IBM didn't want to get fucked by Microsoft again and Apple didn't have a business solution. Creating another ground up OS was definitely not a solution so they were left with two options: Linux and Unix. The only reason Linux is run on as many servers as it is today is because of IBM. They do Linux R&D, produce software, and provide B2B services for the OS. Linux is the official OS of IBM and without their support Microsoft would currently have a monopoly on server software. So, a simple answer to your question would be that IBM has a financial interest in Linux. But it goes deeper than that: IBM has committed to Linux - it's a marriage.
I think you're hesitant to accept IBM because of the whole 70's/80's "Big Blue" stuff, but after Microsoft swept the rug out from under their feet the company's strength was permanently compromised. The consumer market rejected them (hence the sale of the PC division to Lenovo) and until they committed to Linux software was a major vulnerability for them. The openess of Linux enabled them to get back in the game - their customers didn't have to worry about the future of the platform while their immense contributions to Linux enabled the OS to really threaten Microsoft. So yeah - as a Slashdotter, IBM are the good guys. They support Linux and they don't aggressively protect their many, many patents (they use their patents to protect themselves rather than trying to sue everyone they can for $$$). Personally, I think IBM is the most important tech company in the world.
I still haven't seen any conclusive evidence that Linux on mainframe is a good idea. I'm sure running 30 new mainframes is going to cost less than 4000 aging servers. Just about anything would be less expensive than 4000 aging servers.
But I bet that a small farm of modern medium sized servers running Linux on VMWare would be even less expensive. Or Solaris/Niagara. Why would you want to run an open source operating system, whose major benefits are openness and affordability on the what is literally the most expensive and most proprietary computing platform in the world!
These server consolidation projects are just giant boondoggles spawned because the server sprawl finally got insane. It's an endless cycle:
A. Giant server consolidation project that takes 4000 servers down to 30 servers.
B. Department B complains that Department A's application keeps hanging and consuming all of the CPU. They demand their own hardware "for availability reasons".
C. Vendor C demands dedicated hardware for licensing/capacity planning/supportability reasons. Rather than constantly bicker with the vendor over supportability they get dedicated hardware.
D. Department D complains that the IT department is charging outrageous prices for time sharing on the mainframe. After all a dedicated server only costs $XXX.
E. Suddenly there are 4000 servers again.
F. IT department spends some insane amount of money on infrastructure to manage the 4000 servers.
G. IT department budget gets insanely large trying to manage that much stuff.
H. Some CIO gets the idea that all of this money managing servers is ridiculous and we should do a server consolidation project.
I. IT department spends an even larger amount of money on the latest super high availability gear and consulting services so that the can run 4000 commodity servers inside a few big servers. All because it will "cost less to maintain".
J. Go back to A.
Before, .33% failure rate = 13 failures a day. You had well understood procedures for dealing with failures.
.33% failure rate = 1 failure per thousand days. This is a recipe for hell.
After,
But wait...
When you do have one machine fail- it takes down 133 virtual servers at the same time. You raised your risk enormously.
IBM will tell you all about fail-over just like they did our executives.
Half the country down for three days is the reality.
---
Still it is interesting to see a return to the centralized mainframe farm. Sure hope those multiply redundant communication lines don't go down.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
Last week, I attended a presentation at IBM's Australian Development Lab in West Perth, where a lot of the z/OS-related code is maintained and developed.
From what we were told, IBM z/OS mainframes are the *most* reliable platform to host software services (but of course, they'd say that).
The following is from memory, as best as I can remember it, and may not be 100% accurate:
The 'z' in 'z/OS' stands for 'zero downtime'. z System mainframes are engineered for 99.999% availability, or less than 3 minutes of downtime a year (we were actually quoted 'less than 5 minutes', but (1 - 99.999%) * 365.25 * 12 * 60 = 2.63). Apparently, they quite easily meet this requirement - we were told that it is not uncommon for systems to remain online for 10 years or more without failing.
Up to 32 z System mainframes can be clustered in a 'sysplex'. Each mainframe is divided into several LPARs (Logical Partitions), each which can host several VMs. If an application fails, the automated recovery service will attempt to restart it, either on the same VM, a different VM, a different LPAR or a different mainframe in the sysplex, as appropriate in the situation. It is also possible to host a redundant sysplex in a different site, which mirrors data and which the primary sysplex can failover to in the case of failure.
IBM mainframes are used in many major corporations around the world, particularly those where the cost of downtime is very high (think thousands of dollars a second).
I work in a longtime "blue shop" perspective, not always from a software/OS perspective. While I like the 'concept' of running linux on zSeries, I think you could take a look at the requirements and choose a platform that can run the same Apps.
For example, for email we run Lotus Notes on a couple of BIG pSeries (AIX) servers. We could have run it on a farm (technical term) of windows boxes.
For webservers, which you could run on AIX, or linux on zSeries. We have multiple (read: many) x86 servers running linux+apache. Why? They connect to a backend app server (pSeries) which connects to a backend zSeries DB2 (I'd prefer Oracle however, to run Oracle on zSeries requires it to be run in a linux VM).
We definitely subscribe to the school of using VMs whether they are zSeries, pSeries, or VMWare on x86. Even if the x86 server is running ONE application, we still put vmware underneath, as it allows for us to move the image to a newer hardware platform when it's time to upgrade. Even some of the larger x86 servers run vmware but in each partition there is a single instance of apache. Makes for managing storage that much easier (fewer zones, cabling etc).
Would I consider moving our apache on linux on x86 to apache on linux on zSeries? Not really. It's a waste of CPU cycles (MIPS). I'd rather use zSeries MIPS for something a bit more critical like keeping my database up and running than serving out webpages (static or dynamic).
IT isn't not about religion, it's about finding the best tool considering your requirements. I have no problems telling IBM that product XYZ is trash. While my servers are IBM, you won't see IBM disk, or IBM tape, and atleast once a quarter some salesman from IBMs storage group is at my door. He buys me lunch and every quarter he is sent packing. You won't see ibm bladecenters as the thought of hundreds of additional servers to manage isn't appealing (but I'll gladly take 100s of VMs across larger x86/pseries boxes).
I know many of you were expecting to hear me say 5000 linux servers, but there are options for my requirements that did not lead to big "google style" linux farms.
BTW: I have no problems kicking out IBM on x86 if HP/Dell/Sun have a better product, and knowing this and letting IBM know this gives me a great advantage over them, as they very well know I'm capable of bringing in something more suitable. (I *used* to have IBM storage).
Yes, 4000 "small computer servers" times 2000 square feet equals 8 million square feet. But this is unlikely the arrangement. Consider instead a few buildings of data centers, each with 1 or more relatively small rooms. Within a room, there may be a few racks, all surrounded by walking space, and other perhipherals like AC units. Then outside of those rooms, more walking space for hallways. When you factor in all the human space and simple space for ventilation, and then cubicles and monitoring for support personnel it could average around 2000 square feet (40x50).
The same logic can be applied to costs.. $250 million / 4000 machines = $62.5K. Some of that is actual hardware, and software licenses. Some of that is ongoing support from their full time employees on staff to maintain the things.
There are 10 types of cliches in this world. Those that are new, and those that aren't.
How is that different from the hurricane ripping the roof off of the data center and hurling 400 blades into the next county? Or as we saw in SF, a power outage taking out an entire data center?
Any sect, cult, or religion will legislate its creed into law if it acquires the political power to do so.
Yeah, if somebody hits the Big Red Switch, there's going to be a problem. But, if they don't, well, it's a mainframe.
The Linux on these machines is running under z/VM, in multiple virtual machines. When one of them has a software fault, you reboot that one's VM and keep going; the other 132 Linux-running VMs run without noticing anything happened. (It is possible for z/VM to fault, sure. But it's an OS with 40 years of refinement in the "100% uptime" mainframe culture, and its task is just managing the virtual machines.) When something goes wrong with the hardware, the fault tolerance and self-healing features keep things running, and you fix the faulty element with a hot-swap. A properly set-up datacenter is going to minimize external risks, with backup power and such. Proper choice of datacenter location will minimize natural disaster risk.
So, yeah, the big risk is human failure, and these IBM-built, IBM-owned datacenters are presumably going to have extensively trained IBM-employed mainframe personnel, which minimizes that risk.
Now, if some cable company cuts the fiber optic lines . . .
There are a lot of errors in your comments, unfortunately. Of course you can run Red Hat and SuSE concurrently in a single LPAR under z/VM, and multiple versions thereof. This has always been true, ever since Linux began running on mainframes many years ago. You might want to have more than one LPAR to run more than one version of (first level) z/VM, but you don't need many. Two or three for z/VM and Linux is typical and just fine. And it's not as if LPARs are in short supply on mainframes: up to 60 are available on a single machine (30 on the smaller model), so "spending" 1 to 3 is no big deal.
Re: Investing in new mainframes, come on, get real. It's so easy to find market data because companies like Gartner and IDC publish it, and IBM just announced its 8th straight quarter of mainframe hardware growth, something that hasn't happened since before Y2K. It's impossible to do that with "a few showboat customers."
And no, you simply cannot approach the level of virtualization these machines offer on any other system, at least for typical business computing, and still offer reliable service to users. In fact, in IBM's case many of the software licenses are presumably "free," and they still found big cost savings by taking 4,000 machines down to 30. For the rest of the world the mathematics in such situations are even more compelling.
I was involved in a migration to the zOS architecture three years ago. I am currently involved in a similar exercise for a British telecoms company whose name escapes me. In both cases the principle was perfectly sound, but the reality rapidly starts to come down to what can be migrated, when, and why. At IBM application compatibility was a major consideration, and ultimately prevented key parts of the system from being migrated. At the current site, surprise surprise, the problems are the same, plus reluctance to do the work (upgrades, work required on the client's part, age of applications and Plain Old Politics). I wish IBM good luck, and perhaps because there is a better integration of operations and systems they might succeed, but I would be willing to bet that by the end of the process, they will have reached about 80% of their target.
Actually, on a System z9 EC (Enterprise Class), a single CPU chip failure is not a "Call Home" repair event. Only the second CPU chip failure would result in an automatic call, while your business keeps running of course. (There are a minimum of two spares in each machine.) The average time to first failure for a particular machine is somewhere in the many decades range.
OK, just for fun (because it never actually happens in the real world), what happens with a triple failure? If you happen to have a "fully configured" mainframe -- all processors turned on -- then.... your business still keeps running. Yes, the system might lose some processing capacity, but it keeps running. The higher priority stuff (from a business view) takes precedence automatically, and life goes on. This is all on a single machine still.
If you've got an S18, S28, S38, or S54 model, then, at your business's convenience, the faulty hardware can be replaced. (You might do this at night, for example.) The repair technician tells the mainframe to "evacuate" memory on a portion of the machine while the OS and applications keep chugging along, possibly with reduced capacity, often not. (Depends on what configuration you choose.) When the evacuation is complete, the technician can pull a processor/memory group (called a "book"), insert the new one, bring the new one online, and... everything still keeps running. Again, this is all on a single machine -- no clusters required for any of this.