Slashdot Mirror
Server Runs Continuously For 24 Years (computerworld.com)
In 1993 a Stratus server was booted up by an IT application architect -- and it's still running.
An anonymous reader writes:
"It never shut down on its own because of a fault it couldn't handle," says Phil Hogan, who's maintained the server for 24 years. That's what happens when you include redundant components. "Over the years, disk drives, power supplies and some other components have been replaced but Hogan estimates that close to 80% of the system is original," according to Computerworld.
There's no service contract -- he maintains the server with third-party vendors rather than going back to the manufacturer, who says they "probably" still have the parts in stock. And while he believes the server's proprietary operating system hasn't been updated in 15 years, Hogan says "It's been extremely stable."
The server will finally be retired in April, and while the manufacturer says there's some more Stratus servers that have been running for at least 20 years -- this one seems to be the oldest.
There's no service contract -- he maintains the server with third-party vendors rather than going back to the manufacturer, who says they "probably" still have the parts in stock. And while he believes the server's proprietary operating system hasn't been updated in 15 years, Hogan says "It's been extremely stable."
The server will finally be retired in April, and while the manufacturer says there's some more Stratus servers that have been running for at least 20 years -- this one seems to be the oldest.
"It never shut down on its own because of a fault it couldn't handle," said Hogan. "I can't even think of an instance where we had an unplanned shutdown," he said.
This isn't a server that has had an OS uptime of 24 years. This is a computer that they are still using after 24 years that "hasn't crashed". So what. The Amiga still being used from the 80s was a bigger deal. This article is really just an ad for Stratus.
'"I can't even think of an instance where we had an unplanned shutdown," he said.'
Um... I should hope so, since if it had, it wouldn't have had 24 years of uptime. And no photo of the output of "uptime"? I'm starting to think that they DID shut it down/reboot it many times, but somehow ignore this in the "uptime". Nonsensical article.
it DID NOT run continuously for 24 years. It simply never stopped or restarted without admin intervention, two very very different things.While still impressive it is no where near as impressive as if it had run 24 years continuously.
From crowd-funded asteroid mining? Privately? No? Then this story is for LUDDITES.
English has a much better word that would not result in admins thinking this headline meant uptime.
Not quite the same as 24 year uptime. In the same vein, I have a Sun server that is still running since the mid-90's, part of a medical device and used to compile very particular software code for an old small-bore MRI system. We shut it down when the power goes out (very rare), but it's SCSI drives are still good.
Custom electronics and digital signage for your business: www.evcircuits.com
IT application architect meet sanitation engineer.
And no, those were the model numbers, not the CPU, which was the M68 series.
About the only thing non-redundant was the clock card. Voice of Experience. The power supplies had built in UPS's. Funny thing on the 808X systems, the power switch had "Off", "On", and past "On" was another state, which I forget what it was called. But if you replaced hardware while running, you'd push it up (it was spring loaded) to get it to IPL the new hardware.
I loved it because you could fold up 24 physical processors into 12, 6 or 4 logical with quorum voting. Get a bad CPU? It wouldn't miss a clock cycle, it's just lock it out and keep going. You could also run it completely unfolded.
These days, folks would say "so what?" - but "back in the day", your PC had a single core. It was a big deal. And even today, if you get a Check CPU, the system crashes on a PC.
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
This article is really just an ad for Stratus.
Or Phil Hogan's résumé
“He’s not deformed, he’s just drunk!”
"Over the years, disk drives, power supplies and some other components have been replaced but Hogan estimates that close to 80% of the system is original," according to Computerworld.
Then is it still considered the same server? https://en.wikipedia.org/wiki/...
Personally, I have a computer that lives in a case I got in 2003. I am on motherboard #4, power supply #2, processor #2, memory modules #6 & #7, hard drives #4 & #5, etc. However, I still consider it to be the same computer. Perhaps there is something psychological about it, but the name (or in this case the case) has a special significance even if all the guts have been swapped out.
Yes, we would be truly impressed by a server that runs continuously for 24 years. That would be extremely impressive. My nerdgasm would be epic at this point. But unfortunately the way the article is worded doesnt actually confirm it.
Definitely not Hogan's resume.
According to his boss Wilhelm Klink, no one has ever successfully left the company.
Stratus has proprietary redundant *everything* on their machines, and runs in lockstep; they literally have two of everything in there... two motherboards, two cpus, two sets of RAM, etc. If anything weird happens on one side, they fail over to the other motherboard running in lockstep on the other blade in the chassis. Combine that with running an extremely conservative set of drivers that are known stable, and you can get six nines out of the thing. Stratus is typically used for credit card processing and banking applications where it's not ever acceptable to have a machine down for the time it takes to reboot. Really, really, really expensive though. You wouldn't want to use one of these for anything normal.
Even though the system has a character-driven interface, similar to an old green screen system, the users "like the reliability of it, and the screens are actually pretty simple," said Hogan.
Is there any other way to run a serious server?
Escher was the first MC and Giger invented the HR department.
I used to work on Stratus servers, and I think the company was purchased by IBM in the late 90s.
For each running component in the system, there are three physical instances. They use a voting system to drop any disagreement in RAM or the outcome of an instruction. In the 3 years I dealt with them, I never saw a system failure, and the only outages were caused by planned system upgrades. OS stuff. All of the hardware was hit swapped.
These were multimillion dollar machines that basically had the CPU performance of a couple of 68000 CPUs.
I personally witnessed a take out of a Novell 2.x file server which had a 16 year uptime. This was for a school system, and they had forgotten where the file server was. Stuffed in the back of a janitorial closet, and dust covered. That wasn't any sort of fancy hardware.m, but an old microchannel PC.
If you don't need to patch them, computers run for a long time.
Definitely not Hogan's resume.
According to his boss Wilhelm Klink, no one has ever successfully left the company.
I get your joke, but you just confused a large number of young people. Who all need to get off my lawn.
Stratus are an old school redundant parallel architecture. You can take a node off line without taking the system down. Beyond that multiple levels of redundancy with components. Portions of the system have certainly been taken down, but the system as a whole kept running.
No one would consider that kind of architecture now; much too expensive, when other solutions are available now. The key word in the previous sentence is "now". Probably not an ad for Stratus, they don't really exist anymore.
The equivalent now is a server farm. There are systems (server farms) that have been running for over a decade.
Windows 95 had a bug that made it crash when the uptime hit 2^32 miliseconds, or 49.7 days. Since Windows usually crashed much sooner anyway, it took Microsoft years to notice that bug.
Not proprietary, though netware is no longer supported, there's not a lot to go wrong, and some boxes had epic uptimes, as in never died, never rebooted. We had one that the only reason it completely went down was a catastrophic power loss (both PDUs lost power at the same time) . Its uptime was over a decade with over 50 users still accessing every day. All that being said, anything that's still running 24 years after initial boot is impressive and worthy of note. NOTHING running windows would have done that. Perhaps something running on an IBM "z" series could. Given that my IT career began in 91, I'm lucky to be running this long without a reboot.
~corporate tool, but employed~
My A3000 ca. 1991 is still running, and appears to be more reliable and less energy sucking than my 23-years-younger Dell desktop at work. Maybe not so powerful, but way more friendly. Still does email while doing web while c-compiling while rebol and arexx scripting while playing crazy euro MOD files while animating in DPaint while etc. etc. etc. It does pretty much exactly what I ask it to do, nothing more. (And yes, if you don't load Workbench, Amiga is a shell-only system.. until you open a GUI app, that is)
People of a certain age never saw Hogan's Heroes and know Colonel Klink from the Simpsons instead, where Klink appeared as Homer's guardian angel.
There was an article here about a Netware server that *actually* ran for 16 years.
https://hardware.slashdot.org/story/13/03/31/1230255/netware-312-server-taken-down-after-16-years-of-continuous-duty
I remember there was a school somewhere that built a parking garage and covered up the netware server with poured concrete and they couldn't find it but they could still ping it on the network.
I built a webserver on a PowerMac 7200 in 1996 and the machine's been running 24/7/365 since (barring power outages longer than the UPS battery, etc). Not a single component has been replaced, the OS (System 9.2.1) never updated, the software (WebSTAR) only patched until the company went out of business. I'd be willing to bet that there's a lot of servers like this still floating around universities and school districts...
In the late 80's I worked night admin shift on a DEC cluster that had an up-time of over a decade, I would be not be surprised if it made 25 years
>There's no service contract -- he maintains the server with third-party vendors rather than going back to the manufacturer, who says they "probably" still have the parts in stock. And while he believes the server's proprietary operating system hasn't been updated in 15 years
If the OS isnt being patched it shouldn't be used for anything more then some hobbyist system or in a museum. Whoever uses something that cant be updated in production for 15 years after its EOSL should be fired.
Don't mainframes kinda do the same thing today? I know they're not exactly mainstream but not everyhing is well suited to being done by a farm, where you really need serialization and global consistency. If you're Facebook or Google the page doesn't have to perfectly reflect changes someone did 0.01 second ago. If you're doing bank transactions or booking tickets then you really need to know if there's still money in the account or the seat is still free. NoSQL is great if you don't need all the guarantees of ACID SQL. Sadly some people think it's the "next gen" and can replace everything relational databases does today.
Live today, because you never know what tomorrow brings
Did you know Hogan hid a radio in the coffee machine?
...a Beowulf cluster of them?
Why, oh why, Delilah?
Definitely not Hogan's resume.
According to his boss Wilhelm Klink, no one has ever successfully left the company.
alive.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
Old-school pre-1990s telephone switches - you know, those nearly-building-sized things that kept thousands or tens of thousands of phones in a city working - had uptimes measured in decades.
Short of either a scheduled replacement or a physical disaster, they kept running and running and running.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
The server's proprietary operating system hasn't been updated in 15 years
This is a sign of a shitty sysadmin.
Don't try to "add to the joke" if you don't know what you're talking about.
On the show, the POWs were able to come & go as they pleased due to Klink's incompetence.
Yeah the system in the article has been down for maintenance for various reasons. It hasn't CRASHED since it was put into service. A computer that doesn't crash? That's impressive - if it runs Windows. I've been running servers since the mid 1990s and I'd say MOST of them have never crashed.
There may be a conflation of hardware and software going on here. Perhaps the discrepancy is just semantic. If it was continuously powered on and operating for 24 consecutive years then the *hardware* had 24 years of up time. If it was maintained while hot, as fault tolerant computers of this class and era were meant to be, that doesn't make it a different computer.
If I break my hip and get it replaced I'm still me so why should it be any different for a server? Hell, given the anesthetic, it could be said that I had to be rebooted while the server never missed a beat in its workload.
When I replace the perishable tires on my car its model year doesn't change. I consider components like spinning media to be the equivalent of tires - replacing them is an expected part of the things life process. If it got done without impacting the machine's operation then it stayed up. Wish I could do than on my car.
We have a VAX in our data center. Hasn't been rebooted in probably 30 years now. Honestly, none of us know if it's still used. There was one guy, Dirt Ol' Ike, that used to know something about it. He taught me about VMS and anal sex but he retired almost 20 years ago now. Sometimes when I remember the time we had, I log into the VMS machine.
C'mon! How you do replace power supplies and not have to reboot? One of them old fancy computers or something?
..of over 16 years. Including physical relocations during operation. Some less public installations ran longer.
This isn't a server that has had an OS uptime of 24 years. This is a computer that they are still using after 24 years that "hasn't crashed".
Not even that. My understanding is, when Stratus fails over processors it is just a quiet reboot. Didn't turn off the power, yay.
In the good old days the mainframe boys would hot-swap mainframes by running the new processor in lock-step with the old one, even across vendors (It is rumoured that Amdahl made some sales this way.) The Voyager computers have been running for 40 years.
When all you have is a hammer, every problem starts to look like a thumb.
IBM was a reseller for Stratus but Stratus is still an independent company.
When all you have is a hammer, every problem starts to look like a thumb.
You do realize that Hogan's Heroes is still on TV, right? The joke probably makes far more sense to youngsters now than it did a couple decades back when there weren't so many TV channels dedicated to old TV series.
OK, digging into that... Stratus is doing some weird and cool stuff. Not running processors in cycle-for-cycle lockstep like the mainframe guys (at least, not with their x86 offerings where Intel would never permit the level of systems integration that would be required) but at the memory access level instead, as in, if two processors are running the some code on the same data, then they must access memory with the same pattern. Hard to see how that could be made to work without some kind of hypervisor, which they most probably use. Cache effects would be a nightmare.
When all you have is a hammer, every problem starts to look like a thumb.
I had a NT 3.51 Server that had an uptime of 8+ years. I was on a project to convert all of the machines from Token Ring to ethernet and saved this one to the last just because I couldn't bare to be the one to take it down. The NT 4.0 servers we had we were lucky to have an uptime of 30 days.
Sounds like Tandem. My company still has one of those things. Redundant everything, I always joke that you could physically shoot the computer and it would stay running!
Stable means unchanged. If it hasn't been updated in 15 years, of course it's stable.
Contribute to civilization: ari.aynrand.org/donate
Dude, I'm only just 30 and reruns have been on since I was as young as I can recall!
"an old microchannel PC" - so relatively fancy in fact. The quality and reliability of IBM's Micro Channel machines (and their small number of licensees) was a notch or two above the typical AT clones of the time. In particular they were designed with some attention to airflow and cooling, rather than just a box with a fan in it, so would be more likely to survive a dust-covered existence.
-- Ed Avis ed@membled.com
Yeah, if you wanna believe that nonsense. Ask Stratus about the single flood sensor
on the floor of the server, if you have the stomach for it.
CAP == 'churned'
I personally witnessed a take out of a Novell 2.x file server which had a 16 year uptime. This was for a school system, and they had forgotten where the file server was. Stuffed in the back of a janitorial closet, and dust covered.
Was there a sign on the door saying "Beware of the Leopard?"
How do I downvote this article?
I used to work at Stratus. Most components were duplicated (disks, IO boards, power supplies). Everything was hot swap. CPU's were duplicated *pairs*. Each pair ran in lockstep, instruction by instruction, and results were compared within a pair. If one pair pf CPUs disagreed (between the CPUs in that pair), but the other pair agreed (between the CPUs in that second pair), the first pair was taken off line and the second pair continued processing.
Each pair of CPUs (and their associated memory) were on a separate board. The faulty board would light a red light, and the admin could pull that faulty board with the system running. A replacement board could be installed, again with the system running transactions. When the new board was installed, a process would start of synchronizing its memory with the content of the memory of the good board. This was done with the system running (processing bank transactions, for example), so the bus bandwidth between the boards had to be fast enough to be able to handle the rate at which the memory contents on the good board was changing. At a certain point the memory state of the two boards would be in sync and the new board would start processing, again in lockstep with the board that had been running all along. The repair process was so easy that one engineering director there had what he called the "mom test" - he have his mom come in and see if she could fix a system that had been forced to throw a fault. Red light? Pull that board out and put a new one in. New board's lights went red-yellow-green (as memory was brought into sync), and you're running fault-tolerant again. Easy peasy. (When the boards failed, they'd "phone home" and Stratus tech support would know there was a fault often before the customer did. They'd ship a replacement part overnight to the site with the bad part. Anyone who worked at Stratus in those days knew the story of the FedEx driver doing a repair for a customer.)
OS upgrades were done by un-synchronizing the OS drives and upgrading one at a time. One would be taken off line for an OS upgrade. The system would have to go through a restart to run on the upgraded OS drive (that would be done in a planned maintenance window), but once it was running, the second OS drive would be mirrored to the running OS drive, at which point the disks were redundant again. The unplanned downtime was zero, the planned downtime was minimized.
It became a challenge for Stratus when CPUs became nondeterministic (instructions wouldn't necessarily process in exactly the same order, making lock-step processing a real problem). At least one CPU architecture transition was driven by that issue. And clearing the heat from 4 cpus in a small space was a thermal challenge. But they were reliable beasts, expensive enough only to be used for workloads involving real money (e.g., financial transactions). Back when the founder (Bill Foster) was still CEO, it was a great place to work. When he left and the MBA's moved in, the company got sold to Ascend Communications, then Ascend got bought by Lucent, the non-telecom part of the business was spun off (as Stratus Technologies), and yes they are still in business.
see subject
Yeah, I'm in my late twenties and a lot of folks my age know old TV shows. Nickelodeon's Nick at Nite had a lot to do with it - that channel was left on almost all the time in everyones house I knew, so inevitably you had I Love Lucy and Petticoat Junction reruns where Rugrats was playing an hour before.
Just too expensive and got replaced by lots of smaller cheaper boxes.
When I started working on them I was surprised to find the predominant programming language was PL/1. I never expected to use that professionally but ended up doing it for about 6 years.
I was there when "I personally witnessed a take out of a Novell 2.x file server which had a 16 year uptime. This was for a school system, and they had forgotten where the file server was. Stuffed in the back of a janitorial closet, and dust covered." Small world!
"Stratus server that has operated with no unplanned downtime since February 2001 -- and hasn't even been rebooted in more than four years."
Two years untouched, I returned and saw its screen had gone blank. Linux distro, booted from DVD, in-RAM, quite a chore to bring everything back after shutdown and reboot. But it wasn't needed: ctrl-alt-F2, init 2 to be sure, init 5 and presto the desktop was back. Singing along. Whatever the issue it didn't take my old Linux down. Happy for life's little joys.
I'm not as surprised as other people making comments about this article. In 1993, I ported Raima Data Manager (at the time, a network-model DMBS running 12,000 different commercial applications (you never heard of it, because it simply worked) to Stratus VOS. The manager of the Stratus office in Bellevue, WA gave me a tour. In the glass-walled machine room, he opened up the Stratus machine running the office - the center of the company's northwest US sales operation - and pulled out a board. I looked out of the glass walls in horror. After a few seconds, the manager pushed the board back in and said, "Look at this." So I looked at the console. The messages, to the best I can remember, said, "Board 9: CPU. Removed." "Board 9: CPU: inserted... testing... OK... Online." The salesmen in their offices never even looked up.
Two other things struck me at the time as being radically different from what I was used to. First, during the port I accidentally used the debugger command to step *into* a low-level C-language routine. The message that came back let me know that the source code lived on development disk 1 of XXX machine in the Los Angeles office, and because I didn't have permissions allowing me to see that code it wasn't going to show it to me. Wow - seamless wide-area networking in 1993. Second, I learned that Stratus VOS only supported a (highly-capable) Stratus terminal, an that my programs had to work with that and nothing else. I asked, what if I'm running a Wyse 50 Whizbang 7? The manager said that I'd simply register that terminal and its characteristics with the operating system - there was an easy way to do that - and the operating system would take care of any necessary translations. Wow again: something Unix got wrong: of course the operating system should take care of supporting different kinds of terminals! (Just like disk drives: my programs should not know or care about low-level details like how to write to a disk or terminal.) Finding something Unix got wrong is rare indeed.
Stratus VOS (a descendant of Multics, cf. Unix) got a surprising number of things right. Having a server actually running "next to forever" doesn't surprise me.
I personally witnessed a take out of a Novell 2.x file server which had a 16 year uptime. This was for a school system, and they had forgotten where the file server was. Stuffed in the back of a janitorial closet, and dust covered.
I'll go one better on that... I know of one that was up for a couple of decades and finally failed, and when they went looking for it, they had to break through some drywall into an odd corner of a closet where it had accidentally been sealed off by construction contractors.
Someone had to do it.
I'll go one better on that... I know of one that was up for a couple of decades and finally failed, and when they went looking for it, they had to break through some drywall into an odd corner of a closet where it had accidentally been sealed off by construction contractors.
You mean the one everyone read about 16 years ago
Slashdot Article
has probably been running longer.
Does that count as a computer?
Obligatory BeOS plug goes here ;)
... is not to talk about server uptime. To anyone. Now you've just jinxed it.
Disssssssss.......misssssssed!!
For what it's worth, we called them "coffee pots" back then...
When I managed NT 4sp6 servers - as soon as Task manager showed around 500 idle hours, it was time for a reboot, because magical shit would start happening..
In Soviet Russia the insensitive clod is YOU!
Given that the version of the OS that supports that machine hasn't been updated for more than a decade, that machine probably has been running continuously for a lot more than 10 years.
If it was bought in 1993, the CPUs were probably PA-RISC, not 68K. I can't tell for sure, because the picture was not a Stratus machine.
The current generation of CPUs are functionally dual socket zeons in 4U rack enclosures. The heat envelope allows for up to 24 cores. Operating systems supported are VOS (the original proprietary OS -- which is still being developed), Windows Server, Linux and VMWare.
An engineer who ran for Congress. http://herbrobinson.us
"not exactly mainstream?" Unless you're talking about real industries, like insurance, I suppose. Sure, Facebook and the like can get by with never-consistent kill-them-all-sort-them-later distributed farms of commodity PCs, but there are still some businesses which need a modicum of reliability in their data processing.
There are probably on the order of 10000 System z installations. Yes, that's a small number relative to x64, but it's still very much "mainstream", particularly when you look at how they're used.
In practice, System z machines these days are pretty much all running a bare-metal hypervisor (derived from IBM's VM OS, which was the first commercial virtual-machine OS) hosting various "LPARs" (logical partitions, i.e. virtual machines). The OSes in those LPARs - which may be zOS, zLinux, z/VM, z/VSE, TPF, and possibly others - will be "IPL'd" (rebooted) frequently or infrequently, to handle those configuration changes and patches that require it, depending on how the organization likes to schedule such things. The hardware itself and the hypervisor are likely to stay up for years at a time. Hardware upgrades are probably the most common reason for a hardware shutdown.
That said, z isn't a completely fault-tolerant architecture like Stratus or Tandem (now part of HPE). There are various fault-tolerant options for z machines, but I'm not aware of any configuration that's like Stratus' "open the cabinet and yank out a CPU card" level of tolerance.