COBOL Turning 50, Still Important

Death Metal writes with this excerpt from a story about COBOL's influence as it approaches 50 years in existence: "According to David Stephenson, the UK manager for the software provider Micro Focus, 'some 70% to 80% of UK plc business transactions are still based on COBOL.' ... Mike Gilpin, from the market research company Forrester, says that the company's most recent related survey found that 32% of enterprises say they still use COBOL for development or maintenance. ... A lot of this maintenance and development takes place on IBM products. The company's software group director of product delivery and strategy, Charles Chu, says that he doesn't think 'legacy' is pejorative. 'Business constantly evolves,' he adds, 'but there are 250bn lines of COBOL code working well worldwide. Why would companies replace systems that are working well?'"

Comments

Oo, oo, oo! I know!

[Samschnooks]

Why would companies replace systems that are working well?

Because the director of IT or whatever his title is will want to be able to put on his resume that HE moved a company from a "Legacy" and "Outdated" system to a modern "web based solution" that enables "greater productivity" among the workforce saving "millions of dollars". Now, he can put that on his resume and go for the CTO, CIO, or whatever jobs.

I've seen it and it works.

Re:Oo, oo, oo! I know!

[Jurily]

Because the director of IT or whatever his title is will want to be able to put on his resume that HE moved a company from a "Legacy" and "Outdated" system to a modern "web based solution" that enables "greater productivity" among the workforce saving "millions of dollars". Now, he can put that on his resume and go for the CTO, CIO, or whatever jobs.

If and only if they're able to pull it off. It's also a nice way to end your carreer if you fail.

Re:Oo, oo, oo! I know!

[fuzzyfuzzyfungus]

I'm guessing that this story involves enough java to float a battleship; but not quite enough to keep the interface responsive...

Re:Oo, oo, oo! I know!

[trash eighty]

no if you fail you just get promoted out of harms way

Re:Oo, oo, oo! I know!

[plopez]

If and only if they're able to pull it off. It's also a nice way to end your carreer if you fail.

no, you collect a bonus and bail out before it crashes in flames leaving someone else holding the bag. See also the bank failures for examples of this. See a pattern? I hate MBAs.

Re:Oo, oo, oo! I know!

[Tubal-Cain]

How can I be more alert in the morning?

add more Java

Hey, you're right! That really is the solution!

Re:Oo, oo, oo! I know!

[coryking]

Depends. Staffing is one bit of the equation. You've also got to factor in "how easy is it to extend our existing application". Can you tie your COBOL stuff into your customer-facing billing system on your website? Can your call center be updated so the CSR's are all doing data entry on a web app or a "real" desktop application? I'm sure both are possible but I'm also sure both have price structures that are insanely high, but just a notch less insane as "hire a team of devs to dump COBOL"

There is one more thing I'll toss out, but I'm not sure about. Location. If you are located in Bumbleskunk, USA, the talent pool is pretty small and you will have a hard time getting programmers to relocate (after all, if your company sucks, they are basically gonna have to move again to work some place else). I wonder if there is any correlation between "Uses COBOL" and "Is located in Bumbleskunk". In other words, how many companies headquartered in say, Boston or Silocon Valley use COBOL vs how many companies in say Cowtown use COBOL.

Re:Oo, oo, oo! I know!

[nurb432]

The last CFO i worked for did that. Was hired to 'rescue' the company but instead he ruined the company, fired most of us that stuck with the company even during the dark times. then he left, with a bonus. ( i hear he destroyed the next one too )

1 year later the company filed bankruptcy.

In reality it was the best thing for me personally, it got me out of the slowly dying industry i had been in for 20 years just before the collapse. But the way it went down was just wrong.

Re:Oo, oo, oo! I know!

[blind biker]

This is not necessarily tied to MBAs as much as it's tied to corporate psychopaths - the ones most likely to succeed in the modern chaotic corporate world (especially in larger publicly-traded companies). They will lie, manipulate and use everything and everybody only to further their own interest. If the company, their workers or even the economy of the whole USA will suffer, they won't care one bit - no conscience.

Why replace it?

[Anonymous Coward]

Why would companies replace systems that are working well?

So I can have a fucking job?

Re:Why replace it?

[Extremus]

I am willing to offer you a good one, but I couldn't find your address anywhere.

Re:Why replace it?

[artor3]

Learn COBOL, and you always will. My dad is a COBOL programmer for the NY state government. According to him, around 95% of their COBOL programmers are within 10 years of retirement. The youngin's (as he calls them) are in their mid to late forties.

If you know COBOL, you are absolutely guaranteed a job there.

Cool

[bigsexyjoe]

Does that mean my Java skill set is likely to keep me employed for the next 30 to 40 years?

Define "working well"

[Just Some Guy]

Before I start, I know all too well that you can write good or terrible code in any language. Still, most COBOL code I've seen is written to run well on hardware that is no longer relevant. A recent experience with an ex-coworker illustrated this pretty well for me:

Said fellow, call him "Joe", had about 30 years of COBOL experience. We're a Python shop but hired him based on his general coding abilities. The problem was that he wrote COBOL in every language he used, and the results were disastrous. He was used to optimizing for tiny RAM machines or tight resource allocations and did things like querying the database with a rather complex join for each record out of quite a few million. I stepped in to look at his code because it took about 4 hours to run and was slamming the database most of the time. I re-wrote part of it with a bit of caching and got the run-time down to 8 seconds. (Choose to believe me or not, but I'd testify to those numbers in court.) I gave it back to him, he made some modifications, and tried it again - 3 hours this time. I asked him what on Earth he'd done to re-break the program, and he'd pretty much stripped out my caching. Why? Because it used almost half a gig of RAM! on his desktop and he thought that was abhorrent.

Never mind that it was going to be run on a server with 8GB of RAM, and that I'd much rather use .5GB for 8 seconds than 1MB for 3 hours of intense activity.

So Joe isn't every COBOL programmer, but you and I both know that he's a lot of them. But back to the direct point, how much of that 250GLOC was written with the assumption that it'd be running on 512KB machines or with glacial hard drives or where making the executable as tiny as possible was an extreme priority? Doing things like storing cache data in hash tables would've been obscenely expensive back in the day, so those old algorithms were designed to be hyper-efficient and dog slow. Whether you think that constitutes "working well" is up to you.

Re:Define "working well"

[thethibs]

Nice story, but it doesn't say anything about COBOL.

I have a similar story about 30 programmers who spent two years writing java code and delivering nothing useful because the requirement called for two different architectures: one best served with a batch system, the other best served with a real-time system. What they need is COBOL and C, but what they know is java and struts. It's been another four years since I ran screaming from the building and they still haven't delivered anything useful.

Inept programmers will screw things up in any language.

Re:Define "working well"

[TheRaven64]

You see that kind of thing from lots of programmers who only know one language well. This is why a good programmer always keeps up with modern architectures. I've seen C programmers who put things in globals rather than passing them on the stack, because that was faster before caching (now it breaks locality of reference, and moves something that was in a register or on the stack to an indirect reference where it needs extra loads and causes extra cache churn).

I've seen programmers who grew up with Pascal carefully turning multiplies into sequences of adds and shifts. Great, except that something like the Athlon can do two multiplies in parallel, but only one shift at a time (because most code contains a lot more multiplies than shifts), stalling the pipeline.

Another common issue is aggressively testing for potential special cases for optimising, ignoring the fact that branches are very expensive on most modern CPUs and the cost of the checks is now often greater than the saving from shortcutting the special cases.

Java programmers are not immune to this, and often optimise based on old versions of the JVM. One favourite is to add finally everywhere, making the code very rigid, believing this makes it faster. In a modern JVM, finally is completely ignored; the VM already knows if a class is not subclassed and will do the same optimisations whether it is declared finally or not.

There's a reason why the rules for optimisation are:

1. Don't.
2. Don't yet (experts only).

If you write good algorithms, your compiler will usually produce reasonable code. If this isn't fast enough, then make sure you really understand how your VM and target CPU work, before you try optimising. The experts only part isn't a joke.

Re:Define "working well"

[Just Some Guy]

Nice story, but it doesn't say anything about COBOL.

I think it says a lot about The COBOL Way, or at least the way things were done when those ancient codebases were being written.

Re:Define "working well"

[david.given]

One favourite is to add finally everywhere, making the code very rigid, believing this makes it faster.

I think you mean final here, no? finally does something else.

1. Don't.
2. Don't yet (experts only).

Very true.

I'd also add the additional rule: You don't know it's slow until you've benchmarked it. All too often I have seen, and I should add that I've perpetrated this myself, people spend ages painstakingly optimising parts of a system that felt like they were cause speed problems, when actually they weren't.

I once spent about two or three months designing and implementing a clever caching system for properties in a UI framework. It was subtle and complex, and I was very proud of it, and it was a total waste of time. We eventually threw it all away and stored the properties in an AVL tree. Yes, this involved looking up the properties in the inner loops of all the UI component redraw methods, but compared to the amount of work of touching every pixel on the screen, this was trivial. And it increased flexibility, reduced complexity and code size, and vastly improved maintainability.

Re:Define "working well"

[TheRaven64]

I think you mean final here, no? finally does something else.

Yes, sorry, final.

And it increased flexibility, reduced complexity and code size, and vastly improved maintainability.

Reducing code size is important for three reasons:

1. It means that a single human can understand more of the program at a time.
2. It means that there are fewer places to look for bugs.
3. It reduces instruction cache usage.

This last part is something I should probably have mentioned in the last post. Instruction cache churn is one of the biggest performance killers on modern CPUs. It is particularly instructive to compare C++ templates and Objective-C. In C++, your compiler will generate a new set of functions for every template instantiation (well, not quite, but close enough). In Objective-C, it will only emit one copy of a method and then use run-time lookups to determine how things work. The Objective-C solution to the problem is slower. You can generate lots of microbenchmarks that prove that it's slower. Except, somehow, the C++ program ends up being slower overall, because the Objective-C program can keep most of the code in cache, while the C++ program is constantly thrashing the instruction cache. If you run a couple of programs concurrently, each one gets even less cache usage, so you end up spending even more time waiting for main memory.

This is almost a corollary to your comment: it isn't slow until you test it in place. On a modern computer there are so many potential sources of performance issues that you can't always take a microbenchmark and generalise it. There are lots of cases where one option is slower when you do it once but faster when you do it a few million times, or faster when you do other seemingly-independent things. Microoptimisations are no longer a good way of ensuring that an entire program runs quickly.

Re:Define "working well"

[McSnarf]

Hmmm...

This revived some slightly old memories.
I remember a talk by the local FORTRAN compiler guru in the mid-70s.

After talking about some intricacies of the IBM FORTRAN H compiler, he gave some examples of the compiler's abilities. Summarizing it with: Don't try too hard to optimize. Leave it to the compiler. It usually knows what it is doing.

And that sums it up rather nicely.
I'd rather work on code written by someone else who concentrated on writing readable code than on code written by someone trying to be clever.

(Note to people born after around 1980: Yes, we too believed it would be cool to write cool, complex code, with the odd assembler routine thrown in for good measure (which, btw. didn't really save much time), badly documented and demonstrating our superiour coding abilities. Looking back, we were idiots who should have been fired. Don't repeat our mistakes, OK?)

Re:Define "working well"

[mcrbids]

There's a reason why the rules for optimisation are:

      1. Don't.
      2. Don't yet (experts only).

If you write good algorithms, your compiler will usually produce reasonable code. If this isn't fast enough, then make sure you really understand how your VM and target CPU work, before you try optimising. The experts only part isn't a joke.

Except that there's a clear and definite time to optimise - when performance is in the crapper!

Just 2 days ago, I heard complaints about a well-written (but very old!) report that was still taking as long as 5 minutes to run when accessing a large data set. Taking a look, I found that it was using an old template system for output using regex, and for a complex report, the regex was killing the server. So I replaced it with a different template engine using simple string replacement, and reduced > 5 minutes reporting time to about 15 seconds. Further looking there found a simple 3-element index in the database cut the time down to 2 seconds.

Now the report comes up INSTANTLY in most cases.

Optimising is necessary - but only AFTER the fact!

Re:Define "working well"

[Unoti]

You don't know it's slow until you've benchmarked it.

This is similar to a mantra of continuous improvement: "In God we trust. All others must bring numbers." The point of that saying is that if you want to make a change to the process, you must support your assertions with real metrics.

Re:Define "working well"

[linuxrocks123]

> I've seen programmers who grew up with Pascal carefully turning multiplies into sequences of adds and shifts. Great, except that something like the Athlon can do two multiplies in parallel, but only one shift at a time (because most code contains a lot more multiplies than shifts), stalling the pipeline.

That strength reduction of multiplication to bit shifting might not be valuable anymore for modern CPUs surprised me, so I looked up the instruction latency and throughput for the Core 2. To see for yourself, download http://download.intel.com/design/processor/manuals/248966.pdf and look at Appendix C.

The timings show that integer multiply, IMUL, is still quite a bit slower from the bit shift instructions (SHL/SHR). So, turning multiplies into adds and shifts is still a Good Thing.

However, these days, a compiler can do this type of thing for you much of the time, so doing the transformation by hand is usually not necessary. Also, since optimizing the source code in this way does take time and decreases source readability, the only cases where people should really still be doing manual strength reduction are in performance-critical code where the compiler, for whatever reason, is missing the optimization opportunity.

---linuxrocks123

Re:Define "working well"

[TheRaven64]

You are missing the point. When you pass something as a function parameter, it is either in a register, or in a bit of memory that is guaranteed to be in the cache and will hopefully (e.g. on modern x86 chips) be aliased with some hidden registers. The cost of accessing it is zero. If you put the data in a global, then you are accessing it via two more levels of indirection - read up on how your linker works - and so you are requiring at least one, probably two, more cache lines to be used and an extra set of load and store instructions for something that could otherwise stay in a register.

Re:COBOL Jokes

[larry bagina]

have you heard about object oriented cobol?

It's called ADD 1 TO COBOL GIVING COBOLPLUSONE

DOH!

[Daswolfen]

That's what I get get for learning FORTRAN in college rather than COBOL...

at least my mad HTML skills..

oh wait... all websites are in FLASH or PHP now...

DOH!

Ok.. im going back to watch a movie on my Betamax or HD-DVD player....

Re:DOH!

[Brandybuck]

Believe it or not, there is a LOT of Fortran out there. I've run across quite a bit of it in the past few years. A lot of scientific, engineering and other number crunching apps were written in Fortran, and there's no reason to rewrite them just because they're thirty years old. The apps might have brand new GUI and visualization front ends, but deep in the heart there is some Fortran code encapsulating the domain specific math.

Re:COBOL Jokes

[TheRaven64]

Surely you mean ADD 1 TO COBOL GIVING COBOL. In C, ++ is an increment, not an add-and-assign-to-new-variable operator.

Adequate Languages

[j. andrew rogers]

COBOL is a perfect example of an "adequate" language, like most programming languages that are in common use. Adequate languages linger forever if there is a tool chain to support them because there is little in the way of economic rationale for replacing them.

The reason adequate languages proliferate over the long term is that only inadequate languages get replaced, and "ideal" languages become hopelessly politicized by purists and ideologues, leaving the adequate, practical, boring languages as the sound business choice. It is a real-world example of perfect being the enemy of good enough, but for economic reasons good enough always wins.

COBOL, not so bad

[Ancient_Hacker]

As just one data point, a certain place replaced a COBOL app that ran with millisecond response time just fine on a 2 megabyte 1 MIPS mainframe, replaced it with a spankin' fresh Java app that ran about 2000 times slower on an 8 gig 16-CPU, 800MHz very expensive water-cooled mainframe.

Now it could have been due to having a bunch of neophyte Java weenies doing the coding, but I'm just sayin', when there's three orders of speed and 3.5 orders of RAM, there may be something significant in the implementation language.

Live a passionate life

[cryfreedomlove]

When I was working on my CS undergrad degree, more than one professor told me that I was really limiting my job prospects because I declined to take the elective COBOL classes. I knew enough about COBOL then to know that I could not drag myself out of bed in the morning to make a living with it.

The ranks of COBOL programmers out there are living drone like lives without passion or joy. That's not for me. I code for the love if it.

Skill Sets are disappearing

[1c3mAn]

Why is no one updating Cobol code? Because the skill to interact with other systems is disappearing.

As a Mainframe Utilities Programmer I hear it from customers all the time. "We can't touch that system because the guy who wrote it retired." System here just represent the code, but also the server backend stuff like database design.

I have heard stories of an IT department being 10 man team. In the 80s that team had everyone dedicated in maintaining the mainframes. Now, they still have 10 people but only 1 person is there to work on the Mainframe.

So now you have code from the 70s that no one understands, running a mission critical application, and you think the IT manager is going to touch it? He is praying it doesnt break on his watch or he might get a call from the COO. Even if it breaks, it is better to patch it then rewrite it because the database behind it is so vital to all the rest of the application that it cant be changed either.

The issue mainly is that no one is teaching old skills anymore. Skills that are still required, but really arent 'sexy' for young college students to learn. Even the name "Mainframe" has grandfather connotation to it while if people actually looked at the IBM Z Servers, one would see how high tech these systems actually are.

Re:Skill Sets are disappearing

[Pig Hogger]

If by "high tech" you mean "pay $250,000 for the same speed you can get on a commodity desktop in 6 months"... then sure.

People realized how stupid it was to waste money on mainframes when commodity hardware is moving so quickly.

"Commodity" desktops will never be able to process 2500 simultaneous transactions in the same database. Even in Beowulf clusters. This is why there is still some big iron around.

Re:2 jokes, 1 question

[berend botje]

When you have working code in COBOL, really battle-hardened proven-beyond-doubt COBOL code, would you really trust a mechanical translation into another language?

I wouldn't, no way! And there is no way to completely test the new code either, as the specs never existed or at least are missing and/or outdated.

I'd rather keep the working COBOL code. Even if that means I have to deal with grumpy old geezers to maintain said code.

Does this add up?

[Allicorn]

How does this add up?

1. Around a third of UK companies say they have even at least one COBOL program somewhere in their enterprise.

2. Around three quarters of all UK electronic business is coded in COBOL.

I'm aware that there are allegedly pockets of COBOL here and there with some fairly significant nuclei of usage within certain business sectors but seriously... 80% of all electronic transactions?

Monster.co.uk search for single keyword in title, 11th Apr 2009:

Java: 173 hits
C++: 142 hits
PHP: 95 hits
Perl: 39 hits
COBOL: 1 hit

This doesn't seem to suggest a great demand for COBOL coders at present which - one would think - suggests little use of COBOL.

I've heard this "the world secretly runs on COBOL" story countless times over my career, but seldom seen more than a few lines of COBOL actually deployed in businesses I've worked with. Is the whole thing just a weird industry myth?

Re:Does this add up?

[PPH]

You are confusing the amount of development done with Cobol and the number of transactions handled by Cobol systems.

In my experience, the reason that Cobol still hangs on is that management continues to pursue the philosophy of "patch it just one more time" rather than "port it to something more easily maintained". As time goes by and continual patching gets more difficult, expensive and riskier, the amount of development goes down. But the systems are still up and running.

The ultimate in unmaintained systems was one at an outfit I used to work for where they lost the source code. Its gone, never to be modified again. It was lost before the end of the last century and was suspected not to be Y2K compliant. The solution was to write and run a routine on the applicable database to bump all of the date fields back (20 years, I think). Then, another small routine was written to post process the output add the time back. Management is grinning from ear to ear because, unlike all the other apps that cost millions to maintain, this one costs them nothing. As long as those responsible retire before 2020, that is.

None of you guys are getting it.

[slashdot_commentator]

Its not just the mythical "mission critical" aspect that keeps businesses dependent on COBOL. MANY of those programs required either financial analysts to "vet" the COBOL program, or lawyers to "vet" the COBOL program complied with laws (privacy, methods of determinations, etc.).

Not just are you putting in the cost to refactor the program from scratch, not just are you risking a bug costing your company hundreds of millions to billions of dollars, but you also have to take in the costs of expensive NON-programmers to "bless" the new program.

Then also realize that the new whizbang technologies like SQL and java will RUN LIKE A DOG compared to the COBOL program. That's because mainframes are optimized data I/O machines. They're not great for making intense calculations or retrieving indexed relationships, but they are a BEAST when it comes to pulling out gigabytes of user data, and then making simple calculations and updates to each. It also sounds like top notch COBOL programmers programmed to the machine for efficiency. That's not really done anymore by generic programmers.

New shops don't have to consider COBOL. But any large company (and gov't) could potentially take a huge hit on their finances (in legal issues) if refactor project has a bug. You can roll the dice, or you can just pay another couple million/year and hope nothing ever forces you to consider replacing your legacy COBOL programs that no one knows how they work, or how to change them.

The older generation isn't always wrong!

[ErichTheRed]

I've had an interesting run through IT environments so far. Each one of my employers has successfully used what would be called a "legacy system" to do core business transactions. I'm not old by any means, but I definitely see no reason to get rid of systems that are performing well.

The qualification for that statement, of course, is "performing well." The side effect to using older systems is all the bolt-ons you have to use to get newer functionality working. My current employer uses a midrange system from the early 80s to run the core of the operation, and it has tons of extra software and hardware riding on top of it to do modern things like access without terminal emulators and message queuing. The time to consider a replacement is when these systems become too unwieldy or brittle to manage. For example, if a transaction-processing system needs a daily FTP feed from some source, and it doesn't get it, will that blow up the whole system? If the answer is yes, it's time to fix the problem or replace the underlying system if it's bad enough.

I'm very skeptical of anyone who comes in and says, barely looking at the existing system, that it needs to be ripped and replaced. A lot of it stems from the fact that IT hasn't matured all the way yet. People still come into the field knowing little more than the Java they were taught in school, and don't have the big-picture attitude you need to really understand IT. You may think I'm an old fart, but I'm really not. I've learned the basic rule of IT -- you're not there to play with computers and have fun. You're there to make sure the business you support is able to do their work, preferably without calling you in every night to fix something.

Inhouse web apps?

[ClosedSource]

In a lot of environments in-house web apps would only serve the purpose of being trendy. I suspect a company who is smart enough to keep their working code around would probably resist the temptation of unnecessary "webizing" their internal apps.

COBOL not quite dead

[BrokenHalo]

Quick disclaimer: IAACP (or rather I was in an earlier life, when I had to be).

COBOL's main drawback was never any real technical issue. It is simply that it can be very tedious to do. Having said that, once you understand it, it is also a very easy language to code in. But there's certainly no reason to throw away the code just because it isn't trendy any more.

Having said that, I do remember working on a site in the UK back in the '80s where the COBOL source to some important routines had long since disappeared, and my job was to patch the binary directly. Now that DOES take a bit of concentration. (Fun, though.)

Not a heroic life

[QuestorTapes]

> I declined to take the elective COBOL classes.
> I knew enough about COBOL then to know that I
> could not drag myself out of bed in the morning
> to make a living with it.

COBOL also encourages "heroic" programming. At a shop I worked at, they were very disparaging of the new systems using relational databases, and proud of the mainframe COBOL stuff, because it never went down.

If you don't count the 4 times in one year a hardware failure caused critical business systems to go offline for 2 days to 3 weeks at a time while teams of COBOL coders used tools to manually delete and rebuild tables and indexes when various failures caused -extensive data corruption-.

And the corrupt data caused by operator errors in nightly batch processing.

And the recoding to fix a major financials bug that went undetected for 10 years until we compared the numbers on the external PC system.

You see, that was "normal operations." by contrast, when a network failure occurred and the relational systems we built went offline for 45 minutes, and data recovery was "re-run the job", -that- was a disaster caused by the "sloppy PC programmers and their tinkertoy PC systems."

COBOL's great stuff...if you like being paged at all hours to manually fix data.

Re:It's the maintenance, stupid.

[ralphdaugherty]

* I apologize for the profanity, but any program that can't change a fucking constant is a broken program. Or did they copy/paste 6.55 all over the place?

      Your comment is idiotic. Do yu think there was only one change to the minimum wage in all the decades that California payroll system was in production?

      There was a long thread on that system, and the real reasons that that California bureaucracy didn't want to modify the payroll system at that time was covered in the thread.

      I also provided a solution in the thread that didn't require modifying the system. The California thing wasn't about technology, it was about politics.

      There, I've wasted some more of my life on this idiotic issue.

  rd