What is Mainframe Culture?

An anonymous reader asks: "A couple years ago Joel Spolsky wrote an interesting critique of Eric S. Raymond's The Art of Unix Programming wherein Joel provides an interesting (as usual) discussion on the cultural differences between Windows and Unix programmers. As a *nix nerd in my fifth year managing mainframe developers, I need some insight into mainframe programmers. What are the differences between Windows, Unix, and mainframe programmers? What do we all need to know to get along in each other's worlds?"

Comments

Programming in COBOL

[cratermoon]

I've never actually written any COBOL myself, but here's what I've learned from trying to teach Java to former mainframe developers.

• COBOL is actually remarkably like wordy assembly language
• The typical mainframe programmer, being steeped in COBOL, will think of everything in "records".
• Mainframes are case-preserving but case-insentive. Like DOS, a token can be any mixture of case an it will work. Thus, a mainframer might wonder why 'PRINTF' doesn't work for 'printf'.
• On the same topic, a mainframer will assume that something like the Java type 'Account' and the variable 'account' actually don't distinguish anything, and will be confused when the compiler refuses to assign to a type.
• MOVE CORRESPONDING is COBOL's big hammer. It will take all the values of the elements of one record and copy them to the "corresponding" fields of another record. There is nothing like type-checking for this. This will cause mainframers to be confused about why you can't assign a linked list to an array and have it "just work".
• Not that mainframers will grasp "linked list" or "array". Actually, they won't really get any of what we call the standard data structures and algorithms learned in the first year of any CS program.
• COBOL programs have no scoping rules. EVERYTHING is global. Thus, a mainframe programmer won't understand why an assignment way over in some little function in another library to a variable called "date" won't affect the "date" value in the code everywhere.

Coded on mainframes, code in *nix now

[Naum]

Dating back to Burroughs machines and managers who didn't know how to use a text editor, preferring to use punched cards and "punched card emulation", but yet they could do a randomizer calculation without a calculator in their head within a second or two.

I've also had the opportunity to train mainframers in shops where MVS platforms were displaced by *nix based platforms. So, here is a subject that, no doubt, I can speak about:

The major factors/differences:

First, most of the mainframer programmer contingent has been moved offshore or is being done by NIV programmers. Really not much of a career path here, but OTOH, a great deal of critical systems (charge card processing, airline reservations, utility company systems) are still coded in MVS COBOL/DB2 (or IMS, a hierarchical mainframe database platform for IBM MVS). To convert these systems means you need to be able to understand these systems, and please don't give me a business analyst -- the days of their expertise are long gone, and the metamorphisis of systems over time means business knowledge is embedded in the code.

Mainframers don't get GREP. I've tried so many ways to impart this wonderful tool, but all I get back is puzzled stares and bewilderment, for anything more complex than what could be accomplished with a non-regex, or simple wildcard search.

Globals. This is something that put me aback 6-7 years ago, when I made the leap into Unix programming, and traded C/REXX/CLIST for C/Perl/etc... COBOL is structured into divisions and all your data declarations are laid out and globally accessible. Though many COBOL systems are quite complex, with a "program" actually being a driver for a whole hierarchy of 20-40 sub-programs, and the necessity to restart at a given point in processing can make things quite complex.

Approvals, checkoffs, signoffs, and procedures. They're largely absent in the Unix (and most webdev work) world, but mainframers have grown accustomed to reams of authorization and approvals for even simple changes. Lead times of a week or more, along with VP signoff, QA signoff, user group signoff, fellow developer signoff, etc.... Even getting a downstream system to agree to test changes may take a formal request process and budgetary allocation of thousands of dollars. This is probably the biggest divide, and future schisms will be prevalent, as data center leadership trys to impose this kind of checks and balances on developers not accustomed to these obstacles. IBMs trouble and difficulty in the web server world offer a prime example -- business being told that it'll take 3-4 months to get a server online, and folks who know better just can't understand that.

Lack of user tools. A big part of what I did as a mainframer was building tools, using BTS and File-Aid to allow developers and testers to create their own test bed and automate the test process. On Unix side, the tools come with the OS, and awk, Perl, and all the other CLI goodies make automating testing a snap.

File in/File out vs. piping. Mainframers have a tendency to see everything as file-in/file-out. In a way so do *nix coders, but a seasoned *nix programmers sees the tools all being able to feed eachother. Rather than step1 filein fileout, step 2 sort filein, out fileout, step 3 filein, reportout, etc...

On the age thing, most of the really skilled mainframers now, like myself, do Unix or migrated to Java. Others are awaiting retirement, or head over to six sigma teams, business analyst roles, or seek refuge in management, escaping the axe that clears the way for the offshore coder.

Paper over softcopy. Got to have that printed listing, and the sticky notes (and before that, paper clips). I still remember a senior manager telling me when I first broke in how his appraisal of a programmer was how many fingers he needed to act as placeholders when he perused a program listing.

The difference IMHO

[FJ]

I'm a mainframe sysprog but I've coded on Unix & Windows. I'm also rather young (33) for a mainframe sysprog. Here are the differences.

The first difference is the difference of work running on a system. Unix & Windows development typically takes place on dedicated machines. The changes are then applied to a separate production machine. On a mainframe development & production are often the same LPAR (Logical Partition) or the same physical box. Because of this development gets the low priority. If you run out of juice on a Unix/Windows box you either get a bigger one or you cluster them together. In the mainframe you either redesign it to run more efficiently or you start shelling out $$$ for a bigger machine. Normally your only choice is the redesign.

Software on a mainframe is horribly expensive and the faster the machine the more it usually costs. This is an old way of spreading the pain of software development. The big guys pay more because their machines are faster but the smaller guys get to pay less. Imagine if MicroSoft decided to charge a lot less for Office if you ran it on a P5 instead of the newest processor? Some software on Windows is licensed by the CPU, but I've never heard of the speed of the CPU being a factor. Do you think you'd get that fancy new PC if the software would cost 10x as much?

On a mainframe software development is a slow process with lots of checks along the way. Nobody just "slams in a change" unless they are either 100% sure it will work and it fixes a critical problem that is impacting business, or they want to be fired. Banks frown heavily on downtime. Unix & Windows systems seem to be more tolerant of this (with the odd exception being email - how email became the most important application is beyond me).

Once you develop, debug, and get a mainframe program running you can usually forget about it. There are programs running on mainframes today that haven't changed in 30 years. That is a pretty good return on investment. I've dealt with both and it seems to boil down to "pay me now or pay me later". Installing stuff on a mainframe take a lot of up front work but if you do it correctly you can expect it to work well when you are done. Windows programs are easier to install and develop but you have the constant reboot issues, memory leaks, and just plain annoying mysteries to deal with.

Mainframes (in my opinion) have far far far superior system diagnostic tools. If a program is running slow I can determine if it is CPU, disk, database contention, or any other resource shortage. This is mainly because there is so much running on any given mainframe that system diagnostic tools need to be very good. The tools on Unix and Windows are good but they don't need to be as complete because the environments are far less complex.

Program debugging tools on a mainframe can be awful. Interactive debuggers are the exception, not the norm. They tend to take up CPU which drive up software costs which the finance department hates. I've seen good interactive debuggers but they suck CPU and make the finance department hate you.

Batch controls on a mainframe are far superior to Unix or Windows. This is mainly because the mainframe started life as a batch system. Once you understand and master JCL it is really a good system. Batch on Unix and especially Windows is more of an after thought. You can run batch, but the tools to monitor failures, schedule dependencies, and validate results are not as good.

A programmer must know how a program is going to run on a mainframe long before you run it. You need to know how much disk, CPU, and memory you need and how man lines of output you are going to use. If you exceed this by too much your program will be automatically canceled. This is because you are not the only one using the system and if you exceeded what you said you needed your program could have a problem. That can be painful but it stops program loops if done properly.

The "just reb