US Requirement For Software Dev Certification Raises Questions

dcblogs writes "U.S. government contracts often require bidders to have achieved some level of Capability Maturity Model Integration (CMMI). CMMI arose some 25 years ago via the backing of the Department of Defense and the Software Engineering Institute at Carnegie Mellon University. It operated as a federally funded research and development center until a year ago, when CMMI's product responsibility was shifted to a private, profit-making LLC, the CMMI Institute. The Institute is now owned by Carnegie Mellon. Given that the CMMI Institute is now a self-supporting firm, any requirement that companies be certified by it — and spend the money needed to do so — raises a natural question. 'Why is the government mandating that you support a for-profit company?' said Henry Friedman, the CEO of IR Technologies, a company that develops logistics defense related software and uses CMMI. The value of a certification is subject to debate. To what extent does a CMMI certification determine a successful project outcome? CGI Federal, the lead contractor at Healthcare.gov, is a veritable black belt in software development. In 2012, it achieved the highest possible Capability Maturity Model Integration (CMMI) level for development certification, only the 10th company in the U.S. to do so."

Comments

So that's what the model is based on

[SuperKendall]

'Why is the government mandating that you support a for-profit company?"

Works for Obamacare.

Re:So that's what the model is based on

[icebike]

Exactly. The Supreme Court already ruled you can be forced to contract with a private company for many different things. That cat is out of the bag.
Expect more of this in the future.

As for certifications, like virtually all of them, this one (CMMI) is totally useless in assuring quality.
 

Re:So that's what the model is based on

[Bill_the_Engineer]

As for certifications, like virtually all of them, this one (CMMI) is totally useless in assuring quality.

Proof:

CGI Federal, the lead contractor at Healthcare.gov, is a veritable black belt in software development. In 2012, it achieved the highest possible Capability Maturity Model Integration (CMMI) level for development certification, only the 10th company in the U.S. to do so.

Re:So that's what the model is based on

[whoever57]

Works for Obamacare.

There is a difference between a mandate to buy something when there are competing suppliers of the product and a mandate to buy something from a single for-profit supplier.

Re:So that's what the model is based on

[icebike]

You can choose not to have a driver's license.
You get fined for not having health insurance.

Re:So that's what the model is based on

[mrchaotica]

The question in the summary left out an important word:

"Why is the government mandating that you support a [particular] for-profit company?"

This would be a lot less of an issue if the company in question didn't have a monopoly on providing the required certification.

Re:So that's what the model is based on

[Anonymous Coward]

As for certifications, like virtually all of them, this one (CMMI) is totally useless in assuring quality.

Yeah, that CMMI stuff is old hat for waterfallers, but don't worry, by 2038, the government will have updated its requirements to mandate that all projects shall be conducted using Agile(tm) methods under the direction of a Certified Scrum Master(tm).

Re:So that's what the model is based on

[DarkOx]

There was a social contract your Obama support ilk changed the rules and just expect the rest of use to go along with your tyrannical theft of the freedom we thought we had. Its you people that should get the hell out, go build your workers paradise somewhere else; write back with how well it works out for you.

Re:So that's what the model is based on

[YumoolaJohn]

Just like you don't have to enter an airport. Therefore, when you do enter an airport, you consent to being molested by the TSA.

Sorry, but I don't buy that sort of logic.

Re:So that's what the model is based on

[s.petry]

Exactly. The Supreme Court already ruled you can be forced to contract with a private company for many different things. That cat is out of the bag. Expect more of this in the future.

More? Or did you miss that pretty much every state requires you to hold at least liability insurance to get a drivers' license? And that certainly isn't even the only case before ACA.

What? What planet are you living on? There is no insurance requirement to get a drivers license, and no requirement for a drivers license for that matter. Most states will require you to have insurance in order to register your car, but that is not the same thing as having Drivers License or State ID.

Care to retract your fabrication and start over?

Re:So that's what the model is based on

[ClioCJS]

Yay! I could be completely free only if I was homeless and unemployed! USA! USA! USA!

Re:So that's what the model is based on

[rahvin112]

The majority voted for something you don't like. Cry me a river, it happens all the time. I didn't like GWB but you didn't see me running around telling jack asses like you to leave the country because you voted for the son of the bitch.

Either you support democracy or you don't.

Re:So that's what the model is based on

[Anubis IV]

There's a big difference between " a for-profit company" and " this specific for-profit company". Even as someone who wasn't a fan of Obamacare, I can appreciate that mandating that everyone procure insurance from a company of their choice from among a wide selection of companies who are all competing against each other for your money is one thing, and that mandating that everyone get certified by the one and only company that the government has declared we must use and who has effectively been granted a monopoly by the government is something else entirely.

Re:So that's what the model is based on

[Anonymous Coward]

Been a while since I worked for a company that cared about the CMMI (UPS back in '96 or so) but IIRC a company can not reach the highest level of CMMI. Only project teams can reach it. So just because CGI Federal had a project team with the highest level of CMMI doesn't mean that was the team working on Healthcare.gov.

I also remember in my CMM training that they taught us that the highest level of CMMI (5 I think) should be reserved for things that essentially affect people's lives (medical equipment software, nuclear power plant software, etc...) and trying to reach anything past level 3 introduced inefficiencies in the development cycle that were unwarranted expenses to most software development.

But I agree with your overall point, CMMI certification is a waste of time and money.

Re:So that's what the model is based on

[Anonymous Coward]

Not really on topic, but the original form of Obama care allowed people to buy insurance from the government, it's the republicans that required that that be dropped, and that people be required to buy from a for-profit company.

Re:So that's what the model is based on

[jayveekay]

The government does not compel you to drive.
The government does compel caregivers to provide health care to you.

Re:So that's what the model is based on

[rahvin112]

If you are comparing genocide to obamacare you are an idiot. If you are trying to make a rhetorical argument by arguing the same you're an even bigger idiot.

Re:So that's what the model is based on

[OhPlz]

You generally need insurance to operate a vehicle on a public road, not to get a license.

That aside, you don't need auto insurance in New Hampshire, which is still a state, last I checked. The state calls for its motorists to be responsible. Is responsibility non-existent in the other states? NH has plenty of other similar types of freedoms. No helmet required on motorcycles or bicycles for adults. No seatbelt requirement for adults. The state runs liquor stores right on the highways. Non-felons can open carry, no permit required. Despite all this, the state continues to be a fine place to live. No mass-hysteria.

The government doesn't exist to run people's lives for them, or to protect them from themselves. To me, the right to make bad decisions is a hallmark of freedom.

Re:So that's what the model is based on

[Anonymous Coward]

Actually, it was a great idea. The idea that insurance cannot refuse anyone for pre-existing conditions and have to price insurance to actually cover the masses actually fixes a whole host of issues with the way it used to be. Just read about people that have been trying to buy insurance or too scared to use what they had bought due to pre-existing conditions clauses. I don't much care for the rest of the law, but this one facet made it bearable.

The argument that you are no longer free to freeload doesn't bother me. Yep - if you didn't have insurance, you're a free-loader, unless you could self-insure (in which case you didn't care about this law in the first place) Why? Because you fell off the curb and hit your head, resulting in a few hundred grand of hospital care while you're in a coma, or anything like it. i.e, accidents you are at fault for, or no one is, and yet you will still expect to get care. Insurance covers that percentage.

Re:So that's what the model is based on

[EdIII]

That's some pretty harsh fucking judgment you have there.

Way to blame the victims.

The whole point to insurance is spreading the risk. Somebody is going to get sick at some time. I do have pre-existing health conditions, and guess what? CANCER IS A PRE-EXISTING HEALTH CONDITION YOU JERK.

So don't speak down to me.

I'm fully willing to help pay for my share of the risk. However, you need to face one simple fact:

THERE IS NO WAY ON GODS FUCKING GREEN EARTH THAT ONCE SOMEBODY HAS SOMETHING HAPPEN TO THEM THAT THEY CAN AFFORD THEIR HEALTH.

How many good people (in your estimation apparently) were paying contributors, only to get really sick, and then go bankrupt due to medical debt? Even when they had insurance? How about afterwards if they survived the crippling debt? Everybody is a walking pre-existing condition at some point. Get over it and stop blaming the victims for getting sick, and for sure, stop punishing them.

Getting sick doesn't just ruin your health (and possibly kill you) it completely guts and destroys you financially.

So before you go calling me a freeloader again buddy....

1) FIX THE FUCKING ECONOMY. I'll pay for my insurance, but dammit, you have got meet me halfway. You can't demand something and then refuse to give people the ability to do it.
2) FIX THE FUCKING MEDICAL INDUSTRIES. The reason why I can go under, lose my houses, go bankrupt, is because a medical operation can actually cost a million dollars. That's beyond ridiculous.
3) TAKE PROFIT OUT OF THE FUCKING EQUATION. This is a big one. If you want to force it on everybody, than you need, NEED, ABSOLUTELY NEED, to reduce the costs and make it as efficiently as possible.

I've seen those stats. The US spends many times more person for health care and actually receives less than 80% of the same benefit that other Western countries do. That's with nearly 5 times more money being expended!!!

Here is what you don't understand, and neither does that other asshole.

YOU CAN'T AFFORD HEALTH CARE IN THE US.

There. The Truth.

Minimum wage does not even begin to cover basic living costs, and health insurance companies fuck you at every turn.

You ever hear that saying you can't squeeze blood out of a turnip? Same thing here. You can't demand that the working poor pay for health insurance when the middle class can barely afford anything either.

I know young people that turned down medical insurance because they could not afford their half. You seem to want to drag them through the mud for it.

That's great. When it comes to deciding which one of the absolute necessities needing to be sacrificed for that health insurance, are you going to do it? Are you going to tell them that they need to go hungry for a few days? Have the power shut off to their overpriced apartments/shacks? Lose their vehicle so they can still spend over a hundred a month taking the bus?

That last one surprised the hell out of me. I moved from a much smaller place back to a city after giving up my car. Biked to work for a year or two. The actual costs of bus fare were $4 PER DAY. That's $120 per month. Take that out of minimum wage and push their faces in the dirt huh?

You just don't get it. You can force it all you want upon me and others that are on hard times. Unless you fix the fucking economy I will never have the money to survive, and if you penalize me in the coming years by absorbing my tax refund, you only push me under slowly.

So pass your fucking Obamacare and shove it down our throats. Not saying it doesn't have benefits. For Christ's sake, at least have the fucking decency to hike up minimum wage the amount needed to pay an average insurance policy.

Proof!

[Cornwallis]

That CGI "achieved the highest possible Capability Maturity Model Integration (CMMI) level for development certification..." more than proves that the entire model is useless!

Re:Proof!

[Anonymous Coward]

I've learned that to get successful software, you simply cannot do things "by the book". That's why Skunkworks projects happened, exactly BECAUSE if you go "by the book" (or "follow the process") stuff just won't get done, or will get semi-done spectacularly crappy.

Re:Proof!

[drdread66]

Actually, CGI has some great talent in both engineering and project management. How do I know this? Because I have worked at CGI Federal for three years now. The company's track record of successful deliveries is enviable in the Federal space. I say this based on 10+ years of experience in US Govt software development and contracting.

Of course, none of this is relevant to the CMMI discussion. Bringing up the CGI bogeyman as a counter example to the value of CMMI is purely intellectual dishonesty and FUD-mongering.

Not defending CMMI (IMO it's completely worthless), but I *am* defending CGI.

Re:Proof!

[tricorn]

I remember working on a product produced by a company that proudly trumpeted their Six Sigma certifications. Had a problem with a board that was sold with the explicit feature of being able to do read-modify-write bus cycles on shared memory (each board had a section of on-board memory that could be shared with the other boards across multibus).

Unfortunately, it turned out that the target board would get memory corrupted when you did that (interfered with refresh cycles, I believe it was). Once I figured out that was happening, I contacted the company.

Six Sigma is all about repeatable and documented processes. Well, they documented it all right. They documented that they had no idea what was wrong, that the person who had designed the hardware had retired, and that they had no one there who was qualified to even understand what I was talking about. I guess since the problem with the board was repeatable, that justified their Six Sigma level! They continued selling that board, with the same claim of capability, for several more years.

Ever since then I've had little respect for that type of certification - worried more about the proper process than about the actual results.

Re:Proof!

[hey!]

There's a big difference between people who are capable of doing things "by the book" making an informed decision not to do so, and people deciding to do things in an ad hoc manner because they can't master the "by the book" method.

Every successful project, in my opinion, requires both discipline and risk taking; the art is knowing how much of each the project you are currently managing needs. Every project should have a bit of a stretch built into it, otherwise people get sloppy because they've become complacent. But *too* much risk, and they get sloppy because marginal additions to risk become meaningless to them.

You want control and measurement and all that rational stuff, but developers aren't automatons. They need motivation to care about those things; if they're just going through the motions a formal methodology becomes so much dead weight. So excitement, challenge, novelty, even a whiff of fear can be healthy things. But not chaos, impossibility ,blue-sky goals and outright terror. An excessive dose of medicine is poison.

Bogus from the beginning

[russotto]

CMMI was always SEIs way of trying to reduce programming to bricklaying (only with a lot more paperwork), leaving academics like them as the only real thinking people in the process. It can't work and will never work.

Re:Bogus from the beginning

[Impy the Impiuos Imp]

As part of becomming CMM 3, we had to uabe code reviews. We paid a shitload for some asshole who wrote a book to come in and teach us.

"Do your review before you even make sure it will compile!" he swore. My skeptic bullshit detector went off -- transparently he was trying to amp bug find statistics to make the process look good.

But nevermind -- he got his giant check, the ignorantly savage management had a cover story of doing a good job, and we ate a shit sandwich.

We never did find any real bugs in the several years more I was there, though we did find many coding standard violations. My colleagues had grave difficulty understanding those violations were not actually bugs -- they were to reduce the chance of bugs, but none found were ever actually a bug in effect.

Snake oil & the supposed intellectual in their own field.

Re:Bogus from the beginning

[david_thornley]

If you're not using code reviews, chances are your code sucks. I don't see any need to pay somebody big bucks to tell you that. Similarly, coding standard violations increase the chance for bugs, and it's worth making code conform.

In my experience, with very good people, we find a lot of bugs in code review. If you're not finding bugs, either you're superhuman or you do need instruction in code review.

Re:Bogus from the beginning

[olau]

Regarding code reviews: why do you think they are about finding bugs? While you can probably discover some problems through code reviews, a far more important goal is making sure that people are not turning out shitty code that will blow up the first time someone has to do any maintenance on it. You really want to make sure that people write understandable code.

Re:Bogus from the beginning

[locopuyo]

uh yeah, but it is good to at least make sure it compiles before you send it off for review.

CMMI is a scam

[drdread66]

In 2005, my employer at the time decided to go for CMMI level 3 because it was required by a govt customer for their project. Certification achieved. Then in 2007 my employer opted to shoot for the moon and go for CMMI level 5. Again, certification achieved.

Two years later I left the company, because it was clear that CMMI level 5 was going to kill the company. CMMI level 5 introduced a high level of bloat, inefficiency, process overhead, documentation requirements, and (worst of all) process rigidity and attempts yo manage the development process by statistical analysis. Our delivery times more than doubled. The cost of delivering projects more than tripled. And the Holy Grail of reduced defect density? Nary a sign of such improvement. As far as I could tell, there was -zero- impact on code quality.

Our customers started abandoning us, our reputation circled the bowl, and everyone who had any business sense left the place in droves. What was a $100M/yr contract software development house is now down to 1/4 of the staff and revenue it had in 2009, and I fully expect their parent company will close their doors this year.

I firmly believe that CMMI Level 5 killed that company.

Re:CMMI is a scam

[russotto]

I would fucking kill for software developers to be licensed like an engineering displine, do you realize how much more those of us with a clue would be worth if we could dump all the morons who managed to install a compiler or IDE on their Linux box and suddenly think they are 31337 h4x0rz programmer gods after they managed to run a shell script on their own.

Too bad you'd also dump nearly all the "morons" who wrote the fucking compiler and the Linux kernel and the drivers for it and the IDE and the shell.

Of course, the problem with that is that any sort of proper certification would weed out 9 out of 10 employed 'developers' instantly.

To anticipate the No True Scotsmen answer: No such proper certification has ever been described. Every system proposed which would weed out 9 of 10 employed developers would leave at least 99/100 good developers in the weeded-out bunch. And that certainly includes CMMI.

Re:CMMI is a scam

[drdread66]

Nice way to go fully orthogonal ad hominem while not addressing the actual subject at hand. Did you find your debate skills in a cereal box? Froot Loops, perhaps?

OK for the record: I wrote my first multi-thousand line program in 1978. I was 12 at the time. I hold a PhD in experimental nuclear physics, a PMP certification (project management), have forgotten the details of approximately 119 programming languages that I have learned over the decades (although for some reason, good old fashioned K&R C sticks with me like a bad habit), and don't bother with certifications until my employer wants me to get one for some reason or another, at which point I do what any professional does: I go buy the book, read up until I feel confident I can pass the exam on the first try, and then pound the exam into dust.

The PMP is one example; I had to get that to satisfy the company policies when they promoted me from Chief Engineer to manager of an entire operating division that numbered about 150 people. I took the mandated class and then the exam, out-scoring every single one of the multi-decade experienced program managers who were working for me when I took over the division. Then I went on to grow that division from $24M/yr to $35M/yr in 2 years, when the company split my division in half because it had gotten too big. Then I took the $20M piece and grew it to $35M again in another two years, at which point it was (again) the single largest division in the company.

So: you are demonstrably, provably wrong in your assumptions about what I know about software engineering, business management, and probably everything else you think you can guess about me based on a single post. You also clearly don't understand the complexities of CMMI, how a company earns such a certification, and what the implications and resulting process burdens are downstream of the cert.

Surprisingly, there is one nugget of half truth in the steaming torrent of verbal diarrhea that was your unprovoked attack. You said "Your business was fucked long before CMMI even if you couldn't recognize it." The truth is that there were things wrong with the company, but we were doing just fine overall. The problem is that CMMI added so much friction to the way we worked that the previously minor problems became huge ones. The fundamental mistake the company made was playing along with SEI's demand that we apply CMMI to the entire company rather than just the division that had the mandate to attain at least Level 3. That project was OK with taking 3 years to develop 50k lines of code, and was more than happy to see costs in the $5-10 per SLOC range. The average customer is not OK with those parameters. Incidentally, the project in question was ultimately canceled (not just our part, but the entire acquisition), with sunk costs so high that it makes the taxpayer in me weep like a baby who just dropped the ice cream ball out of the cone. But we made every deadline, met every cost target, and hit every defect density goal. Thank you, CMMI, for making our customer so happy with our performance that they gave us upwards of 97% of the maximum award fee (it was a CPAF contract) but making the overall project so expensive that even the US Congress choked on the bill and killed the thing.

Hey kid, running a software business requires much more than just disparaging other professionals whose skills and history you nothing about. You need to learn what you don't know, then get back to me about...

Oh let's just cut to the chase: go fuck yourself.

Good for sausage manufacturers

[MichaelSmith]

High CMMI maturity levels are really only achievable if you are in the business of mass producing something. They emphasise continuous refinement of production processes, as opposed to research and the development of totally new products. You can write procedures for R&D but they don't allow you to include steps like and then a miracle happens.

some of it is useful

[Goldsmith]

I've worked in the past as part of the DoD Acquisitions Workforce.

CMMI is really just part of a broader obsession in DoD with project and program management. Abstractly, these are good things. When implemented correctly, they make debacles like healthcare.gov nearly impossible. Good planning, budgeting and in-progress evaluation are generally applicable to basic research projects, software development and building ships. We all want to work on projects which are well run.

The problem is, blindly stepping through the predefined process of project management has nothing to do with actually managing a project. You still need good managers who can recognize problems in the technical fields they're working with, understand what to do when problems crop up and are empowered to act. DoD in general fools itself into thinking it has people like this because the paperwork is done right. I suspect that's a fairly common problem.

We all know there's a problem with treating the "talent" (i.e. programmers) as interchangeable blocks using these systems. I think treating management the same way is worse. The ideas that management is mastery of a process and operates solely for organizational interest over individual interest are flawed, but central to things like CMMI.

CMMI utterly useless in my opinion

[Guillermito]

I live in Argentina, where any software company getting a CMMI certification can apply for a tax cut. Because of that, CMMI was all the rage around eight years ago or so. Turns out CMMI was so utterly useless and cumbersome that at this point most companies prefer to forget about the tax cuts rather than bother with being CMMI certified. Only companies seeking government contracts continue doing so.

Silly Billy

[bfr99]

Yet another dig bites man story. Government requirements often mandate testing and certification by third parties, For example, FCC emissions testings.

It's a joke

[Anonymous Coward]

I have 30 years IT experience, last 15 as "design lead". Big projects, small projects, lots of programming.

My company bought in IBM on a project, and I was told I was going to be working under a "Certified Master Architect". Great! This was going to be great learning experience, right?

Day 1, in walks this 22 year old kid, freshly graduated. And, by virtue of the fact that IBM corporate had some certification, all their designated architects automatically became "Certified Master Architects".

Re:We need to combine CMMI, SOA, Six Sigma, ISO 90

[MichaelSmith]

I worked on that project.

Licensing software developers == nightmare

[Anonymous Brave Guy]

The trouble with the idea of licensing software developers is that no-one really knows yet how to develop software well in general. At most, so far, we have some people who have found practices that worked well on previous projects in their parts of the software development world, and sometimes when the stars align they share their ideas for mutual benefit. This is still a long way short of the standards found in true engineering disciplines.

I suspect the inevitable result of licensing today would be that a lot of consultants who talk a good talk would convince the relevant officials that they knew best, and some sort of dubiously authoritative body would be created to mandate that everyone else should follow the will of the consultants. Imagine a world where Robert C. Martin's claim that if you don't do TDD then you can't possibly be a professional software developer actually carried the force of law.