Slashdot Mirror
Glitches in Massive Government Databases?
HBergeron asks: "Rather then post this as another YRO in the litany of new government datamarts there is a more fundamental question for all the coding Slashdot readers out there. This story, in Government Executive magazine, outlines the range of programming glitches in what is a relatively simple database. As a matter of public policy (and taxpayer money) is this level of non-functionality to be expected in these sorts of projects? Is the contractor just ripping off the taxpayers with bad code? How hard is it to write software like this that works?" The article focuses on the SEVIS database, but have others noticed similar trend in other government information systems?
And the government system of going with the lowest bidder is bound to cause some problems as the more expensive engineers would no doubt bring better experience and know how with them. When you bring in the inexperienced because they are cheap, you frequently end up spending more in the long run than if you had paid for the expertise up front.
It's like they say, you get what you pay for. Cheap prices are only cheap if your time has no value.
I have been pwned because my
If the government has a harder time keeping track of people, maybe it will be less ambitious. Never mind.
This seems to be on par with other things the government tries to keep tabs on. They can't keep track of paroled felons, the database of people who can't vote is horribly flawed, and the soundex database that the airlines use doesn't work either.
Granted, this needs to change, but this isn't the first time the government has failed to provide adequate information regarding lists of people.
They've been doing that for years. Toilet seats for $10,000; hammers for $7,000? Not only that how much money is wasted on the old "that's exactly what I asked for, but not what I wanted"?
-- Some days you're the dog; some days you're the hydrant.
This make me glad I don't pay taxes
Neo: I just had a deja vu.
Morpheus: What? What did you see?
Neo: I saw the same Bush pass by twice.
Morpheus: Was it exactly the same Bush?
Neo: I dunno... could've been some kind of father son thing.
Morpheus: A deja vu is a glitch in the database. It usually happens when they change something. Particularly, votes.
====
Crudely Drawn Games
If everything were in hexadecimal and you didn't have to convert, I think that would solve this and many other problems.
-Libertarian secular transhumanist
EDS do a lot of systems that don't work, or don't work properly, or run massively over schedule and budget, here in the UK as well.
I just can't understand why governments insist on using them with the track record of cock-ups they have; they're not even cheap.
Using HTML in email is like putting sound effects on your phone calls. Just say <strong>no</strong>.
I have been working for municipalities for 25 years. I have yet to see a major program work well or work at all without overruns. I have chalked it up to me lacking a MBA or Degree in Computer Science. I am just a poor hobbiest that thinks for a million or three you should get what you pay for. But like shrinked software there must be no implied warrantee or garentee it will work. Man I think for a couple million give me a couple coders and little hardware and sit back. Open source here we come.
Grey Coder
Smile the Joke is on you
...Except perhaps to the executives the magazine is aimed at. Early versions of software are generally pretty buggy, particularly if the target keeps changing, and most especially if it is in response to a hastily crafted law. The only thing that's surprising about this is that the output is taken so seriously by law enforcement officials *prior to completion*.
Don't they have some donuts that need eating?
I've seen this sort of thing happen before.
Government departments are pretty-much obliged to go with the lowest tender, even if the people running the tender know that the winning bidders are a bunch of incompetents who couldn't organize a fsck in a brothel.
So, the lowest bid wins, and then even if they actually are well-meaning and try to do the right thing, they have such limited resourses that they usually have to resort to working too few staff too many hours.
The result will not be quality code.
Quidquid Latine dictum sit, altum videtur (anything said in Latin sounds important)
When it comes to government, failure is rewarded with more money. In fact, failure is often cited as proof not enough money is being spent.
Besides, why bother doing a good job if you know you'll get paid either way? That's what the tax collector is there for!
This scares the shit out of me. I just submitted my paperwork to be registered for SEVIS, and I need to travel outside the US... in fact, I'm going to be out of the country when they're expecting it to be done. They're going to courier it to my hotel in Australia. I'd figured there was nothing to worry about, but if I can't get back into the US... I'm fucked.
Anyone have any first-hand experience with registering with SEVIS? Do the bugs screw over many students, or do they just make the lives of the people in charge of international students hell?
[TMB]
If Doyle (Wisconsin's current Gov and Former Attorney General (MS SELL OUT)) would get his head out of M$'s behind we might not have a state budget deficit.
Got Code?
I think this movie shows what a *real* totalitarian state would look like: the danger to our freedoms is not from corruption but from incompetence. Programs like TIA creep me out because I'm absolutely certain that the Feds will find a way to fuck it up and throw some poor guy in detention because the computer skipped a byte and came up with his name. Ditto for the PATRIOT Act. Few people have recognized this, but what's frightening about Ashcroft is not that he's a fundamentalist autocrat, but that he's an incompetent fool. If innocent people suffer from the government's extension of powers, it won't be due to the GOP taking out its enemies but because some FBI secretary got a virus on her computer.
I'm not a libertarian; the government indirectly pays a large portion of my salary. However, the extension of government power worries me, because the more control they have, the more opportunities to fuck our lives up.
How hard is it to write software like this that works?
.NET or J2EE, I pass it by doubly fast.
Harder than you imagine. If you remove the pork-barrel politics, directives of what technology to use coming from the clouds, and the recently potty-trained project team members, there isn't much left to give the project a chance at success. Most of the project's time is probably spent learning the difference between JDBC and EJB or at meetings discussing the differences between JDBC and EJB. The remaining time is spent accomplishing little by discussing the well-presented but vacuous system requirements for the project. Whenever I see a job posting for a database project for a government agency, I pass it and look for projects worth doing. If it mentions
I don't like this conclusion, but I've worked on, interviewed for, or heard about enough of these projects to realize they are pretty much all the same and for what seems to be all the wrong reasons.
Vote in November. You won't regret it.
I have first hand experience with this subject after spending 2 long years working with a State level government agency to develop motor vehicle registration software ...
... its more about "we're writing software for what we need RIGHT NOW".
... as legislation, beaurocracy, and agency regulations expand, so do the requirements of the software. For example, the Bureau of Motor Vehicles in an unspecified state put their first computer system in place in 1968. Since then, the scope of the BMV has expanded at least 10-fold.
The problem is not so much about "how hard is it to write software that works"
When governments sit down to write software, its usually done through private contractors. So, a group of beaurocrats have a pow-wow and come up with a spec that generally reflects the type of work that the agency is doing "now", without much future consideration.
15 years later
Complicating the issue, "upgrades" are usually in the form of applying a new "layer" to the system somehow. As of 2003 in this unspecified state, the typical motor vehicle registration passes through 4 different systems before arriving in the central (OLD and limited) database at the state.
Complicating the problems even further are the many new layers of regulatory bloat -- meaning, the BMV is using software that met their needs in 1968, but doesn't meet their needs now. For example, (and this is how data goes bad), they're required to track whether or not somebody's registration is under suspension. However, back in 1968 registration suspension wasn't even a blip on the radar. To handle the problem after the "registration suspension legislation" was enacted, an "exception" had to be built into the system... if the street address field contains a special message, it indicates that the registration is under suspension. Ultimate problem... fields in the database are being used for purposes they were never intended. The age of the system does not allow for it to be updated properly.
Skiers and Riders -- http://www.snowjournal.com
Somehow "they" have had UFO technology which would make petroleum obsolete since the '50s, conspired to kill JFK to keep it a secret, brainwashed Chapman to murder Lennon, created a secret government database tracking everyone's cash transactions, control us by putting chemicals in our water and thought patterns in satellite broadcasts. Oh yeah and "they" also were behind the 9/11 attacks as well.
Yet "they" can't even figure out how to keep track of whether or not foreign students went to class or not.
Look no further than the fiasco with Canada's National Gun Registry to see how badly the government can screw up a database project.
After living through that I no longer have any confidence at all in the government to be able to implement any IT project competently. One billion (with a B) dollars to develop a database that essentially matches a gun serial number to a name and address. And they're still not done. HOW? Somebody please explain that one to me...
That's MS BS. (And the cry of incompetent programmers for decades.) Even if we agree that all software has bugs - and I don't - that canard says nothing about all bugs being equal much less anything about all software having about the same number of bugs.
Any competent manager would know that experienced coders are usually FAR cheaper than inexperienced ones because they make fewer mistakes due to ignorance or indifference ("it works for me, so it's done!"). That gets you to the point of dealing with the more subtle and intrinsic bugs (e.g., due to conflicting requirements) quicker and cheaper, and the apparent cheaper cost of inexperienced developers is only achievable if you plan to release after coding is finished, not after testing is completed. Which is pretty much every MS *.0 release, now that I think about it -- got to get to market first, even if it's pure crap!
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
That said, the 'bug' mentioned (that of getting a non-related set of students) smacks of shit data entry rather than a programming failure.
Lord help you, America, if you acquiesce to this sort of intrusive crap.
Before you know it, you'll be as bad as Europe, and soon after, as bad as Britain.
You'll be being persecuted on the basis of bad data, bad laws and stupid politicians.
That's our job.
Stop it.
oh brave new world, that has such people in it!
That's exactly what the CIA wants us to believe! Saaaaaay...aren't you the same Professor D who was involved with the faked moon landings!
C - A language that combines the speed of assembly with the ease of use of assembly.
Unfortunately, no technical details were given (such as programming language(s), database system, and general architecture type. However, a number of the bugs (like the one about not being able to graduate from a BA to Masters, and the one about the birth of a child) suggest an underlying poor relational design. If I knew more about the overall architecture, I could comment more about how many bugs one would expect. Certainly, other people have commented that you can have lots of bugs *before* you release and start taking the program seriously... I understand EDS has often gotten away with shoddy work for the government. I don't know why (political payoffs).
Really really terrible system over here.
that "sévices" in French is pronounced "sevis", and means "cruelty".
Fixing broken EDS code is a large part of my job, the SEVIS project is no doubt another example of EDS shoddiness. The EDS business model seems to be as follows: - Collect $200/hr from client. - Pay h1-B $15/ hr to produce complete choss. - Management keeps the other $185/hr. for second vacation homes etc. But I suppose it is better that this project fail, at least we can count on EDS for something. MM
Of course it's a taxpayer ripoff. Thank goodness we're not getting all the government we're paying for.
I'm an American. I love this country and the freedoms that we used to have.
I think there's an even darker side than what's being presented here in the brief - consider what happens when one of these 'glitches', whether techinical or PEBKAK, cause inaccurate information to be propogated through the linked government databases such as the TIA? Does joe traveller get strip-searched at every airport he goes to because someone "accidentally" put his name onto a terrorist watch list? Where does the government's responsibility to be accurate and precise with our information end? If the Credit Reporting Agencies are any indication, I think we have a potentially huge problem on our hands.
After September 11, it was decided that the system had to be used starting early 2003. This was years earlier than in the original plans.
If it's anything like my work, it probably started out as an Access (etc) database created by someone with limited knowledge of database design to try to make their job easier. It worked well enough and someone decided to get a contractor to flesh it out (perhaps explaining why the contractor referred questions back to the govnt: they are limited by using existing database or something). I don't think the contractor is necessarily ripping off the taxpayers, but who knows. Still, you'd think they'd do a better job of it. I think it's something started small, but got too big too fast. Happens all the time in companies, but has bigger impact when it happens in the government.
I worked on this stuff a little bit, quite tangentially, for a university in the late 1990s, and it was clear even then that there were serious problems. This is what I recall, so YMMV. Basically, the initiative that prompted SEVIS has been going for several years, even before Sept. 11th.
In the late 90s, The government was coming down hard on universities to get their information in order for submission with some ridiculously short deadline that they couldn't possibly meet. When asked what information was to be collected, the government folks hemmed and hawed and wouldn't provide specs at all. My boss even went for a meeting with the head of the program on the government side, who skirted the issue of actual specifications for universities. There was a small "beta" program arranged between some East Coast universities and the government to try out some new software, but if you didn't get involved in the beta, you didn't get any information at all. This left individual universities out in the cold as to what to do. The situation was at a stalemate, with the government and consultants, probably EDS, wasting gobs and gobs of money trying to implement a system and failing miserably. The client software they were hawking for universities was overpriced ($1000s) garageware that didn't work, had a Windows 3.1 interface in 1998, and was completely proprietary.
Sadly, SEVIS is the second or third try at implementing a system, and complete failures again are no surprise to me. This whole project has been a disaster from the start and it should be audited big time.
There is one database used for payroll on which millions (if not tens of millions) were spent, and the end result is a system nobody is completely sure of, and which requires all the deparments involved to completely change all their procedures. And its even less flexible and problematic than what it is replacing. AND this is a custom application!
Until government starts paying tech people what they are worth in the private sector, you will always have poorly skilled bullshitters pedalling their wares to the public sector, who is suffering from the illusion that throwing money at a problem will make it go away.
Manipulate the moderator system! Mod someone as "overrated" today.
You are definitely on to something there about the "lowest bid." But from my experience software often starts spiraling out of control during the requirements process. I have a little theory that bureaucrats are the main people to blame for this as they have a knack for wanting to add software features to simplify their work and complicate the software. So, who has the most bloated software project requirements? I believe the answer will be who has the most bureaucrats?
Nearly half of all people are below average
This is the same old software engineering problem, over and over again.
A user who has never written a *COMPLETE* system specification, acutal has no idea what that is, who only knows what he/she does not want.
Software developers/coders/bodies who are not SME's (subject matter experts), making system / software decisions without either the knowledge or guidance to understand the ramifications of those decisions.
Neither users, nor software development companies want to deal with these issues, they would rather just get the money.
That is why most large software development/ service companies have such bad reputations.
According to SEI, (Software Engineering Insitute) over 70 per cent of all software development projects are terminated as failures.
How hard would it be to do this the old-fashioned way, on paper, with filing cabinets? The federal government could build a records repository and have the schools mail in the forms.
Mea navis aericumbens anguillis abundat
sounds like they took a prototype that sort of worked and decided to put it into production. what a surprise!
the good ground has been paved over by suicidal maniacs
How much will it cost them to pry my gun from my cold dead hands?
This is ofcourse the result of terrorists! :)
Much easier to wreck havoc on a government project and cause disruption through buggy software than to take the time to learn to fly and then hijacka nd hit public buildings. The most positive thing is ofcourse that you, the terrorist, is not killed
If the US really wants to get rid of all potential terrorists, they have to evict everyone of a certain religious and cultural group and then close the borders and let no-one in and out. This is ofcourse not possible, hence all these measures taken by the gov is virtually useless.
If you mod me down, I *will* introduce you to my sister!
Disclaimer : I have worked for a number of Financial Institutions and Large Corporations.
My experience with the problems of these sorts of situations is as follows:
1. Sales droids underbid each other to get the job and commit to ludicrous time frames
2. Project teams end up with short development time and are always pushing to reach the deadlines in time.
3. Client changes their requirements, but will not change their expected delivery date. Either they refuse due to business need, or they do view their change as an actual change. More often they view their change as a "clarification" - even if it contradicts what their specifications orignially said.
4. Agressive job market has Project Managers kowtowing to Client demands.
5. Multiple departments are clients, but pay different amounts into pool. Each department seeks to maximise their benefit at the cost of other departments (despite fact part of same organisation - politics)
I mean, really, the problem exists in the fact business units will often not sit down and commit to producing clear, unambigious details of what they want & need. Bugs creep into the process when your dev's are working frantically to meet the deadlines and handle the unexpected change request.
And now a pithy little quote to put on your wall:
-----
Programming to Requirements is like walking on water.
It's easy to do when everything is frozen.
If the government is going to depend on this, I would think it would be important to stop the printer problems and problems where one univeristy accidently gains access to another.. Couldn't this cause more problems for national security, and maybe cause a greater terrorist threat?
If someone malicious worked there who wanted to delete a single record from another univeristy and then call up immigration to report them, it doesn't sound to hard to do. It also doesn't sound to hard to add someone who shouldn't be here, allowing them access to this country anyways! Whats the point!!!
Paint.NET, a Free Image Editor, with Source Code Available!
I can see how the individual campuses could be in the hurt locker. For obvious reasons, they are not going to want to do manual updates every time an address changes. That means integration with existing systems that (hopefully) allow students/faculty/staff to update their own records.
Many campuses (at least state campuses) are running really old student records systems. Imagine that the web front end for the users is built from screen-scrapes of old mainframe applications. To make it worse, now try to interface 20+-year-old, non-relational, pre-SQL student records systems with a new distributed database model.
The result is that the schools will have to reallocate senior programmer (who else will know how the records system works) time bumping other, mission-oriented tasks in the name of National Security.
Sometimes, SEVIS crashed under the stress and expunged the day's work. The delays and headaches led some schools to close their student offices and ask employees to work nights and weekends, when traffic was lighter.
That's as good as putting a logo on it. So I went to Netcraft and checked the SAVIS web site, egov.immigration.gov. And sure enough: The site egov.immigration.gov is running Microsoft-IIS/5.0 on Windows 2000
I bet that the database is also running on Win2K...
Oh golly, you mean that when you put a high-volume site under Windows, you get (gasp) crashes and data losses? No way! Who would have thought that? Obviously, the poor Dept of Justice was a victim of an unheard-of, unexpected problem.
Not.
--
Mad science! Robots! Underwear! Cute girls! Full comic online! http://www.girlgeniusonline.com/
Some of the worst estimated cost blow-outs that I've seen in government projects, have come
from the government managers hiring consultants to develop their requirements.
Quite often those consultants can charge almost as much as the entire development team charge
for the project.
http://jesus.everdense.com/
"You better bet we'll make a fuss, if you charge to spy on us!"
An angry throng of more than 100 students at the University of Wisconsin at Madison stood shouting in military cadence at a panel of school administrators, who'd called an emergency campus meeting in April. The students, about half of them from foreign countries, denounced the school's plan to make foreign students pay for a U.S. government database to monitor them.
The administrators pleaded their case. Under new homeland security laws, all U.S. schools have to register their foreign students in the database, known as the Student and Exchange Visitor Information System (SEVIS). The system keeps tabs on the courses students take, where they travel and whether they've had disciplinary problems or been arrested.
The university estimated the cost of computers and staff to comply with the program would soar past $300,000 each year.With state budgets slashed, foreign students would have to foot the bill.
The protesters said the plan was unfair, because the money wasn't taken from the general student fund. But they were more outraged that foreigners were being singled out for inspection in the first place. Unmoved by the officials' remarks, they shouted again.
"We deserve a better fate! You know we make Wisconsin great!"
"It was a madhouse," says Mike Quieto, president of the teaching assistants' union at the university, which is leading the campus opposition to one of the most ambitious and controversial security initiatives the government has undertaken in the nearly 2-year-old war on terrorism.
The Wisconsin students aren't alone in their opposition to SEVIS. Many students and school officials charge that the system was an overreaction by the government to the Sept. 11 terrorist attacks.
Two of the Sept. 11 hijackers were in the United States on student visas. Soon after the attacks, the wide-ranging 2001 U.S.A. Patriot Act required that every university, college, trade school and student exchange program begin monitoring their students. Supporters of the program defend it as a necessary security measure.
SEVIS is a test of how well the Homeland Security Department, which manages the system, will handle a much larger and more complicated system it plans to create to track more than 35 million visitors who enter the country each year. That system, known as U.S. VISIT, will collect and share information, including such identifiers as fingerprints, about every person who crosses the border.
SEVIS is supposed to be incorporated into the U.S. VISIT system, which will "coordinate our border information and our enforcement and compliance efforts," Asa Hutchinson, the Homeland Security Department's undersecretary for border and transportation security, explained during a speech in May. U.S. VISIT is "just the latest step in [the department's] information modernization," he said.
That modernization includes initiatives such as upgrading cargo security and airline passenger monitoring. Since SEVIS is one of the first building blocks of this expansive security system, it's a gauge of how well the department can manage such projects.
For months, foreign student advisers from about 5,000 institutions have been frantically entering records into SEVIS to meet an Aug. 1 deadline for registering this fall's class. But they have been hamstrung by protests and an endless string of technical glitches with the computer system, which range from the mildly frustrating to defects so severe that some universities have been forced to close their foreign student offices for days waiting for repairs to be made.
Officials at the Bureau of Immigration and Customs Enforcement, the Homeland Security agency in charge of SEVIS, insist the system will help close holes in immigration policy and keep some would-be terrorists out of the country. But in the meantime, foreign student advisers are bobbing and weaving through SEVIS using jury-rigged workarounds to trick the system into doing what the
Now that its "Terrorist" Information Awareness you'll know they've got a bug just as the Hellfire missile smashes through your bedroom window, maybe you should change the phase to death, incineration and the hot pursuit of anyone who looks un-American...
Any sufficiently advanced man is indistinguishable from God
These technical problems are typical of premature deployments of immature software, inadequate design time, understaffing, unrealistic deadlines, etc. oh, my head is beginning to hurt! This is too much like my last 18 projects in government development ...
English -- gotta love it! / The engineers refuse to refuse the rocket until the refuse is removed from the launch pad.
Yes.
Look on the bright side, if they can screw up simple projects like this how far do you really think TIA (Total Infomation Awareness) is gonna get?
TIA Alias Search: Commander Taco
Output: Mexican terrorist. Leader of collbration site for social dissidents.
Just far enough from the truth to get somebody sued....
Let me take this one on. The reason is because many consulting companies, especially these days, will go to about any lengths to get a contract because they have been bleeding so bad for so long. Get a few contracting companies bidding on a software project and enough will low ball the estimate that even the guy who looks safe in the middle is low balling. At the same time they are putting on the little sales show and saying why you should choose them. They'll tell you all about their methodology and why because of their methodology it will be done on time, within budget, and flawless. They will show you all the areas that they are going to put an expert on and how experienced their people are. Then the contract gets signed and from the start the project ends up undermanned with not enough time or budget to accomplish it but at this point their already screwed and sales has a new feather in their hat Next, the project is rushed to completion using resources that are not always experts in what they are doing.
Just my 2 cents
Sound familiar anyone "I know you have never touched that technology before but your a really good programmer and we don't have time to find that expert we sold them"
The nuclear materials database was found to suffer from flaws in Microsoft's SQL server back in 2001. This series of flaws led to the amount of tracked materials being incorrectly reported although the actual information was being tracked correctly. The origional slashdot article can be found Here.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Don't come to the US.
Really! Can anyone else think of what kind of message these "programs" are sending to immigrants?
Come to the US, and be spied upon, harrassed.
Looks like we may have to take that statue in New York harbour down.
Since it doesn't mean anything anymore.
By and large, most programmers are total idiots, especially government contractors. When you're hiring for government jobs , you're not getting Microsoft/Apple/Adobe quality engineers applying, you're getting DeVry grads. If programming was so easy it would be unionized and low-paid. There just aren't enough good programmers running around and lots of layed off can't find a job crappy ones
All of the things described in that article are common problems in many databases, though those databases don't affect the lives of millions so drastically. There needs to be much more testing done to these systems before they are used in the real world. Errors like these would not be that bad in a small database that doesn't handle vital information, but for a massive government database this is just unnacceptable. The government needs to be more cautious with these projects.
read my blog
musings on politics and technol
I've worked in many government departments, always as an independent contractor. My experience is that government databases, and the apps that talk to these databases, are generally a mess.
I think there's a few issues involved:
- a lot of the code is written by people with skill levels that wouldn't be accepted in the outside world. I've got no doubt that there's highly skilful coders working for government, but there's a lot of duds as well and often your code is only as good as the weakest coder in the team. Furthermore, a lot of the weakest coders are the designated "experts" for legacy infrastructure, such as databases, and you find yourself having to rely on their input far more than you'd like
- a lot of government stuff has been outsourced, and generally the outsource partners seem to be less diligent for government contracts
- governments are forced to send everything out for tender, and may need to give work to the lowest bidder or face internal inquiries. In the private sector, you get to eliminate a lot of tender responses, either because you know the respondent is incompetent from past experience or the price is so low that the respondent couldn't possibly deliver a quality service; in government, you don't have the same luxury of filtering out the crap
- in the private sector, a lot of mainframers moved across to Unix, then Windows, in response to changing demand. While these people may not be expert in all 3 areas, they at least know enough to be able to hold a technical discussion with someone else from another area. In government, a lot of mainframers have been doing the same job forever, and you need to tie everything back to their narrow view of the world to get anything out of them whatsoever. You'd better not try to talk "Web Services" with these guys... There's a level of financial support and job security for these types of people in government that allows them to keep doing this, that simply doesn't exist in private enterprise
Now that I've pissed off everyone working in govt IT, I have to reiterate that some of them are extremely good. The issue is that the system doesn't work on a "survival of the fittest" system such as applies in private enterprise.
Innocent people do suffer when the GOP is "taking out its enemies".
Read, L
But from my experience software often starts spiraling out of control during the requirements process. I have a little theory that bureaucrats are the main people to blame for this as they have a knack for wanting to add software features to simplify their work and complicate the software
Bingo, that's it. I'm involved in a government software project at this very moment, and we are having a hell of a time because the requirements process produced confused, conflicted and inappropriate requirements. IMHO, it's not so much bureaucrats slapping things on (though there's that), but that the process is (a) largely conducted without the input of technical people who could provide early sanity checks; and (b) multiple organizations each with their own agendas get involved and the result becomes this ungainly mutant composite of requirements.
I know this because Tyler knows this.
I've worked on some government contracts, and in my opinion a big part of the problem is in the GSA schedule rate structure that the Federal uses for contractors. It is much more profitable for a contractor on a government project to put many junior people on a project rather than a few senior people, for the same amount of revenue. For instance, a junior developer may cost a contractor $50/hr with overhead, but the contractor is able to bill the government for that junior developer at $150/hr., a spread of $100/hr. A senior developer may cost $100/hr with overhead, but can only be billed to the government at $175/hr, a spread of only $75. Furthermore, the contractor can bill more hours of junior time than senior time under a given budget cap, compounding the effects of the greater spread. Thus, the incentives for contractors are to use as many junior developers as possible on a project, to increase the profit margin.
Unfortunately, It's a rule of thumb in this industry that a few good programmers are a lot more productive than many unskilled ones. The result is that many government IT projects are shoddily built by well-meaning but inexperienced developers who are put in that position by a contracting structure that fails to recognize the realities of the IT industry. Contractors are just responding rationally to the incentives that are presented to them.
These numbers are examples -- in fact the situation may be even worse. Federal government contracts vary in their rate structures, and many are stingier than this. It may well be impossible to bring on a senior developer as a subcontractor because the maximum hourly rate that the government will pay on a project is lower than the cost of the senior developer.
A prime contractor that I worked with staffed a large WebObjects project for the Department of Defense with a dozen or so low-paid, fresh out of community college drones. Every six months -- when a project review was due -- they would bring us on board as subcontractors for six to eight weeks. In that time, two or three of us would take the code base from where it was four months ago and bring it close enough to the required progress to get the contract renewed, and then the prime contractor would say "goodbye" and toss us out. Four months or so would pass by, with their people making little meaningful progress, and we would get a panicked call for six or eight weeks of more work to get by the next project review. (Did I mention that the prime contractor didn't pay the bills for one set of work until they needed us for the next project review? It got so bad that at one point we had to treat them as though their credit rating was zero, and demanded that payment for each week's worth of work be deposited in an escrow account before we would continue.)
By the way, this rate structure is not unique to government IT projects. Other types of government projects display the same professional services rate structure. When I worked for a (then) Big Six accounting firm as an economist, most consulting projects for corporate clients were staffed with a ratio of one partner and two or three senior managers to six or eight associates. However, the Federal government group was staffed with a ratio of one partner and one senior manager to twenty or so associates. I talked to the senior manager, and he told me that (a) the associates in the government contracting group were paid much less than we were on the corporate side since they billed out at a lower rate, and (b) the only way they could make money was to use lots of cheap associates because senior people could only break even at best at government rates.
Ya know, it'd be nice to see a GSA person squirm over this sort of thing in front of Congress some time. Then again, Congress may be part of the problem, as they'd rather generate lots of jobs for constituents, instead of a few.
--Paul
All too often, what happens is that someone either sits down and knocks out a database out of thin air (with no understanding of the underlying business needs), or a team sits down with the latest and greatest methodology and tool set and forces the process into their pet methodology and tool set (again, with no understanding of the underlying business needs). I don't think perceptive /.ers need to be told what happens next.
But why does this happen? IMHO, we have dismissed the value of generalists, those who understand the underlying business model, the people and processes involved and the technical means available. At best, most teams have a business expert, a development expert, a database expert and a project manager, who likely does not have business expertise and, hence, cannot act as a proper liaison between the business and technical sides.
The generalist, being neither fish now fowl, and being more mature/older to boot, tends to be the target of early retirement and layoffs. And yet, these are the very people most needed by both public and private sectors. They are the ones who can make sense of the customer spec and present it to the team in a fashion that makes sense. They are the ones who can explain development realities to the business side in language that they can understand.
Does this mske sense, or is it the nostalgic longing of a generalist?
I can think of several widely used systems that are backed by databases that work just fine. Slashdot wasn't built on a massive budget. Amazon doesn't have a history of "bleeding" data from one user to another. Google and Yahoo are certainly capable of handling tremendous loads.
I see several possible problems here. First, it is possible that this software was rushed into use before it was ready. Given the political pressures involved, I suspect that is part of the problem.
Second, I doubt that all of the programmers involved are of guru caliber. I don't intend to malign them. Even assuming that you have nothing but above-average programmers, when you build a huge project with lots of designers and coders, there are going to be miscommunications and some details that just aren't communicated.
Third, I would bet that this project has so much design documentation done up front that it is impossible for anyone to wrap their brains around the whole thing. This is, at best, a 1.0 release. And there are going to be design flaws in it. And the guys writing the code aren't likely to have a broad enough overview of it to spot them all. They also undoubtedly tripped over a lot of things that weren't specified up front and should have been. It is the nature of the game. But they weren't free to just choose a good solution when the questions came up.
The projects I cited at the beginning were developed by small teams with a vision of what they wanted to build. Within the constraints of the tools they had to work with and the general idea of what they were building, they were free to change the rules. They could refactor to their hearts' content. That is not going to be the case on huge government contracts.
Everything that we know about open source, agile/extreme programming, etc. doesn't apply to this kind of project.
The net will not be what we demand, but what we make it. Build it well.
"Bureau of Immigration and Customs Enforcement officials deny that, saying any glitches in the system are to be expected from a project of its scale."
Software faults will be present when the contractor involved is unwilling to or incapable of applying a rigorous formal verification processes. Individuals without formal training and without thorough comprehension of university level theorem proving practices may be unqualified to or incapable of applying these rigorous formal verification processes.
In the case of critical government systems such as SEVIS which can have profound negative effect upon the lives and futures of human beings, it should be both the expectation and requirement that the contractor involved apply rigorous formal verification processes either to the entire software system or to some sufficient subset thereof to ensure at least that data integrity is not violated.
This may involve at least formally proving correct, all operations which may potentially violate data integrity.
Of course a correctness proof will be of value only so far as the formal specifications correctly and fully describe the intended operation of the system. However, carrying out this process will itself assist in identifying errors and shortcomings present in the formal specifications.
The article also seems to suggest that this particular contractor does not have adequate knowledge of the problem domain to provide useful and timely support. Not only is this itself unaccepable, it very likely has contributed to the presence of inadequate and invalid formal specifications. Surely the federal government has had previous relationships with contractors which are more generally knowledgable of immigration policies and practices.
While I disagree with the policy which has given rise to the need for this system, attempting to satisfy this need has given further evidence of shortcomings in the development and deployment of critical software systems.
When the lives and futures of human beings are at stake, especially when this is so through not fault or knowledge of their own, we have an obligation to do better.
but have others noticed similar trend in other government information systems?
Posting as AC to protect the innocent.
A friend of a friend used to work at county goverment IT. He was responsible for maintenance of their social services database, which was an IMS system that ran on their mainframe. Out of curiousity one afternoon, he ran an EasyTrieve against the database, and discovered one street address that was receiving 17 support checks, for a total of $16,000 each month in probable duplicate payments. Other addresses were receiving smaller numbers of checks, but there were greater numbers of these addresses. He figures that there were over $1.7 million in excess welfare payments each month.
When he presented his findings to the head of the department, he was told that the important thing was that the money got into the hands of the needy, and that accurate record keeping was less important. He then tried to change the data input programs to warn the operator that a duplicate address was likely, and got shot down. He then tried to put some alternate keys on the table (there were no keys at all, it seems), and again got shot down.
It seems that the operators, while they used the terminals, felt that their paper records (stored in teetering piles at the back of the office) were more accurate, and only used the computer system when they couldn't find the person's file.
for the government to collect my personal information, I at least want it to work, damnit!
Never heard the phrase "good enough for government work"?
Nuff said.
Derek
While I don't like to reply to my own posts, an additional point of discussion came to mind:
Certain government agencies have rigorous requirements for verification of system security, why not also for ensuring and maintaining the integrity of data stored about human beings?
Tell everyone how bad the program they are in charge of is progressing? Or keep claiming success until it's too late?
One of the first professional programmers that I knew well had a very simple system for producing good code quickly. He produced the manual first, which became the spec. Then he wrote the code to do that. Very simple. Thousands of lines of good C code a day.
Sadly, he wrote code to control industrial lasers and not important things like operating systems or games. On second thought, he was working on a computer moderated pbm game...
Veteran, Bermuda Triangle Expeditionary Force, 1992-1951
In the past few years I've noticed a striking increase in cases of consulting firms delivering "less for more". The discussion usually goes something like:
PROJECT LEAD: I've found this bug in your code, and was wondering if you could clear it up before we go further.
CONSULTANT: That's outside the scope.
PROJECT LEAD: It's *your* code!
CONSULTANT: Well, I'm going to have to check with my PM back at the office to see how I bill this.
PROJECT LEAD: IT'S YOUR @#%! CODE. SCHMUCK (stay with me, this is happening in New York)!
CONSULTANT: Maybe I can revise our estimate.
PROJECT LEAD: LOOK YOU MORON, ALL YOU HAVE TO DO IS DO A COMPARE HERE INSTEAD OF AN EQUIVALENCE (Oh, did I mention the code was in Perl -- good thing too, consultant would have been on a plane halfway to Timbuktu if it had been in Java)!
CONSULTANT: Oh, so you found the problem. Good, so I won't have to charge for any more time.
PROJECT LEAD: F$&! OFF. TAKE YOUR DAMN CODE. I'LL REWRITE THE THING FROM SCRATCH MYSELF IN HALF THE TIME AND HAVE FEWER ERRORS (IT Managers take note, this is why you need in-house staff with programming skills).
Just a little morality play for IT in the 21st Century.
Maybe its just me, but I see horrible code everywhere. Anyone who has worked in corporate IT will tell you that most commercial enterprise and specialty software out there is rather horrible. From flaky conferencing software like Centra, to bloated sales and marketing tools like SalesLogix, most software my company uses is pretty shabby. We bought one of "the best" financial systems out there, and even that's crumby!
Take a look at the database structure for some of these "enterprise" applications and you'll probably laugh. Even funnier is how SalesLogix sorts columns in its GUI... it sorts dates as strings rather than numerically. Its bad enough this kind of thing gets written into a product, and even passes QA.
And these are not cheap applications either. Alot of this software costs tens of thousands of dollars. Its a joke.
They bomb pharmacies, don't they?
And don't forget the DNC either!
I honestly don't think either POLITICAL party has the moral high ground, except when it furthers their own agenda: personal gain, power, influence.
So give the partisan line a rest already.
This journalism is so biased it's amazing.
First off SEVIS is one of many computer systems controlled by the government. Pointing out flaws in SEVIS then inferring that all government computers can be flawed is a pretty bad argument.
Secondly, look at the home page on govexec.com. Most of the articles critisize the government, ex 'As bioterror threat grows federal capacity to respond shrinks', 'Agencies' electronic storage of records lags, archivists say,' and 'Pork-watcher skewers military construction bills.' This article is slightly biased.
Finally - the more complex a system is the more likely it will be to have bugs. SEVIS interacts with all univerities. I would rank that pretty high so far as complex systems go.
That's already going on.
Ask David Nelson. Any David Nelson who's had the misfortune to have to fly out of a US airport.
Tech Public Policy stuff
I'm sending a copy of that Government Executive article to anyone I run across thinking about getting a college education in the USA... to support my advising them NOT to.
There are worse things than not getting an education in the USA. Jail is one of them.
Tech Public Policy stuff
Clinton needed to look "presidential" for a day. He may even have needed a vacation from his family vacation. In any event, , and perhaps counted on the indifference of the press and public to a negligible society like that of Sudan, and killed wogs to save his own lousy Hyde (to say nothing of our new moral tutor, the ridiculous sermonizer Lieberman). No bipartisan contrition is likely to be offered to the starving Sudanese: unmentioned on the "prayer-breakfast" circuit.
The thing is to be aware of this before coding and take steps to compensate and reduce the damage from those bugs.
An excuse after the fact, the canard seems infantile.
All software has bugs. They are most certainly not created equal. Just because there is no way to expose them, doesn't mean they're not there.
("it works for me, so it's done!")
OUCH!
There is a big difference between partially working for a few things for a few people and never failing for everything for everybody. Some of the critical knowledge lies in the application area and is neither achieved readily or even necessarily consistent.
> Sadly, he wrote code to control industrial lasers and not important things like operating systems or games. On second thought, he was working on a computer moderated pbm game...
Lemme guess: given the way your post is organized, you didn't learn a thing from him.
I think that, for most managers, the analogy which would most make sense to them is that Programmers are pretty much the same as Lawyers:
The goal, in either case, is to take a set of rules -- often arcane and ancient (programming languages and operating systems Vs. rules and laws), and combine them in such a way as to allow the client to achieve their wanted ends.
The judge would be the rough equivalent of a wetware execution unit.
Once they accept the lawyer analogy, then you can ask just how reasonable it would be to expect a lawyer to accomplish a lawsuit according to a tight schedule. Although it is doable, the tighter the schedule, the higher the price (often exponentially so).
I came up with this analogy because I ended up, a few years ago, self-representing myself in a reasonably complex lawsuit (It was about 4 years old by the time I got pulled in). With a couple of months heavy research I was able to do well enough in the courtroom (before the chief justice, and later at the Court of Appeals level) to reasonably impress just about every lawyer I dealt with in court.
I achieved this by pretty much applying my programming experience almost one-to-one. I simply treated the legalese and rules of court as a programming language. Old precedents were treated much like code snippits.
If you look at old slashdot postings, I think you can see that good programmers don't have that tough a time with laws and even court decisions. I submit that it's because the paradigms aren't really that different.
Free Software: Like love, it grows best when given away.
Its just as painfull for the tech guys on the government end too. I remember one of our major outsourced projects ran about two years behind because we had a maniac at the helm of our org. Every time the contrator would show up to demo the software he would add to the list of manditory features. If only they would have had the balls to say NO!
As far as multiple organizations and composite requirements goes, I have never done that and I am less of a man for having though about it. Makes you wonder how people can be trained managers and not see that trane-wreck about to happen.
Nearly half of all people are below average
I work as a government contractor, and I would have to agree that the degree of work done there is appalling. After witnessing the bureaucracy first hand I have to disagree with some opinions expressed here. The problem is not incompetent personnel (although plenty exists) but is instead caused by the system in which government employees have to function in. Bottom line: There are too many cooks in the kitchen who couldn't fry an egg. I feel that I am a talented developer, but my potential is squandered in the government sector. The problem with government IT projects is not always the people involved, but the red tape which prevents any progression as a whole. You say "Adopt Linux!" (and I couldn't agree more) but the reality is that in the government workplace who is listening? Not only that, but who is listening who doesn't report to another squad of imbeciles? And the cycle continues... Don't be so quick to blame the individual; it's the system which needs fixing. Government jobs are as stable as employment gets. Why? Because no one rocks the boat... everyone there is only concerned with preserving the system as it exists today so they continue to receive paychecks. There is absolutely no incentive for those in management positions to encourage innovation. Simple as that, money makes the world go round.
Its because the people who make the deals are not the people who implement or use them afterwards. They've normally moved on with a golden handshake.
Yay me!
I must say that while working at a VA hospital last year I was very much impressed with their computer system.
The VA is the largest healthcare organization in the US, so they have the resources to build their own system. Contrary to what I was expecting, it is intuitive, just plain works, and IMHO blows away the stuff from Cerner or Meditech.
They have been working on it for twenty years, so it has the advantage of maturity, but even the newer bits such as windows interfaces running on Citrix are nice and stable.
Some background on the system can be found here.
Seems that is is mostly implemented in MUMPS so they score poorly in the buzzword complience department.
Anyways, I was just surprised that the government sometimes does seem to "get it".
Wait! But what about Linux?
Time to end the sarcasm for the day..
Linux is not necessarily "cheap". It requires a substantial investment of time and effort in learning how to install, configure and use it well. My time is worth a lot of money. And I have very much gotten great value for my investment of time in learning Linux. In the end, Linux is certainly a bargain, but it ain't "cheap".
I have been working in the USAF for about 8 years. 6 of that in WAN (longhaul voice and data), and 2 in Infrastructure and security. I'd like to offer another side to your story:
.NET 2003 server with M$ SQL, etc., etc.
.NET is something that, as they say, they 'have been working on for
>but it needed to be done before the end of the fiscal year
This is how it works: The USAF has a budget. Each area gets a small slice. If filters down to each office having about $10k ~ $30k for operations that year. That money has to last all year. About 20% of that is kept in reserve funds. If that money is not needed by August, we are free to spend it. At that point, we develop a wish list and try to get that aproved. By time all this happens, we have about 5 weeks to spend the reserve money.
No one in the military likes it. All our contractors hate it. If you want it changed, write your congressperson and have them change 50+ years of bad management practices...
>The quality of code would've been greately improved if we coded, say 40 hrs/week instead of pulling all-nighters.
I have spent countless days and nights working overtime. So have a lot of my coworkers. In times of exercise or, God forbid, a war, we go to 12+ hour days. 6 days on and 1 day off are common during exercises.
Contractors always make fun of us for sloppy wiring, half-assed installs, unpatched servers, etc... When new equipment arrive, we usually have a few hours to determine where it will go and when. We are usually told that the old equipment stays in place until the new stuff is operational. This leads to massive misuse of rack space. and cluttered wiring.
Also, just like your code suffers from 40+ hours, my wiring suffers when I have to spend my Saturday morning connecting a new router.
No one likes to work overtime. Your work suffers just like mine. You may lose a contract because of your bad code. People could lose lives because of my bad wiring. Let's both work harder to keep our shit straight, regardless of hours worked.
>They insisted on
This is becuase we have a very nice license with MS for their stuff. We get good support, including semi-annual "Best Practices" reviews by MS inspectors. The US Gov paid for MS tools, we should use them. If you don't like it, write your congressperson. Personally, I'd love to be able to use Squid on Red Hat. Unfortunately, we don't have the money to spend on more software licenses after we bought MS stuff.
>asked if it would work with a win2003 server as opposed to a win2k
Our upgrade paths are fixed by MS. This absolutely sucks. Our systems require specific patch releases from MS. Once they stop supporting those patch paths, we have to upgrade. Agian, if you don't like it, write your congresscritter.
>but they didn't even know how to install windows
I'm throwing a bullshit flag on this play. I find it difficult to belive that no one knew how to install Windows. In the USAF, we have a NCC department that does nothing but install, configure, and maintain Win2k servers.
There may have been an internal power play based on getting Win2k3 server training. That is an ongoing military issue. Your boss tells you to do something. If you do it and screw it up, they ask if you were trained to do that thing. If you were not trained, then you go to federal-pound-me-in-the-ass prison for working on something without proper training. If you were trained and you screw it up, then you get in trouble for not folowing the training guidelines for whatever it was you broke.
Everyone working in a military NCC can install Win2k Workstation and Server. Many of them are MSCEs or higher. They could probably install Win2k3. They just wanted official training on that product before they tried something and broke it.
>Installing a a windows server is a mountain of a task for them.
No it isn't.
>Installing
I'd rather you do it wrong, than for me to have to do it at all.
I heard somewhere that a congressmen or senator asked various agencies how many H-1 and L-1 visa workers were currently in the US, and nobody could give a reliable answer. Does anybody have specifics on this? It seems student visa databases are not the only visa tracking problems.
Table-ized A.I.
I am sorry if I am totally off base here, I didn't read the article and I never worked for the government...
It is my understanding that all code has a likelyhood of bugs. There's some industry statistic that all projects have on average some % bugs. The question then becomes, how are bugs handled?
I work for a firm where deadlines and time to market are the most critical thing. By the time our competitors are doing their requirements document, we have our prototype out. Of course, the code suffers and is rather mish-mashy because of that. I would imagine that with government contracts, deadlines are key and the same "time to market" mentality dominates. However, here's where the difference lays.
If our version 1 has bugs that affect our users, we better come out with 1.1 FAST, or our competitors are going to start gaining customers. We know this and it's important to us, so any bug that affects the user is given critical priority. On the other hand there are bugs in the code, probably some of which we know about, that have been around for 20 years. But since they don't cause us to go out of business, we don't divert resources.
On the other hand, the government isn't going to lose customers, and neither is the contractor (his job is done, he got paid) So while deadline pressures may be the same in both cases, the priority of maintaining and fixing code is higher in the competitive business world than otherwise.
Ecce Europa - Web Design for Business
I'm amazed that I haven't seen a single poster bring this up yet so...
The key to quality software; flexible, extensible, fault-tolerant, maintainable, and all of those other adjectives that 'good' software is supposed to have is very, very simple.
It's called Unit Testing. It's not brain surgery. I have worked on several medium to large scale projects (500k-3.5m lines) in several languages and environments, and I've yet to bomb one hard using this methodology, despite the usual client shenanigans.
Every time I write a functional subunit, I start by writing a series of tests (based on the spec, hopefully) that define 'doneness' for that subunit. Every object in the system has it's own set of tests. The test harnesses are chained together, so I can hit a button, so to speak, and run all of the hundreds to thousands of tests at once.
Whenever I check in new code changes, I run the test suite. If a test fails that previously worked, then I broke something. This plus good OOP practices (low coupling, high cohesion) allows you to make changes on the fly without the kind of 'The Money Pit' syndrome (fix one thing, another breaks) that is described in the article.
I am certain that the system in question was NOT developed with these methods. Most development organizations that I come into contact with pay lip service to the concept, but don't want to spent the perceived extra $ up front. The thought of all those developers writing TESTS when they could be writing CODE scares the willies out of them. But it pays for itself. It really does, every single time. Don't tell your boss, and try it on your next project. This is old news - google has a ton of info on it, and there are some good but unnecessary books also.
In the dot.com glory days, we had a huge system, running several hundred transasctions per second on a geographically distributed system of clients. We made fundamental architectural changes without a hitch, switched servers live without a hitch etc, and made a zillion little changes, all live, and all without a hitch (well, other than really stupid human errors, like locking out the client upgrade system with a bad password... oops). We had zero budget, and 2.5 developers. Unit tests are the Way, and if any company that doesn't mention them in your first meeting, run like hell.
So quit your job, pack your bags, and move on out to snow country!
I've read article and many of the replies here. Several /.ers definitely describe flaws in government contracting processes and hiring practices that I've experienced, but I think they are missing the point. I think there is an additional, more fundamental flaw, that has been overlooked - or at least didn't get modded up high enough for me to see - maybe i should go trolling for a different set of opinions :^).
My experience tells me that the problems begin when we fall into the trap of trying to solve problems with a reactive mindset instead of a proactive mindset (proactive being favorable). We allow daunting problems and/or a need for revenue to back us into a corner time and time again and every time we are forced to hack our way out. Some of that is just old-fashioned survival, but a lot of it can be avoided with deliberate forethought, planning, discipline, and a commitment to quality and detail.
Avoiding clusterf*x has to be an institutional effort, whether the institution is a huge goverment agency or an tee-tiny, independent software shop. Everyone in the organization - operations, sales, IT - has to be on board with the policy that "if it's worth doing at all, then it's worth taking the time to do it right...the first time." I said "fundamental" earlier b/c that has to be something like lesson #5 in life's little handbook - we all heard it all too often when we were kids, we know it's true, and yet now we don't pay it any mind.
I do think the failure to heed that simple maxim usually starts in business development and snowballs by the time it gets to IT, but it really goes both ways. Everyone has to be responsible for maintaining the discipline required to produce quality.
What happened with this system is everyone involved got themselves all in a panic like a drowners who not only won't let you save them, but pull you under too. It's understandable given Sept 11, but "undertandable" and "right" are two different things. Legislators threw money at a situation they didn't try to understand. Deal-makers when after that money, promising to solve problems they knew they didn't understand. Developers enabled deal-makers by claiming to understand. No one took the time to do it right the first time.
P.S. It doesn't have to be this way.
Here's a quick bulleted list of items that I see as causing problems with the government projects I've been involved in.(yes, I've been doing reports for management waaaay toooo muuuch )
/. to me while I sat by the pool. I am trying to do stuff, as I can, so that I can feel better about the projects I work on.
;-) get more competent people on the project in order to recover from past errors that are really biting us in an um, bad place. Progress is being made, but man, it is slow. (Think glaciers and you'll get an idea.)
1. Requirements change over time, but inadequate funding available to really redo the design
2. The software was developed and designed for a 4 year life cycle and unfortunately we are in our 5th/10th/20th year of maintenance
3. Old software is compared to brand new commercial apps and their features, which didn't exist when the software was originally written. (Think of comparing a telnet, ASCII based application to a GUI app. And yes, ASCII based systems do still exist and are used for real work)
4. Use of cheap, inexerienced people in order to make a higher profit margin
5. Unable to charge a per usage/license/maintenance fee, only able to charge for the development costs up front, so potential profit avenues are reduced.
6. No integration planning on the government's side due to lack of expertise, politics or not realizing that different projects should be coordianted among contractors.
7. Eye Candy tools sold to higher ups without adequate technical input into the chosen packages.
8. "We've never done it that way before" uttered by company management, customers, developers, etc, which leads to long involved discussions and training basically as to why new Method A might be better than old Method B.
Now, the really, really hard part, what to do about it? I dunno, if I knew that, I'd be getting paid a lot more money then I am now and I have my underlings read
Myself, I am working on educating people in our company about other ways and means to make money from software and trying to insidiously
You guys have only a vague idea of how government works. Or if you do know, you have had vastly different experiences than I. I worked for the Dept of Revenue and Finance for a state government and this is exactly how things worked. Each government body, Natural Resources, Transportation, ect, ect each got a set budget based on how much they spent the year before and how much money the senate/governor allotted them from the total state budget that year. The big deal in our agency was to spend ALL of the money that was allotted to us by the end of the fiscal year otherwise next year we would lose what we didn't spend and our budget for next year would be shrank to the same ratio as what we spent the year before. Our budget would actually decrease if we didn't spend the all of the money in our budget!! Logic would tell me that if you can justify your budget but can manage it well and run UNDER your budget you would be REWARDED not the other way around. I digress. So, anyway, at the end of the year the director of the project would tell us to make a "wish list" (he actually called it that) and we got to buy pretty much anything we wanted, including contractors, and do anything we wanted. So, need a PDA? You got it, a new workstation? Got it. You name it, as long as we didn't deplete the pot we got it. Kind of crazy how the system encourages waste like that but it does.
I just live here....
When janitors by day get programming contracts sometimes they get confused. That is the origin of the adage - Garbage In, Garbage Out.
How is it you ask that janitors or other unqualified people get these programming contracts? Not to slag janitors ('cause it is nice clean work)
I can think of only one reason. Kickbacks come to mind...
From excellent karma to terible karma with a single +5 funny post...
Remember that the contractors are being told what to do by government employees. These employees have legacy technology, legacy methods, and have "always done it this way before". Many times the contractor will point out (and complain within reason) about decisions being made, but at the end of the day, they're just a consultant, and if someone more political can politic a worse solution, then that's what gets implemented. Been there, done that. Interesting stories to tell.....
The result is that every once in a while a random student is arrested, delayed and/or deported because of a glitch in the database. As was mentioned in the article, students weren't too worried before SEVIS came into service, but now, the government is acting like a rampaging despot -- but they get to blame it on an opaque, mallfunctioning computer system. What more could they ask for?
Now, if the government wants to deport a student for (say) calling Bush a moron, all they have to do is induce a glitch in SEVIS and then have the student stopped for (say) speeding. One quick look in sevis, and they're in jail or on their way out of the country.
This whole randomness aspect is what has the foreign student body nervous, and it's almost impossible to pin down. It seems like a pretty good opposition supression system.
. . . But as a corrolary to Occam's razor says: "Never blame on belligerence what can be explained by simple stupidity." -- and we're talking about the government, after all.
Free Software: Like love, it grows best when given away.
Maybe the bugs where put there by a malicious programmer (slashdotter?) who disagrees with all this tracking nonesense. No better way to sow the seeds of discontent.
I was talking about something similar to a colleague about how I wouldn't be a part of a research proposal for the DoD because I fundamentally disagree with concept of a Big Brother State. He said that he actually new a researcher who had taken on defense-funded research projects in the past and just tanked-it on deliverables because of his own ideological sentiment. Not only was the research rendered innocuous, but it tied up government funds that could have been used on other more incideous projects.
Not an approach I would champion, but interesting nonetheless...
All your base are belong to us!
Remember in late 2001 when the US Department of Interior was ordered by the court to take more than 100 of their web servers offline due to abysmal security? Hired white hats were easily able to gain access to the US Indian Trust database and found no security measures or even audit trails in place. Worried that this could be contributing to the agency's continuing mismanagement and loss of allegedly billions of dollars belonging to Native Americans, Judge Royce C. Lamberth ordered the DOI to "immediately shut down Internet access from any computer, server and system in the department that has access to individual Indian trust data."
The defense counsel noted that the fact that they took down over 100 mostly unrelated servers "...just shows you how inept they are. They don't even understand how these systems relate to each other so they just pull the plug on the entire system."
And now last month they were ordered to disconnect their servers again after refusing to let a court-appointed special master test the security measures they've supposedly put into place since then.
Sounds like an endemic problem for government agencies, at least at the federal level.
The common thread is typically not an engineering failure, but rather, egos, politics, and ill defined requirements.
Like many things in life, there is usually no big conspiracy to screw things up, they just get that way "one good idea" at a time (e.g. The road to hell is paved with good intentions).
This issue is a bit more complicated than you think.
SEVIS, like most web-oriented applications, consists of a browser form based interface, application logic, and a database. There's also an XML-based batch update capability. It's all reasonably modern and buzzword compliant.
Looking at the manual, the main problem you would expect to see is rejected transactions. Basically, the system won't accept transactions which indicate something that violates immigration policies. That makes it inherently obnoxious.
But on top of this, apparently the database back end has problems too, since data is reportedly "bleeding" from one school to another. That's not something that an end user (a Designated School Official) can cause; school officials don't have broad enough access rights.
Any major error in the data for an individual tends to prevent transactions which could correct the error. Designated School Officials can only enter transactions which are consistent with the existing data. Anything else requires approval from what used to be the INS, and is now part of Homeland Security. So that creates a huge load on an organization that's always been behind on paperwork.
All entry points to the US can access some of this data, and it's used to decide who gets in, who gets turned away, and who gets sent to a detention center.
The whole system is very Russian - great power over individuals, exercised ineptly.
"You want a vision of the future, Winston? Imagine a boot stepping on a face for eternity" - Orwell.
Well, in an African/Middle East country (no names), the prime minister actually signed a multibillion contract with MicroSoft to supply Software for govermental reasons. Here is what i see:
/*now i can sleep in peace*/
1) The deal was a total ripoff, technically speaking the e-gov as we may call it as WAAAAAAYYYY behind.And the money paid is worth much more than what is being supplied.
2) The system is not reliable enough, so gov. personals still do things by hand (yes, you can stand in a que of 50 persons just to get one lousy signature, or pay a 50 cent fee).
3) From the looks of it, they still have a very long way ahead of them till they get things in shape.
4) Finally, this deal was done almost 4 years ago, and this is the current status, how much time do they need..
Thats what i call a rip of, personally i see the problem as "unqualified technically enough ppl take technical decisosns, thinking he is the wisest ass on earth", putting the tax payers money at stake (i bet he would rethink the decision if it was his own damn money.)
The lunatic is in my head
no text
I don't know if this is the same as the US but this is how it works in some of Australia's largest government organisations.
They go for the lowest Development/Support cost out of all the tenders. That is if it makes to tender rather than just been given to a existing contracting company.
This activity then leads to clauses in the support contract that state when things going wrong or there needs to be a change to the system the supporting companies charge a arm a leg for the change. IE $60k AUD for an automated ftp job that took 2 days to write and test!!!! This then leads to the Government Deptartment/Orgisation often deciding that the bug/extra requirement is not worth the big $$$$ to fix, hence you end up with a buggy system that is relied apon. Often these bugs are quite small but the contract makes them too costly to fix
It said "windows 98 or better" so I installed Linux
--there are very few "pure" local governments in the US any longer, and the REASON is that the big federal dollar carrot. Our federal tax rate is extremely high, and most local governments are forced to accept BOTH federal money back to them,after first having a significant portion skimmed for the federal bureaucracy, and always with strings attached "do it this way or no money". that's control, which means the feds control everything. For example, several states have voter referndums, and have voted to allow medical marijuana, but the feds say no, so it's still "illegal". It's pervasive, and crosses into all avenues of governments, from roads to schooling to you-name-it. It's even gotten down to miniscule and obviously supposed to be local zoning, you'll find a lot of it follows federal guidelines now, and most will be changing over. We have a theoretical Union made up of soverign States and the districts and territories,etc as you outlined, but because of how they are funded,and a plethora of federal laws, regulations, edicts, findings, orders and directives, they all (mostly, conversationally/generally speaking now) are just agents of the feds. I think you'd have a hard time finding many local government exceptions to that, state or county. It's an illusion, just like born-with constitutional "rights", independent judiciaries, the jury system acting as a check on bad laws, "real" specie backed money, and free and honest elections.
According to the article, several completely innocent students have been jailed due to malfunction of SEVIS. In my view this is absolutely unacceptable; the responsible people are criminals and should go to jail instead. Since arrested students never know if they are scheduled for some years in Guantanamo, they might even resist and people get killed. The US can and should follow their constitution.
Here in Portugal, there was a glitch in the database system handling social security information. Data was lost, it took a bit over a year before I finally received the welfare I applied for when I lost my job. Like most people, I didn't have a years' worth of financial backup. As a programmer, I tend to blame this kind of situation on poor design. Quick hacks are often rewarded over proper solutions that take a few weeks to implement. Legacy systems, especially big old complex ones, have a tendency to evolve to a point of being almost unmaintainable. Backup tapes are used to solve the unreliability problem, instead as of a last resort to reach for when everything else fails. Governmental systems are rarely stable. Every few years, a new government is installed and laws change, programmers are replaced, and soon little is left of the original design. There is too much risk in starting over with a fresh, proper design, so the status quo maintains. Once in a while, a backup tape is unreadable and disaster is inevitable.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
The reason the same gang of idiots keep getting work from the government is that they're the only people willing to bid on it. Government procurement is so bizarre that unless you have a team of specialists putting the bid proposal together you have no chance of getting it. Every large company I've worked at had a special "Federal Systems" division whose job it was to deal with that sillines.
The problem with most of these projects seems to be the lack of that standard incentive scheme most companies run which goes:
Do job really badly -> lose job
Here in the UK, government IT projects go vastly over budget and fall apart time after time, yet the same companies get hired again and again to do the contracting.
One way of doing carry forward in a controlled way by allowing carry-over and combination with the new budget for a designated purpose, i.e. I combine the remaining $1000 from this year with the $2000 from next to buy that new router next June.
See my journal, I write things there
Well ya-know,
... specialist with little or no idea of commercial market technology application/architecture requirements.
.... .... ..., I suspect (no proof), as above .... Well supposedly if it could not be done correctly, competently, successfully, ... then the OEM/OSD vendor and government contracting folks should state in the R&D SOW requirements (or someplace in the signed contract) potential problems and possible failures to meet expectations. Everyone involved are supposed to know what they are signing, the requirements, and limitation. So, if you don't get what you paid for, then whose fault is it in a court of contract law. ....
... just like the trickle-down tax-cuts are already helping US.
..., Okay.
Low-bid may not be the problem, competent selection of appropriate technology products and capable competent vendors as a low priority to cost may be the problem.
I mean, technologist (Tech-Advisors) are not the government employees on the selection/review panels. They are Legal, Contracting, Acquisition, Logistics,
As a matter of public policy (and taxpayer money) is this level of non-functionality to be expected in these sorts of projects? No, I suspect (no proof), as above
Is the contractor just ripping off the taxpayers with bad code? Yes, I suspect (no proof), as above
How hard is it to write software like this that works? Not very hard
The article focuses on the SEVIS database, but have others noticed similar trend in other government information systems? Yes/No I suspect (no proof), as above
REMEMBER: It appears the game is to maintain government management levels, while reducing worker-bees and pack-mules (contract out wherever and whenever possible). This will simplify personnel management, provide vendors to workload and blame when problems and cost escalate, maintain the military industrial complex, and funnel more tax dollars into the growing economy at an appropriate level
OldHawk777
Reality is a self-induced hallucination.
So, never take what anyone says very seriously
Unaccountable leaders are masters, and unrepresented people are slaves. How do US and EU fare?
I've been a programmer for the government for 7 years now, 5 of that has been at one agency. There's several problems that arise in a government programming shop, many of which are beyond the control of the programmers themselves reguardless of proficiency or work ethic.
We can't really use the best tools for the job, we use the tools that someone on high has mandated as the agency standard. I've been in more than one situation where we're using the wrong tool for the job, but we have no choice. Also, on the rare occasion that new and/or appropriate technology is used for a project, training to use that technology is usually given after the project is due to release. Management pounds out "ambitious" deadlines to try and win contracts, which obviously bites their programming staff in the ass. I recently worked on a project that had about 5 months of design development and test time, the developers caught wind of it 2 months into the schedule, and we didn't get all the requirements until 3 months in, which left us 2 months of actual design programming and test. We had no one with experience in any of the tools, technology or language, not even the project lead. Fortunately we had a great team. I strongly believe that any skilled set of commercial developers would have had a hard time in the same situation.
Not all government programmers are inept, and those that are inept are impossibly difficult to get rid of. There are very intelligent, self teaching, skilled programmers in my shop, and for every one of those programmers I can point out 3 that need their hands held while they write their code. The pay in the government sector doesn't attact many highly skilled individuals. The skilled people in this sector either get skilled after they've been hired, are in dire need of work (which means they'll leave as soon as something better comes), or they feel a patriotic duty. I have a sneaking suspicion we don't have a lot of highly skilled and patriotic tech geeks. Anyways, as I was saying, those inept folks are hard to get rid of. You have to have piles of paperwork and documentation proving that you've tried to rectify the situation, and then the person working on cutting the fat usually comes under fire by the union. Essentially the boss gets skylined and comes under intense scrutiny by the union to decide if her/she is just "gunning" for the poor employee. It's easier and less stressful to just try and get another shop to take the useless individual. We've got a couple hot potatoes in our shop, and unfortunately we were the last group to discover that they are useless, which means we get to keep them.
If you want things to change, get involved, take a healthy pay cut, work under stifling beaurocracy with inept slackers and make a difference!
It is hard to write any software without passion I think. The stuff I write for myself is much better than the stuff I write for the man.
M@
Krispy Cream is people
There are also a lot of contractors that like to just milk GOP projects for money without actually doing the work. The result is money wasted. This is probably where your $20k/toilet seat figures come from. Projects now require so many controls on them to make sure this doesn't happen that the cost inevitibly is higher just to perform the controls.
Yes, there are politics at play, and if you have a good product everyone will want it [to meet their slightly-to-very different needs]. However, as an engineer down in the lower trenches I can tell you there are technical issues, and they are frequently not the GOP's fault. They are the fault of extremely untested crap that contractors try to pass off for securing continued funding on a time (not product or milestone) basis intake. Enter the politics at this stage...
I'd like to provide an example from a project I have been working on, but i'm afraid of aformentioned politics...
I work in IT for a municipal govt.
The problem with government is that you don't have a unifying purpose such as say a profit motive. Each government agency is like a separate business with different agendas. Getting them to work together for a common good (like say the tax payers) is very difficult.
One of the biggest problems in government is the notion that you can't get rid of anyone. We have no policies that say you cannot get rid of dead wood, just an organizational lack of intestinal fortitude to do just that.
The other big problem is that many govt workers are just hanging around until they can get a state or federal retirement. Great for them but not so great for the organization or the citizens.
If you are lucky you will find some very good IT people in govt. If you are unlucky you will find zombies waiting around for for retirement with a scope of reference that revolves around the last time they actually could do anything (usually back in the 1980s or early 1990s). If you are really unlucky your PHB will get all of their information from industry fish wraps or from Gartner reports...you wouldn't want to say think for yourself or get informed...
The funny thing is that govt exists to serve the citizens. I almost never hear govt employees talk about the citizens or look at what may be in the citizens best interests. Doing things better and cheaper seems to be in the citizens' best interests. Chasing technology for the sake of technology doesn't seem to be. Working together across the organization seems to be in the citizens' best interest. Building empires and conducting turf wars to protect and expand the empire is not.
1. $300k to manage this? The bare minimum to use this system is: an Internet capable PC with a printer and Adobe Acrobat for each DSO (Designated School Official) and the Primary DSO - which they probably already have, an 11-character field in your database to store the SEVIS ID of your student, a pair of programs to generate the compliance reports. 2. Most school officials that I know of immediately went into a defensive panic mode as soon as SEVIS was renamed /introduced (INS has been working on automating this for years but 9/11 accelerated the process and added need for additional tracking). Most of them preferred to debate and defer the issue rather than working with it to make things better for everybody. The whole I-20 generation process really isn't that bad...
3. There are students that have complained but the majority of them realize that now there is a simple list of rules to follow and the flow of paperwork has simplified somewhat. They come in, study, finish, then either leave or get naturalized... There is nothing in the SEVIS interface (web or batch XML) outside of data to verify who you are and what you are doing in this country is what you said you were going to be doing with the funds you have allocated to do so.
That entire story smells of sensationalism (find problem, exploit the minority that have the problem to make the problem look bigger than it really is, etc...).
Personally, I hope this system ends up working. It may take a while, but a centralized system is a better alternative to thousands of schools writing their own different/incompatible systems.
-Phil
Shoot questions, first ask later...
Cost is rarely the only criteria on large public sector contracts
..delete everyone's file and create new ones"
With EDS's track record of failure and their financial instability,
it is hard to understand why they still receive these contracts.
This application could be developed successfully on any modern platform
EDS is most likely responsible for most issues raised in the article if they were the main contractor.
The "bleeding" issue is the most serious issue raised.
If the system cannot maintain data integrity, then it is worthless for
the main requirement of monitoring students who may be a security threat.
The "bleeding" issue is probably caused by
failure in database design (EDS resp)
or less likely a serious performance issue
"to create a record for the newborn child..
failure in database design (EDS resp)
"Before extra capacity was added to the system, several student advisers
logging on at once caused SEVIS to slow dramatically"
Flawed operational architecture even if sizing, availability requirements were poorly defined as solution did not scale (EDS resp)
"help desk for long periods, only to be offered a solution that turned out to be wrong"
Failure possibly due to organization, staffing level, poor documentation, inadequate handover to maintaining organization to provide support at the required SLA (EDS resp)
"..several others, she says. "It begs the question,
how rickety is [SEVIS] if you can't do upgrades to it?"
poor maintenance procedures, no regression testing (EDS resp if maintaining)
"The Bureau of Immigration and Customs Enforcement
has yet to name an official to oversee SEVIS."
Inadequate handover to maintenance organization (joint failure)
There's a classic story that the CS professors at Ramapo College tell about a contractor making a database for MVS (our DMV). This story is always told in the Data Structures/Algorithms class right before you get to searching algos.
Basically the gist is that MVS hired a contractor to create a network-based databasing system to track all the New Jersey drivers so that every MVS location could have up-to-date records all the time. The test system worked briliantly. Off a local machine and a database of over 500 people, it was pulling things up instantaneously. The MVS ate it up.
Weeks pass, the thing gets installed on all the computers at the MVS offices, there's a central database built of all the drivers, and everyone is connected to it through either modem or ISDN (this was like 2 decades ago).
Turns out that the contractor used a linear search algorithm (and a wasteful one at that) to do database lookups. Even looking things up by SSN (which should just be a hash table lookup) were handled through this linear search. Everything took hours... Just recalling a record would take like 20 minutes while you sit there and wait.
Whether it's true or not, they use it here to emphasize the importance of sorting and searching algorithms.
-=-=-=-=-=
I'd rather be flamed than ignored.
That's very interesting, but I don't see how it is a rebuttal. You describe a completely insane, wasteful and dysfunctional system where your opinion counts for nothing. You don't have to take it personally, it's obviously not your fault, and you seem like a smart and competent guy. Sadly, smart people in a dysfunctional system will produce crappy results.
I find your trust in the political system touchingly naive, but that's a whole other rant.
This is what happens when you out-source! Companies like EDS will staff a bunch of underpaid H1Bs (H1-B visa holders) and RCGs (Recent College Graduates) and then set an unrealistic deadline. This results in the team not having enough time to complete the application and/or do testing.
Why should management care? The project was probably delivered on time and within budget. It was a success! They saved the taxpayers' money... yada yada yada...
-- a former consultant who worked at an outsourcing firm
Let's just back up /etc/constitution and /etc/bill_of_rights and FDISK the system.
Our web pages are already colored with hex anyway.
-Libertarian secular transhumanist
"to just milk GOP projects"
...
GOP? Did you mean GPO? Or GSA?
Or did you mean to say that the GOP milks the contractors?
No matter.
The sad truth is that the GOP milks the Contractors which milk the GSA, which does the unenviable tast of implementing the rules created by the GOP which is milking the
Of course...
The same is true of the DEM (anti-GOP), so its all a moot point.
You could easily have achieved third post, but instead you go and rant about other people's failures. LOSER.
People are willing to pay for quality when they see it
Anonymous Coward: a poster so terrible they have to post anonymously!! Like I am! And they're still struggling to win Score 5: Funny on slashdot, which they'll never achieve as they've never been to kuro5hin to be properly educated in trolling. People are willing to rate for something funny when they see it, and so far, it seems that no one has bothered to point out how tragic the poster is.
I've seen your nickname too! And man, you post so much crap to slashdot - and you do it 24x7. You must live, eat and breath slashdot.
Yours humbly,
Ta bù shì dà yú
XML is like violence. If it doesn't solve the problem, use more.
Congratulations: you appear to have made up a new word.
The English language doesn't have enough words, now we've been given another one. Oh lucky day!
I had other photo id's, including a student id with my SSN. The officer said that the situation happens all the time. I was amazed that they couldn't reliably use anything but my driver's license number to look me up.
----
I owe the government $3400 in taxes. So I sent them two hammers and a toilet seat. -- Michael McShane