Slashdot Mirror
Spreadsheet Blamed For UK Rail Bid Fiasco
First time accepted submitter Bruce66423 writes "As a sometime computer programmer who was always very sniffy about the quality of the stuff being knocked up by amateurs aka power users, the current claim that it was a messed up spreadsheet that caused a multi-million pound fiasco is very satisfying. 'The key mechanism... mixed up real and inflated financial figures and contained elements of double counting.'"
So a sometime programmer likes to think he is better than people who don't know how to program at all? As a fulltime programmer (which apparently puts me higher in the hierarchy) I think that is just a bit silly.
For those of us who speak English as a first language, here's a translation:
"A messed-up spreadsheet caused a multi-million-pound fiasco."
I think it refers to government financing for some sort of rail transport project in England, but I'm not as sure about that part.
I suspect there are better ways to get all of the above, but that is irrelevant. Does the submitter think that people who use other programming languages do not make such catastrophic mistakes? I think history says otherwise:
https://en.wikipedia.org/wiki/Knight_capital
https://en.wikipedia.org/wiki/Ariane_5_Flight_501
Bugs can be costly, regardless of whether those bugs are in spreadsheets or Ada programs.
Palm trees and 8
"Create Huge Row"
Excel's new slogan: "The backhoe of the financial sector."
#fuckbeta #iamslashdot #dicemustdie
If spread sheet says 2+2=5 , what do you trust? The spread sheet , which has huge amount of computational mathematics behind it, or your eyes which has, well you , behind it. With the state of mathematics education, I'd trust the spreadsheet. Besides the mathematics may have changed since the last version. What was Edgar Allan Poe's law? Or was it Nathan? Let me ask the computer..
A "rail franchise" in the UK is a Government-granted monopoly to run services over a particular rail route for a set period of time. The monopoly is awarded to the Train Operating Company ( TOC ) that basically bids the highest fee.
Rolling stock is provided by the Government, too, in conjunction with the TOC.
Notionally it is possible for Open Access Operators to also operate services over parts of the same route, but this flies in the face of the cushy relationship between Government and TOCs and so generally isn't granted.
This is the state of rail "privatisation" in the UK today.
Spreadsheets -- well, Excel really -- are inescapable in business.
I know personally of complex multimillion dollar deals in the oil and gas business involving buying and selling entire refineries and gas pipelines where the numbers were all worked out on a spreadsheet.
The insurance industry lives on the spreadsheets put together by the actuaries.
The only consistent reason I've seen for Excel users will give up their rows and columns and have bespoke software created is when the dataset gets cumbersomely large. A secondary reason is when the kinds of calculations needed can't be cobbled together with Excel's function and macro tools. Even then, it's not unheard of for users to demand summary/aggregate reports and analytics that they then copy the numbers from into their spreadsheet to do their scenarios.
Just keep in mind the next time you hear about big money moving around in some deal -- somewhere someone probably had a pivot table for that.
Guilty! They should promote her, like the Americans do.
“He’s not deformed, he’s just drunk!”
"As a sometime computer programmer who was always very sniffy about the quality of the stuff being knocked up by amateurs aka power users,
Get over yourself. 'Real' programmers make mistakes too, sometimes mistakes that kill people.
The problem seems to be an accounting/design problem, mixing different kinds of data, not a problem of programmer skill. Since I don't understand accounting, I will not claim that I wouldn't make the same mistake, and I've been a professional programmer for years.
"First they came for the slanderers and i said nothing."
This is the state of rail "privatisation" in the UK today.
Just to expand on that, "Privatisation" is a UK concept that seeks to combine the efficiency and value for money of government with the social responsibility and long-term vision of big business.
It's what you get if you spend so much time flip-flopping between socialist and capitalist governments that even the parties forget which is which.
In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
The solution will be to require certification in spreadsheet design, creation and use. All others will be issued pads, pencils and adding machines.
Certification will require six months of coursework, an eight hour test and a fee.
Get all those stupid computers off people's desks! Things were much better when you had to go to a programmer in order to get software to do anything!
And (not incidentally) it would eliiminate all the productivity that's lost to Slashdot!
Your sarcasm is unwarranted. This is a nice story for us programmers because it's just the kind of anecdote that makes businesses seriously consider hiring more professional programmers. Nobody is suggesting you need custom software for everything.
"Create Huge Row"
And the Daily Mail dedicated several Columns to it.
Nobody is suggesting you need custom software for everything.
But it would be nice for the pay check if they did...
Pro's do their own math when its their own money on the line.
Every time...
The UK Government is proud to present Omnishambles v1.0
Years of underfunding, lack of training and a truly bizarre employment policy, coupled with the repeated use of IT support companies with records for incompetence and failure as long as your arm, have finally paid off.
There is nothing, literally nothing, that the UK Government can do without the result being a massive, over budget, cock-up.
and you, god like programmer,could have done so much better cause you can shit out a widget on command?
might want to get over yourself Mr. Ego.
The calculation could have been done on paper / blood on a wall / notches on a stick
and
carried out by throwing dice / abacus / mental arithmetic
by
morons / normal people / genii
If the process was not validated and the results were not checked, why is anyone surprised when it is wrong?
Some areas can be defined as right or wrong by people with good minds and strong opinions - games
Tax and Financial software not so.
At some point in the process $product needs to be validated using $external_process by $people_who_should_know
I am betting this did not happen.
The submitter is suggesting — no, make that "claiming" — that spreadseets are dangerous because they allow "non-professionals" to program. Now, spreadsheets are the original "killer app" for PCs. Huge numbers of CP/M-based systems were sold just to run VisiCalc, and this probably had a lot to do with IBM biting the bullet and getting into the desktop computer business, with results that reverberate to this very day and the forseeable future. Alan Kay, one of the inventors of OOP and GUI, cites spreadsheets as a tool that turn ordinary users into programmers. Attack spreadsheets, and you attack the entire idea of user-centric programming. The submitter's attitude is reminiscint of the pre-Woz era, when you had to negotiate with your programming staff to do even the simplest computing and programmers were known as "High Priests of a Low Cult". There's a lot of room for sarcasm here.
More than I thought to use. I also could have been sarcastic about the assumption that "hire a pro" is a magic bullet for avoiding fuckups. Really? "Professionals" never make stupid, multimillion-dollar mistakes? Get real.
"Have somebody check your work" is the applicable lesson here. "Hire a pro and you're safe." is just bullshit.
Who is John Galt?
I agree wholeheartedly with this. I would add that the submitter's comments are an example of the phenomenon that the barely-competent tend to delight in the failings of others. Those who are more skilled can recognise that everyone makes mistakes, and don't need to dwell on the mistakes that others make in order to prop up their self-esteem.
Get all those stupid computers off people's desks! Things were much better when you had to go to a programmer in order to get software to do anything!
And (not incidentally) it would eliiminate all the productivity that's lost to Slashdot!
Your sarcasm is unwarranted. This is a nice story for us programmers because it's just the kind of anecdote that makes businesses seriously consider hiring more professional programmers. Nobody is suggesting you need custom software for everything.
And you've missed the point.
It is just as likely that the accounting model was incorrect rather than the implementation. If the spec is wrong (or unclear or incomplete) then you will get garbage out whatever tools you use - excel, c, c++, c#, haskell or real programming in Fortran (assembler if you must). If you don't test and cross check your outputs then you risk not spotting implementation mistakes - whatever tools you use.
Essentially, someone's built a wooden shed the wrong size and in the wrong place vs. the plan - and now they're in trouble for it. The submitter is saying "that's what happens when amateurs use wood and nails to make buildings, if they'd just hired us steelworkers to do the job properly using steel, it wouldn't have happened".
(shrug) sometimes people push their spreadsheet way beyond what it was ever meant to do - either because they're in a hurry, or because they don't know other tools exist.
As a co-op student 15 years ago, I wrote an indoor cellular-propagation simulator in Excel. It was terribly slow, but it worked and printed pretty pictures and got us a half-million-dollar contract that we would otherwise have lost because the customer didn't think my manager's hand-drawn sketches looked techie enough.
I'm not as think as you confused I am.
(shrug) sometimes people push their spreadsheet way beyond what it was ever meant to do - either because they're in a hurry, or because they don't know other tools exist.
Agreed. But isn't that true of any programming tool? Or any tool of any kind?
A spreadsheet application is a tool designed to be used by non-programmers. Someone can lose millions with a paper spreadsheet -- it's just easier using a computer.
Essentially Spreadsheets have 3 problems in this context:
1. They look simple enough so people who should perhaps not make hard decisions believe they can out source their decisions to a spreadsheet.
2. They are fairly opaque. It's hard to look into the structure of a spreadsheet. All you see is the data. When you are programming what you see is the code. You see what your program is doing. Even if you look at the formulas inside your spreadsheet, you'll have variable names like A15. That's not particularly readable.
3. They are hard to maintain. Imagine you having 15 values to add. Now you want to change that program to add up 20 values. In normal programming languages that is just changing one value. On a spreadsheet you now need to change all instances where that array is mentioned. You have no chance of using a constant or searching for "15". It is very likely you miss one thing you should have changed.
And there's another problem which isn't relevant here, but you should consider none the less when thinking about spreadsheets.
Spreadsheets lock their data in complex formats. If you are lucky you can get the data out as some sort of XML, if you are not, it's a binary blob. In any case, should you ever want to access the data with your own programme, you'll probably need to spend more time on getting the data than processing it.
As far as I know, the original Visicalc did some of the things right, so it actually was a moderately useful tool. For example as far as I know, it didn't allow you to scroll. That way the number of cells was limited to the amount of screen space you had... which was not a lot on an Apple II. Nobody would ever get the idea of having 10k data points in a Visicalc sheet, it simply was not possible.
That argument is like saying we should build houses out of loose straw since houses made out of stone also collapse.
You are comparing something as trivial as some business decision with something that is, literally, rocket science.
What's your account name?
*clicketyclicketyclicketyclick*
Sorry, we don't have an account by that name.
Now, you were saying what again?
BOFH
I was promised a flying car. Where is my flying car?
Why should the government asses the bid? I'd say it's up to the company to come up with the money and take their losses. They most likely have other routes that are profitable to compensate. Any wise board always calculates a percentage of their budget as "unforseen" and this is just one of these cases. Or did they use the same spreadsheet for those other lines as well?
I was promised a flying car. Where is my flying car?
"Yes, I know. This is heavy" - Doc Brown
"Cursed is he who rises early in the morning..." Isiah 5:11
A convenient scapegoat.
The whole thing doesn't pass the smell test. The bidder that won, Transpennine Express, are infamous for having terrible service, half their rolling stock out of order, late trains, cancelled trains, etc.
Yet they were promising more rolling stock, better service, wifi on their trains, coupled with both lower fares, and vastly more profits than the incumbent franchisee. Any school kid could told you that you can't get more, for less. Their bid was just not credible on the surface ; the numbers should have been run three times, with different people doing the calculations.
So why not? Well, you might suspect that the people doing the figuring had some kind of motivation other than doing a good job. Maybe a promise of a big fat payoff.
You say that like it'd be a bad idea.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
If you want my hackable PC, you'll have to pry it from my cold dead hands!
"Actually, ISTR it was forced through by a dying Tory government who believe in privatisation as a matter of principle and were desperate to get the contracts signed to tie the hands of the incoming Labour government."
True, and the way they did it was beyond stupid. They had 3 options and chose the worst one:
They created a corporation, Railtrack, whose sole profit came from cutting costs... and gave it a monopoly over track maintenance and improvements.
Almost the first thing Railtrack did was massively underestimate the cost of upgrading this same line being bid for. They estimated £2bn. The final cost would have been £13bn. One presumes that, being guaranteed a bailout, they just didn't care enough to actually do a survey.
Several train crash disasters later, the new Labour govt nationalised Railtrack then converted it into a non-profit organisation called Network Rail. Unfortunately, a recent study has shown that Network Rail is 40% less efficient than even state run parts of EU train services. This is what's swallowing up most of the govt subsidy.
This recent study also set out a series of reforms that no-one understands, but Tories are believed to have hijacked the study and influenced its conclusions.
"Well, yes - because they would know that there is no point competing with a heavily-government-subsidised road system. You can't have taxpayer-funded roads and fully private rail in the same country."
If you exclude the massively greater road infrastructure, road users subsidise rail with something like £20bn a year left over.
Trains are just surprisingly inefficient. They are heavy and manned by a union that has held successive govts at gunpoint for years.