Programming Bug Costs Citigroup $7M After Legit Transactions Mistaken For Test Data For 15 Years (theregister.co.uk)
An anonymous reader shares a report on The Register:A programming blunder in its reporting software has led to Citigroup being fined $7m. According to the US Securities and Exchange Commission (SEC), that error [PDF] resulted in the financial regulator being sent incomplete "blue sheet" information for a remarkable 15 years -- from May 1999 to April 2014. The mistake was discovered by Citigroup itself when it was asked to send a large but precise chunk of trading data to the SEC in April 2014 and asked its technical support team to help identify which internal ID numbers they should run a request on. That team quickly noticed that some branches' trades were not being included in the automated system and alerted those above them. Four days later a patch was in place, but it wasn't until eight months later that the company received a formal report noting that the error had affected SEC reports going back more than a decade. The next month, January 2015, Citigroup fessed up to the SEC.The glitch resided in new alphanumeric branch codes that the bank had introduced in the mid-1990s. The program code filtered out any transactions that were given three-digit branch codes from 089 to 100 and used those prefixes for testing purposes. The report adds, "But in 1998, the company started using alphanumeric branch codes as it expanded its business. Among them were the codes 10B, 10C and so on, which the system treated as being within the excluded range, and so their transactions were removed from any reports sent to the SEC."
I wouldn't call it "remarkable" that it wasn't caught for nearly 15 years. It actually makes sense, as the assumption was that 089 to 100 wouldn't include 10B, 10C, etc. Those kinds of mistakes can happen, and very easily. Just goes to show that you should be more explicit with how you filter data, in many cases.
Sounds like it worked exactly as designed. Should have consulted your dev team before changing the way you name things. Maybe have IT in these meetings?
Proper unit tests would have prevented this problem in the first place.
As would having a separate system for testing, rather than trying to create a test island within your actual real live [that's enough - Ed] production system.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Anyone who has worked in the finance industry on the tech side of things has probably seen eye-searing levels of problems like this. It's clusterfucks all the way down. It always surprised me that something that seems like such a natural fit for software was always, without fail, so riddled with glaring bugs that it's almost unfathomable that you are the first person to notice them. At a lot of shops, the bugs are so ingrained in the process that they can't even be fixed. Working in the finance industry certainly doesn't inspire confidence in the finance industry.
I guess this week we're punishing people for "unintentional" failures to comply with regulations again?
Just curious, are you drunk, or are you stoned?
Citigroup's trading and reporting systems aren't open source, quite the opposite; they are closely guarded secrets.
Are you confused because the output is required to be in a specified format?
An interesting experiment would be to make down-voting cost 2 moderator points instead of 1. The idea being that it would make interesting/insightful posts "stickier" by making them harder to down-vote based on agenda. I have no idea whether it would work or not but, at the very least, it might give casual users more expanded comments to read by default.
Having said that, getting rid of moderators is pure insanity. Community moderation is part of what makes slashdot an interesting site.
nice point though.
The banksters are simply too big to jail and too big to even question. Break up the banks.
"Honey I shrunk the government. And the banksters drowned it in the bathtub!"
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
"He was paid well to provide software that functioned to the defined spec, and he failed to do that."
Not correct at all. The software (apparently) worked well according to the original specification. Then they extended their business to open new branches, but did not adequately update the software. Not a problem of the programmer, but a problem of change management. You could just as well complain that your toy tricycle is not safe on the highway - possibly quite correct, but it's your fault if you are operating it outside its specification ("use only by children up to 30 kg on the sidewalk"), not the tricycle engineers problem.
Stephan
if they were serious, it would have at least been a $100 billion fine.
Anons need not reply. Questions end with a question mark.
No one believes you are the real apk. it is a well known fact that the real APK likes to fuck GOATS.
which eventually became AOL, we were routinely sent CDs with patches on them. Eventually we got the CDs that would patch our beta releases to become public release apps. As beta testers the service was charged at half price. Almost a year into the public release, I got a phone call from Steve, the boss at Quantum, letting me know that the one thing they forgot to patch in the upgrade CDs was the switch to full price. So would you please cut us a check for everything you paid us already for the past year.? Um, no... by the way how many users did this affect? We're not sure. Dozens? Well yeah. Hundreds? Yeah. Thousands? Look, that's not the important part. I believe I offered to pay double the monthly bill until I was caught up. Never heard back, next release placed us at full charges. I bailed once it was AOL, and it was back to Delphi and The WELL.
"Win treats sysadmins better than users. Mac treats users better than sysadmins. Linux treats everyone like sysadmins."
This is a perfect illustration of why "smart" IDs are a bad idea. Any time you encode attributes (like "this is a test transaction") into an ID (like a range of bank branch IDs) you are asking for trouble. Everybody does it, but it's usually just plain lazy and careless. DON'T! Add an attribute that marks the transaction as a test transaction! Then anybody who sees it will instantly know the difference.
An interesting experiment would be to eliminate down votes and only allow up votes and post the voter id next to the article .
It's a management bug. The programming was fine, but somebody failed to make sure it was updated for the branch ID change. It was never intended to handle alphanumeric input, so management should have made sure the programmers knew about the change and thoroughly tested how the software handled it.
Well, there would have been no point in coding it to handle alphanumeric input as a valid ID, so they probably had it handle non-numeric input by setting the ID to one of the test numbers to keep bad data from going to the SEC.
Why?
This really looks a lot like any real mistake that happens all the time.
I see nothing malicious here. Do you? Can you point to the smoking gun that tells you someone did something on the sly here and should go to jail for it.
There is a lot of shit wrong in the banking and financial sectors. People like you though make the truth look like some crazy conspiracy theory when you spout stupid knee jerk reactions like this. Think before hitting that post button next time.
Why is it so hard to only have politicians for a few years, then have them go away?
This sounds suspiciously like they probably had older developers who likely knew what they were doing and the history of the data/application/business who retired/fired/left were replaced by younger cheaper models, who were given a task, did it as best they could without all the prior experience and knowledge (and likely little or no documentation). Having no one else in the organization that understands or sees what is going on, fast forward 15 years, and presto a big problem (though 7 million for a corporation like Citigroup is probably peanuts anyway)...
Stop whining about brats, you're ruining Slashdot.
Change is certain; progress is not obligatory.
Tell me more about the "real apk".
Change is certain; progress is not obligatory.
I assumed that original AC's comment was sarcastic, although that wasn't real clear. Not all of my jokes work, either.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
Written thirty years ago? Optimist! Forty years ago sounds more reasonable to me.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
I wasn't talking to you, APK. But, since you decided to answer, tell us more about the "real apk".
Change is certain; progress is not obligatory.
Sure, there is no additional measured power consumption when using TCP on my network.
Change is certain; progress is not obligatory.
Not BS, the wattage usage did not increase.
Change is certain; progress is not obligatory.
No discernible difference was measured wattage wise. Whatever the difference, if there is any, it's too tiny to many any notable difference to the electric bill.
Change is certain; progress is not obligatory.
He also knows nearly nothing about networking and security, also, he is a horrible programmer who is so ashamed of his work he won't even publish the source.
Just watch as he flips out about all of this, despite that everything I have typed is proven fact.
It also wouldn't surprise me if he did screw goats...it would fit the personality.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Waa, I wanna know who thinks my posts are trolls and redundant. I know I post the same shit over and over, and annoy everyone around me, but I want to know who doesn't like it.
Grow up dude, you shit post, you get modded down.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
It's a single request and response as opposed to ten UDP requests to a server that the resolver performs. So, maybe it isn't. But sure, if you compare /one/ UDP connection to /one/ TCP connection, TCP is doing more work; however we know that modern UDP resolvers don't do just one request any more. Just open Wireshark and check with your PC.
It does prevent spoofing issues, which is where the majority of DNS issues are related to.
Nah, see above.
Change is certain; progress is not obligatory.
On Windows 10, I'm seeing 9.
Which isn't really a problem here, because there is no notable difference in wattage. End of story!
Change is certain; progress is not obligatory.
Doesn't matter apparently, I didn't measure any discernable difference. If there is one, it's so small, it wouldn't impact my electric bill. Fact!
Change is certain; progress is not obligatory.
You really should update your copy/paste, that link to Burns supporting your software goes nowhere, he deleted the post.
Also, insulting me just makes you look like the abused teenager that degrades everyone else to make themselves feel better. It doesn't make you out to be a professional, and it doesn't improve people's opinions of you. It actually backfires entirely on you, as it makes you out to be the one who has lost control of the argument and has to resort to insults to try and degrade the person you are arguing with.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
That would be DBiii, not DBiLL.
Also, he thought I was responding to him, as he couldn't see your offtopic trolling posts, and so was confused why I was responding to something entirely unrelated to what he posted.
Why would I give up? You won't give up despite all the issues with your methods. You won't even acknowledge your numerous failures. Instead, you keep blindly linking to the same things over and over as if you won an argument, and the funniest part is that you link to the ones where I thoroughly thrash you.
So, keep up the bullshit artist, agreeing with yourself, and acting like you WON THE ARGUMENT! You still fail, you still live with mom, you still wrote crappy software.
Oh, and also, Burns suggests uBlock right alongside your software, perhaps you should reconsider using him as a reference since he suggests that horrible software you hate.
Keep up the insults, they prove me right every time. Those who resort to insults have already lost the argument after all.
Insults are the arguments employed by those who are in the wrong.
-Jean Jacques Rousseau
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
dbiii did no such thing, and I don't know who Dbill is, because there is no Slashdot user by that name as far as I know.
It would be an insult; to the toddler. That is not an insult to you, it is a comment on the truth, you seem to have serious issues with your reading comprehension, as you were railing against something that I did not type at all.
You however seem to trot out things that have nothing to do with the conversation occuring, such as my Autism, or that you feel I have a defective brain on almost every comment you type.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Perhaps you should send that to Burns, as he was the one recommending uBlock, not I.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
I've looked through my post history, please point to where I call you names APK.
Third party acting again? Really? Do you think anyone thinks this isn't you?
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
How is that an insult, you don't love your mommy?
I own my own house. I have owned since 2003.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
Um, he recommends uBlock right under your software in the third party tools.
APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?