Slashdot Mirror
How Citigroup Hackers Easily Gained Access
Endoflow2010 writes "Hackers who stole the personal details of more than 200,000 Citigroup customers 'broke in through the front door' using an extremely simple technique. It has been called 'one of the most brazen bank hacking attacks' in recent years. And for the first time it has been revealed how the sophisticated cyber criminals made off with the staggering bounty of names, account numbers, email addresses and transaction histories. They simply logged on to the part of the group's site reserved for credit card customers and substituted their account numbers — which appeared in the browser's address bar — with other numbers. It allowed them to leapfrog into the accounts of other customers, with an automatic computer program letting them repeat the trick tens of thousands of times."
There is no facepalm big enough to express my feeling at that hack. I'm sure they paid good money to "security professionals" to set that up too.
I read the internet for the articles.
Heads need to roll for this one... Amazing. Words escape me.
If you want news from today, you have to come back tomorrow.
I did that at a bank I was working with. It was actually a hidden form variable with the institutions username/password, but grabbing that page before it auto-submitted allowed me to pull anyone's statement. I showed it to my manager, and eventually got a promotion out of it. :-)
When writing our rest services the first thing we considered was how to prevent users from accessing other users data. I don't understand how this could happen to a bank with credit card data. It's ridiculous.
Dealing with credit card information I know for a fact that security implementation is 100% illegal if the allegations are true. Citibank will be fined hundreds of millions of dollars if they follow the law ($100,000 per incident). I mean base level security for this would be only allow that user access to that specific account. If they were able to simply change URL numbers to see other account holders info... wow... just wow.
Seems like the website required to have *some* authenticated sessions. Even though they probably used some stolen credentials (at least one would hope), they must have used their own when they *discovered* it. So the way to find them is to look at the logs and find people who accessed diff acct urls under the same auth token prior to this massive theft. I bet there are not going to be that many of them.
Mind numbingly so.
Really makes me wonder wtf is up with some banks and their incompetence. I registered for online banking with my bank some time ago, and they only allow [a-z][A-z][0-9] for passwords. no ~!@#$%^&*(. In the 21st century. Shame.
Sent from my PDP-11
If you don't understand how a secure negotiation protocol (and the protocol for the session after the fact) works, admit it and either ask someone or read several books until you recognize that you should still go ask someone. I've read more than my fair share of crypto books and papers, but, being an application developer who does only trivial personal server-side development, you can be damned sure that I'd ask for help when working on a username/password system. This goes double if it involves banking.
That any session allows them to go digging around willy nilly is so unbelievably stupid, I can't even find the words.
From TFA:
One expert, who is part of the investigation and wants to remain anonymous because the inquiry is at an early stage, told The New York Times he wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser. He said: 'It would have been hard to prepare for this type of vulnerability.'
/epic facepalm
First, this is NOT a hard vulnerability to prepare for. If the only method of user authentication you are doing is based off a string of characters received from the URL your not even qualified to build an ecommerce site for some mom-and-pop 2-sales-a-week company, let alone a bank.
Second, why is this a surprise to this security "expert"? Anyone who has done development for a website with dynamic content would be familiar with passing information through the url. This is like web design 101. If I logged into my credit card account and saw my CC number in the URL bar the FIRST thing I would think of would be: "what would happen if I typed in another number in there." Security expert my ass, no wonder why some companies have this happen to them, look at the people they hire to test and investigate their systems!
/rant
If what I just said sounded like a troll, it was probably just a failed attempt at humor.
<NICE>
This is what you get when important functions are written by people who do not have the slightest inkling of what network security is about. You can put loads of $$$ into planning and design into specifying authentication, and it all falls down because the grunt who actually does the work doesn't have a clue.
</NICE>
<REALISTIC>
Probably the grunt without a clue is the smartest guy over there.
I know, redundant. But fuck. you've got to be kidding me! I think you are kidding. Nice lulz. This is a joke. Right?
Just need to check something...
This was exactly my thought... "Hmm, we would have never thought of changing the account number. That must be some dark haX0rs voodoo magic."
Like puzzle games? Warehouse51 for iOS
It's a good thing our foresightful federal government nobly resisted the public in '08-'09 and wisely chose to bail out and backstop this vital financial instution, on whom we are so ever reliant for their irreplaceable expertise!
*jerk off gesture*
Information theory is life. The rest is just the KL divergence.
So..which is it? Simple or sophisticated? Or simple?
Is it really that much trouble to add a secure hash of the id to the URL or check against the session if the user has access to that record? Come on, that is BASIC security.
This is a failure in programming (I'll stop short of calling the coders idiots, since I don't know what pressures and time constraints they were under) and testing (this should be caught within 10 minutes with a half-hearted Selenium script). The mistake they made: if user is authenticated, they belong, and everything gets happily processed. Pretty typical, especially for beginning programmers. They failed to check individual resources against what was being param'ed in.
The best thing about a boolean is even if you are wrong, you are only off by a bit.
Saying one would have to take a security course might be pushing it a little. Honestly, it seems like, in order to pull off this attack, one simply needs to notice that your own account ID in in the location bar. This is "hacking" that a twelve-year-old could figure out. In fact I'm pretty sure that I did try this sort of thing trying to "hack" a Pokemon BBS when I was 12 or 13. (It didn't work.)
It's the security solution for Citigroup!
What? I mean, WHAT? Teenie-bopper web developers, tired of having their Star Wars fansites hacked, stopped putting account info in GET strings back in the nineties! What kind of crap programmers... the mind boggles... What BANK would pay for such crap code, and what enterprise-class design team would make such a horrible mistake? This is not a cute little hack, it's a fundamental coding... no, design... no, sorry, CONCEPTUAL flaw.
Everyone involved with this project; design, management, QA, and most especially whomever at Citigroup signed off on the project, should be immediately fired and never work again in this field.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
IF the article is correct about the nature of the vulnerability this quote is the single stupidest and most frightening things I have ever read on the internet.
because this is epic fail.
Never let a lack of data get in the way of a good rant.
Has anybody done some sort of audit of various bank's online security procedures to find which, if any, have a decent setup?
... for which they'll immediately pass the cost to their customers. Do you REALLY think it costs them two bucks to let you use other institutions' ATM? Do you really think it costs them fifty bucks to stop payment on a check? Until we're talking about serious jail time in the pound-me-in-the-ass prison for officers of the corporation, nothing will change. But knowing how congress critters in Washington are all already bought and paid for, I think we have a better chance getting a snow storm in hell.
ELOI, ELOI, LAMA SABACHTHANI!?
This is because, according to a report by Verizon and the Secret Service, the demand for data is on the rise. In 2008 the underground market for data was flooded with more than 360 million stolen personal records, compared to just 3.8 million in 2010.
How is that a rise?
Dailymail and Citi bank apparently use the same QA department.
Yes, the car is locked, but all the cars use the same key. It would have been hard to prepare for this type of vulnerability.
"One expert, who is part of the investigation and wants to remain anonymous because the inquiry is at an early stage, told The New York Times he wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser.
He said: 'It would have been hard to prepare for this type of vulnerability.'"
Really? They were passing a credit card account number in the clear through a GET parameter, without validating it against which session the page load was authenticated on, and that was "hard to prepare for"? Really?
I could have done it better than that. So I guess that makes me an expert, right? (Hint: No. It makes the "expert" a flaming idiot.)
This kind of negligence should be criminal.
Sending the account number out in a URL over SSL should not be that big of a hole.
(Ok, not smart, but the risk lies mostly in the person looking over the user's sholder).
The problem was allowing the change in the URL without going thru re-validation of credentials.
Apparently they set a session flag indicating that validation had been passed, and never bothered
to match that with the change in the account number.
Sig Battery depleted. Reverting to safe mode.
....to breach security by focusing on the vulnerability in the browser. I see what you did here! It is not a vulnerability in the browser. It is a vulnerability in the code and the whole system behind it. You cannot escape your liability with this nonsense.
If you're a baseball fan you'll get the connection here (um, get the name of the stadium): this is so Mets-like an event and an outcome. I recall Casey Stengel's immortal words from when he had the helm in Flushing: "can't anyone play this here game?"
Development is programmable; Discovery is not programmable. (Fuller)
Something as simple as common fucking sense could have prevented this, no filters of any kind needed. He obviously allowed all users to log in with the same credentials at a lower level, and made it dead simple to switch users with a URL hack.
"When information is power, privacy is freedom" - Jah-Wren Ryel
http://slashdot.org/~CmdrTaco
Dear God....
Please remember this story next time your boss thinks it's okay to hire or use just anyone to do QA. PMs and Customer Service agents are not testers! Nor can you do effective testing with only kids straight out of school.
Imagine if buildings got built with no architects, no engineers, just construction workers. Or no construction workers, just engineers. Would you feel safe on the top floor?
It doesn't matter WHAT time or money constraints they were under. This is simply not something that would be acceptable out of somebody that codes for money. To call this a "beginners mistake" is an insult to Web Development 101 students everywhere. If you have to be TOLD that maintaining authentication to a secure website based on the contents of the URL bar is a bad idea, then you do not deserve to be coding for anybody. I haven't EVER coded a website (I haven't written anything longer than a ten-line shell script in 13 years) and I could have told you this was a mind-bogglingly stupid mistake. This is not 20/20 hindsight at work here... it really is that stupid.
Heads should roll, including the programmer(s) responsible for this travesty, and two levels of management above him/her. And the remaining employees in the department should all have to apply for their jobs again (by the new management team), as their suitability as programmers could not have been properly evaluated before if the original moron managed to keep his job longer than a week.
I'm actually willing to cut the testers some mild slack... maybe they chose not to test for the developer having the IQ of a turnip. (Just a little slack... a tester should NEVER assume the developer has the least clue what they are doing when figuring out what needs testing.)
That made me laugh, codepigeon.
One expert, who is part of the investigation and wants to remain anonymous because the inquiry is at an early stage, told The New York Times he wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser. He said: 'It would have been hard to prepare for this type of vulnerability.'
Are you *really* trying to label this as a browser vulnerability issue?
You're either *really* incompetent or paid very well to say shit like that.
sysadmins and parents of newborns get the same amount of sleep.
Should CERT issue an advisory on outsourcing as a hot new attack vector?
Sending the account number out in a URL over SSL should not be that big of a hole
Exposing an internal ID in such fashion is not only foolish, but very much a beginner error. I would expect this from some half-assed forum software - not a bank. That said, I've worked for the government before, and seen the same stupid mistake repeated time and time again. A salted hash would have been a lot less idiotic. The fact that there was no authorization performed makes compounds the issue, however, and one wonder who these people hired to write their infrastructure.
Right, because you can't set cookies with wget or squid.
"In conclusion, the main thing we did wrong when designing ATM security systems in the early to mid-1980s was to worry about criminals being clever; we should rather have worried about our customers - the banks' system designers, implementers, and testers - being stupid."
Ross Anderson, "Security Engineering"
This is super basic stuff in the web world. What they did in this debacle is let you into the bank (citigroup.com), talk to you one-on-one at the teller station (SSL), have you swipe your card and enter your pin (login/password), then let you fill out a withdrawal form for anyone's account and give you the money!!
"Uh... yeah, I'd like to get the money from my account number +1... oh, that one's closed, how about my account number +2, nope, well then +3? Ah, yes, that one please... all the money, yes."
I don't bank with citigroup, and I certainly never will knowing how little effort they put into their security practices.
You could have gotten retirement out of it...
This would be the Data Breach Investigations Report.
How is that a rise?
Basic economics would dictate that with supply being signicantly lower in 2010 than in 2008 (less data available on the black market), the demand for said data has gone up.
Keeping critical parts of the session-state is an absolute beginners mistake. Nobody halfway competent in the are of web-security will do anything this stupid. It is also very easy to spot and exploit. The responsible parties at citigroup should face harsh criminal penalties for this and that includes management that signed off on this trash.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The lowest bidder.
I disagree. There's got to be a cutoff point below which it ceases to be fail and emerges into some sort of parallel universe.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Actually, the vulnerability is in the protocol. Never, ever, ever keep critical state client-side. One of the absolute basics of web-application security. Still violated quite often in practice and also in security-critical applications. I can only guess that this is due to outsourcing and hiring the cheapest possible developers that can barely use some web-application framework or toolkit without any understanding what they are doing.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
This kind of attack is what they teach in the first term of their ethical hacking and countermeasures course.
What kind of morons don't programme against something that's so basic ?
They can't pass the cost to their customers. Or rather, they have already ensured that their price is the maximum their customers can give.
If you have a bank account with Citi, you are probably earning 0.05% interest, and paying for all activities you perform through the bank.
Time to let Citi crash and burn, and move their customers' assets to a bank that isn't completely corrupted by profiteering and shitty service.
It's cheaper for Citigroup to spin its way out of this mess than for it to pay for real security. Because real security requires people with some sense throughout the chain with access to the organization. And that kind of person is a threat to the entire way of doing business that banks like Citigroup do it.
Remember that Citigroup is exactly the bank for which Senator Phil Gramm (R-TX) wrote the 1998 bank deregulation bill that left the global economy exposed to exactly the kind of collapse the 1934 regulations had protected us from since the last time the banks gave unregulated credit until they collapsed. They have learned from the 2008 Crash that they will be given only more money when they fail, so they don't work hard to avoid the risk. The kind of "moral hazard" that banks use to excuse paying their insurance obligations, but which define their own businesses now.
--
make install -not war
Anyone remember? You could gain access to anyone else's mailbox by replacing your own address with theirs in the URL bar...10 years later, a bank still can't figure that out? These are the jackasses we "trust" with all of our money and assets, too.
who immediately went and checked their own bank website for the same vulnerability?
It's hard to even get management to acknowledge the problem, even when you spot them.
1) Spot an Id that's obscure, but knowing that Id means something to the framework that you're using.
2) Report it to project manager, and in this case it's the Technical Director of the company(!)
3) Get told in no uncertain terms that you're spouting rubbish, as a 'tiger team' employed by the customer has done a security audit.
4) Repeat that given a reasonably short amount of time that I could manipulate the framework to drop into an administrative mode with full control.
5) Get told my PM/TD that I am not to waste my time on such nonsense, and get on with whatever it was I was doing at the time.
6) Mention a methodology to my colleagues that I might try, if I had been given time (hint hint)
7) Take a few days off sick leave, after discussing things further with an interested peer.
8) While away peer follows up on my ideas, and demonstrates it on live app, with a manager who has an account at said institution
9) Shit hits fan.
10) Find out that I'm sacked when I return
11) Profit, sued for unfair dismissal. (Yes it's more complicated than the above summary)
Summary; People are stupid, PM want the job done as quickly as possible and Directors want profit as soon as possible - results corners are cut. News at 11
TCS no doubt.
it is entirely and completely bad and serves no good purpose.
whatever step you are using to "verify" the id passed by the URL is what should be tracking this in the first place, by passing an ID in the url you only open things up for some other coder working on a different section of the system to use that ID without realizing it is not authenticated. unique short lived tokens already make this a solved problem, especially if every page loaded gets a new token as well and is only valid for actions connected to that page
Snowden and Manning are heroes.
The webmonkeys should be beaten
I agree. There were a couple semiconductor manufacturers, whose support ticket trackers had the same bug. I ended up helping out other people who had trouble with support drones. All it took was changing the ticket number in the URL. It's as trivial of a bug as it gets, yet I don't think it'll ever die out.
A successful API design takes a mixture of software design and pedagogy.
to a bank that isn't completely corrupted by profiteering and shitty service.
Huh? Is there such a thing nowadays?
Questions raise, answers kill. Raise questions to stay alive.
This doesn't even qualify as a hack. It's more like a tactic a curious script kiddie would try just to see how something worked, and suddenly being pleasantly surprised when some other user's data was handed to them on a silver platter as a reward for bothering.
Sadly, I'm willing to bet this kind of "exploit" is probably far more common than anyone is willing to admit. Like those of us who have ever "left the water running" and only coming to realize it 50 miles down the road.
It's something so stupid, most developers wouldn't bother checking their own work for such a "rookie mistake", simply because they're just that good.
8==8 Bones 8==8
Things like this are an inevitable consequence of commoditizing development and outsourcing it to India & China or onshoring it via H1B holders whose certifications and degrees are printed on tissue paper. As an IT manager for years the quality of candidates I have seen coming from those sources is laughable--they code by flowchart. But those are exactly the kind of 'programmers' banks love to hire, because they work cheap and never complain when you work them to death because you can fire them and they get sent back to the old country. Do they do crap work? Yes, absolutely. But that's not the MBA-holding, PHB manager's problem, because they get to claim cost savings and a promotion for it and push like hell to get as far away from the inevitable consequences as they can before it blows up.
If you are an IT manager, please do yourself a favor and hire experienced natives who really know what they're doing. They will cost you a couple 10G's more, but the difference in the product will save you millions it would cost to fix crappy code and the tens of millions more in liability when your customers learn the hard way how lightly you treated the confidentiality and security of their data.
Do what you can, with what you have, where you are.
All this time I've just been using that trick to get free porn.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
I disagree. There's got to be a cutoff point below which it ceases to be fail and emerges into some sort of parallel universe.
Problem is, it's pretty much the same universe as ours, but they have cool hats.
Jesus was all right but his disciples were thick and ordinary. -John Lennon
Comment removed based on user account deletion
That is probably the most stupid comment I have ever seen. Supply was lower in 2010 compared to 2008. So that explains an increase in demand? That is not how "basic economics" works. Demands lead Supplies.
He said: 'It would have been hard to prepare for this type of vulnerability.'
Well, yes, actually. It's not saying "it would have been hard to prevent this type of vulnerability", it's saying it would be hard to prepare for hundreds of thousands of customers' information getting stolen. That does sound hard to prepare for.
One expert [...] told The New York Times he wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser.
Maybe he's just being horribly misquoted here. The vulnerability can apparently be triggered using a browser in a very standard way, which to a journalist might sound an awful lot like a "vulnerability in the browser". Still, if it's just shoving different numbers in a query string (which the article really, really makes it sound like it is), there's nothing to wonder at.
Yet again, faced with a news article on a topic I'm somewhat familiar with, shoddy reporting shines through. Disgusting.
But "the lowest bidder" is the spirit of corporate America!
Obvisouly it is not that Citibank were criminal morons with absolut disregard about their customers, but that the attackers were sophisticate terrorists (and paedophyles, now that we are talking about it).
Banks often buy ready made software and customise it rather than writing their own from scratch, and there aren't many suppliers of online banking applications... Also the people writing the software and doing the customisation will be under pressure, and are likely to cut corners etc.
However i would expect a bank to have hired external contractors to audit the application, and any semi competent security testers should have found an issue like this. Perhaps their testers relied on automated tools, and while automated tools are good at finding the most well known webapp bugs like sql injection, they are useless for finding logic errors such as this - since the tool has no way to know that the account data its seeing doesn't belong to the currently logged in user.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Wow. Yes, I can see how making accounts accessible via an unhashed URL is really something no one would have guessed would be a problem.
The URL was not the problem (URLs should be readable and uniquely identify a resource, they are not really related to security) - the access control (non-existant) was the problem. Relying on hashes alone would just be security through obscurity. Although they are public information might have been advisable not to use bank account nos in the url, even on a secured connection, but hashed urls do not provide proper access control (which is what they should have had, to check that yes user a really can not look at user b's account, or 1000 others).
They had authentication but not authorization, that's the problem
I don't bank with citigroup, and I certainly never will knowing how little effort they put into their security practices.
What makes you think that other banks are any better?
People used to think that RSA and SecurID were secure a couple of months ago...
Personally i'd rather see hackers publish information like this where the company is forced to admit to the hack, rather than serious organised criminals systematically stealing money and keeping it under the radar so the bank can continue denying the hack.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Actually, the vulnerability is in the protocol. Never, ever, ever keep critical state client-side.
This vulnerability would still be there if the critical state was server side as well, because the vulnerability is caused by improper access to data, not improper authentication. They may well have had no critical state client-side (and no, the account no is not critical state, they were guessing other account numbers).
Apparently there is no security whatsoever on the internet. It's a wonder how the bad guys manage to crawl all the way to the bank while rolling on the floor laughing.
For the past 3 years, I've been getting emails from another Citibank customer of the same name as me on my gmail account. First it started with "offers" but has escalated to PDF account statements.
Despite all attempts to stop this "spam", they are unable to fix the issue because "I'm not the owner of the account".
So, they happilly let customers set email addresses without verification. And continue the sending of personal information despite being told otherwise.
Time to close my account. This was the final straw.
From the article: "One expert, who is part of the investigation and wants to remain anonymous because the inquiry is at an early stage, told The New York Times he wondered how the hackers could have known to breach security by focusing on the vulnerability in the browser. He said: 'It would have been hard to prepare for this type of vulnerability.'" Someone who says this is not an expert.
Never ever trust the browser!
Epic fail, but of course we would not expect any less from Citibank.
Just like in the Sub-Prime pyramid scandal, the wicked and lazy in Citibank shall be punished too!!
.
.
.
/crickets
Yes, *most* banks will, but the big ones (citi, BoA, HSBC etc) usually have their own. Of them the Chinese one appears to have the best security based o what I've seen.
Get a web developer
The difference, is that with the RSA hack, while badly handled nothing was completely compromised. They only got a free pass on the extra security, but most if not all of these systems also have good password policy enforcement, which is why the threat was identified and stopped. It is pretty pointless to count on just the SecureID for security, as it can be physically stolen, it is just an extra layer of protection like properly implemented biometric checks.
Get a web developer
This is the kind of problem you have when you totally code with entry level or outsourced programmers. ANY programmer with a few years of experience would see that one coming.
âoeIn theory, theory and practice are the same. In practice, they are not." â Albert Einstein
Banks make all their money through debt now anyway, why should they give a flying fsck about personal banking security? Heck they should have less security, so that "hackers" (I use that word liberally) can take out fake loans and money in other people's name, and than city bank can sell those debts to another back for big profits. What could possibly go wrong?
Once you are logged in to a Citibank credit card account you can generate a virtual account number, complete with expiration date and cvv. That would have been an easy way to exploit the compromised accounts, without knowing the password, expiration date or cvv.
And lets not forget that CC numbers follow a specific formula that allows the CC number to be identified as a potentially valid number, which will greatly reduce the burden on the hacker to generate potentially valid CC numbers to hack with.
there was, but Ally Bank is letting its interest rates decay (3% a tear ago, 0.5-1.0% today). fee hikes come next, then they get a Close notice from me.
Ever heard of a credit union? It's curious how much service changes when you remove shareholders from the equation...
Note: I was 13 when I wrote most of this. Take with several grains of salt.