Study Shows Many Sites Still Failing Basic Security Measures

Orome1 writes with a summary of a large survey of web applications by Veracode. From the article: "Considered 'low hanging fruit' because of their prevalence in software applications, XSS and SQL Injection are two of the most frequently exploited vulnerabilities, often providing a gateway to customer data and intellectual property. When applying the new analysis criteria, Veracode reports eight out of 10 applications fail to meet acceptable levels of security, marking a significant decline from past reports. Specifically for web applications, the report showed a high concentration of XSS and SQL Injection vulnerabilities, with XSS present in 68 percent of all web applications and SQL Injection present in 32 percent of all web applications."

Comments

Citicorp Hack

[Anonymous Coward]

Then there is the Citicorp hack, where they dont even bother hashing the account numbers in the URL...

Re:Citicorp Hack

[tomhudson]

The *real* Citicorp hack was getting bailed out with $308 billion in loan guarantees, and NOBODY going to jail.

Re:Citicorp Hack

[tomhudson]

Latest stats for the US - 2nd quarter of 2010 from the FBI: 1,007 bank robberies (includes credit unions, savings and loans, as well as the "too big to fail" commercial banks).

Total loot: $7,820,347.96 in cash, $298.88 in cheques. So far, they've gotten back $1,801,073.18, for a net loss of $6,019,573.66

Extrapolated to an entire year, that would still be under $25 million net. A rounding error compared to all the US bank bail-outs.

200

[badran]

I wonder how they test. Some sites that I manage return the user to the homepage on a hack attempt or unrecoverable error resulting in a 200 return. Would they consider such a system as hacked, since they got a 200 OK return, or not.

Re:200

[slazzy]

One of the sites at a company I worked for provides fake data back when people attempt sql injection, sort of a honeypot to keep hackers interested long enough to track them down.

Re:200

[Fallingcow]

Aside from feel-good "adhering to the standards" crap, it makes your site look less inviting to attackers (a 4xx page returning a 200 OK looks, and is, sloppy as hell) which isn't a bad thing. It discourages automatic scanners from marking vulnerabilities that don't actually exist, which can get your site on all sorts of lists that can drive even more (often automated) traffic your way, wasting cycles and bandwidth. I'd much rather Chinese proxy and Wordpress installation scanners get a 4xx FUCK OFF than an erroneous 200 OK.

With accurate status codes, end users' web clients can (possibly) provide them with better information. It makes it way the hell easier to convert your content or processes in to consumable RESTful services (maybe you want to expose your site resources to a mobile app, say) if you're already reliably and universally slinging appropriate HTTP status codes. Makes writing quick-n-dirty remote unit tests or QoS monitors easier. Apache (or whatever) error logs remain useful.

Lots of reasons. I'm sure there are others that aren't coming to mind, or that I don't know about.

Incidentally, we do need a 400-range "FUCK OFF" status code.

Re:200

[jc42]

Why _would_ you [send valid content with a 4xx or 4xx code]? Is there incentive to be standards-compliant, friendly, and heterogenous-mix-of-clients interoperative with attackers?

Perhaps because you know that the "attacks" are coming from sites that don't know they're attacking you, but are merely asking for content.

The specific cases I'm thinking of are some sites that I'm responsible for, which can deliver the "content" information in a list of different formats such as HTML, PS, EPS, PS, RTF, GIF, PNG (and even plain text ;-). The request pages list the formats that are available; a client clicks on the one(s) that they want and presses the "Send" button, and gets back the information in the requested format(s). The data is stored in a database, of course, and converted on the fly to whatever format is requested. Things like PS and PDF are huge in comparison, so we don't save them. The required disk space would be exorbitantly expensive.

There is a real problem with such an approach: The search sites' bots tend to hit your site with requests for all of your data in all of your formats. Some of them do this from several addresses simultaneously, hitting the poor little server with large numbers of conversion requests per second, bringing the server to its knees. Converting plain text to all the above formats can be quite expensive.

How I handled this was to, first (as an emergency measure), simply drop the request from an "attacker" IP address. This gave breathing space, while I implemented the rest. What's in place now is code that honors single requests, but if it sees multiple such requests in the same second coming from a single address or a known search-site address block, replies to just one of them, and sends the rest an HTML page explaining why their request was rejected.

Over time, this tends to get the message through to the guys behind the search bots, and they add code on their side to be nicer to smaller sites like ours.

I've also used this approach to explain to search-site developers why they should honor a nofollow attribute. After all, they get no information from the expensive formats like PS, PDF or PNG that's not in the plain-text or HTML file, so there's no real reason for a search site to request them.

Note that, in this case, we do actually refer to such misbehaved search bots as "attackers". They're clearly DOSing us, for no good reason. But the people responsible aren't actually malevolent; they just didn't realize what they're doing to small sites. If you can defuse their attacks gently, with human-readable explanations, they'll usually relent and become better neighbors. This helps their site, too, since they no longer waste disk space and cpu time dealing with duplicate information in formats that are expensive to decode and eat disks.

It's yet another case where the usual simplistic approach to "security" doesn't match well with reality.

(It should be noted that the above code also has a blacklist, which lists addresses that are simply blocked, because the code at that site either doesn't relent, or attempts things like XSS or SQL attacks, which are recognized during the input-parsing phase. Those sites simple get a 404. But those are a minority of our rejections. We don't mind being in the search site's indexes; we just don't like being DOS'd by their search bots.)

Re:200

[jc42]

Another related problem I've had is that XSS seems to have a wide range of definitions, and is such a vaguely-defined concept that it applies to a lot of valid web applications.

I've seen a number of definitions of XSS that include all cases where a CGI program gets a URL for a third site, and sends an HTTP request there. I have a number of sites whose CGI software is designed to work exactly this way. The data is distributed across several hundred other sites, only a few of them mine. My main sites have small databases where they can look up parts of requests, figure out where the data can be found, pass the request over to that server, wait for the replies, and combine the results into the web page that the client wants.

I don't especially like the idea that self-styled security experts would classify such setups between cooperating sites to be security violations. And I suspect that the folks who did this study would classify our distributed database (and probably google's ;-) as implementing XSS attacks. Our main "public" web sites would be classified as doing an XSS attack on our database sites.

So is there any sensible way to figure out what any given security researcher means by an "XSS attack"? Is there a reasonable way to argue for a more restrictive definition that would permit a flock of cooperating web servers to bounce requests back and forth like ours do, without being classified as an insecure "XSS attack" site?

(Actually, we've known from the start that, in a weak sense, our CGI software can be used to "attack" other sites. Just call it with a random URL; it'll send a GET request to that site. It'll then find that the data isn't in the expected format, drop it after the first data packet, and sent you a "failed" reply. If you do this more than N times, you'll end up in our blacklist, and you'll get a reply explaining why you're blacklisted. An actual attack process would be over in a few seconds, so this isn't very useful as a way of DDOSing some victim site. ;-)

Re:200

[phorm]

Hmmm, how about
a) Have a secondary instance running with dummy/fake data
b) Have a wrapper around queries that checks for attempted injections (perhaps a pre/post sanitization check), if the query is an injection attempt, grab data from the fake DB
c) Watch for people using data from the fake DB, attempt to use a fake (but realistic enough to pass a smell test) CC# are fraud attempts flagged to visa...

Re:200

[lonecrow]

I have been in a similar circumstances and there are a few other ways to handle it.
#1 solution use a link for your main format that you want the search engines to read (html) then instead of links for the other version use forms. You can still use get, and you can style the submit button to look like a link. Sure its a bit more html then a simple link but as a solution it is simple and effective.

what do you expect?

[Nyder]

This is capitalism/corporations. It's all about profit, and spending extra on IT cuts into the bottom line.

Economy is bad, so companies make cuts. Personnel, IT, Security, and everything but the CEO's bonuses get cut.

Re:what do you expect?

[Anonymous Coward]

It also seems to come down to ridiculous timescales. A project is declared, a release date is set in stone. The client overruns their alloted time to come up with requirements/content, the release date stays in stone. The legal teams take forever to draw up and agree on contracts, the release date stays in stone. The IA/UX people miss their deadlines for producing the wireframes, the release date stays in stone. The design team go through a million iterations of whether the drop shadow on the footer text should be mauve or fuscia and overrun their deadline, the release date stays in stone. The client pops up again with dozens of last minute change requests, the release date stays in stone. Then it hits development's desk and suddenly the three month project has to be done in two weeks. Development is almost always the last link in the chain and, as such, always the department under constant crunch time. Developing a complex site with vague specs across half a dozen minds isn't easy, but unlike all the other parts of the chain leading up to this point, it's the part where the client can be most punished if it's not done right, yet nobody ever sees the benefit of allowing sufficient time (and doing sufficient testing).

Re:what do you expect?

[Anonymous Coward]

If I gave you enough time to do development right, the competition would beat us to market, drive us out of business, and you would be out of a job.

Don't think it is any different working for one of our competitors, they will overwork you just as hard for fear of US beating THEM to the market.

The market has shown a surprisingly high tolerance for bugs and security gaps, so we simply can't afford to proactively fix those.

And if you don't like my high bonus....go start your own company. After realizing just how hard and risky it all is, you will feel like you deserve a nice fat bonus too.

Re:what do you expect?

[Ramley]

I am sure your point is a part of the problem, but in my (many years) of experience, this has a lot more to do with a myriad of factors, none of which really outweigh the other by much.

I am an independent developer who works on projects with security in mind from the ground up. Time/budget be damned, as it's my reputation on the line. If they can't pay for what it is worth, I tell them to find another developer.

They tend to learn the hard way — it was a better option to stick with a security minded developer in the first place. 85% of them return as customers.

The problem seems to be that most of the developers I have worked with, be it corporate employees, or indy's like myself, are one of two things, in general: (very general)

1. Lacking knowledge of how to deal with the most common security threats.
2. Lazy, and don't care enough to implement safeguards, etc.

Most of the other excuses boil down to one of the above.

That's my experience out there in the field, working with lots and lots of diverse companies. Of course profit and time to complete enter the picture, but over time, this can be overcome with a lot of experience and a lot of [code] libraries which can be easily implemented, no time lost.

Thats why we outsourced our IT to the Cloud

[Anonymous Coward]

Now its not my problem, its my Cloud providers problem.

Nothing new here

[vikingpower]

I am on a project for ( smoke-testing ) the core app. of a major european airport. Same problems there. Management, after having been informed, said: "Not a priority". I guess only their bonuses are "a priority" ? I am thinking seriously of giving pointers to the whole project to Anonymous.

Re:Nothing new here

[delinear]

The problem is that the media seem to be in the pocket of big corporations, so when Anonymous inevitably find one of these exploits and steal a bunch of data, the media never seem to hold the businesses who left the door open to account. The lack of security should be a massive topic of debate right now, but instead, outside of certain circles, it's a complete non-issue. During the coverage over here of the various exploits of Anonymous, I don't think I once heard any searching questions asked of the global corporations who allowed a bunch of teenagers to make their security look like the equivalent of a balsa wood door on Fort Knox (and that includes the BBC, who should be the least biased since they're not privately owned, but still either don't want to offend the PR departments of companies who feed them half of their content or just believe the company line and don't bother digging deeper for the real stories).

Re:Nothing new here

[Just Some Guy]

the media never seem to hold the businesses who left the door open to account.

To a point, I understand their logic: you don't blame the victim. But a company publishing SQL injections in 2011 should be dragged through the mud and humiliated. Maybe someone needs to start a newsroom consulting company where reporters call for technical clarification:

Reporter: Hey, Amalgamated Bookends got hacked by someone who replaced the BIOS on their RAID cards with a webserver. Who's in the wrong?
Consultant: Wow! That's a pretty ingenious trick. I hope they catch that hacker!

Reporter: Hey, Shortcake, LTD got hacked by someone who added "?admin=true" to their website's URL. Is that bad?
Consultant: See if Shortcake's sysadmin is somehow related to the owner. It bet it's his nephew.

Reporter: Hey, Sony...
Consultant: LOL dumbasses

Uh huh

[TheSpoom]

Security auditing company produces report that conveniently shows that their services are desperately needed. News at eleven.

Re:Uh huh

[justdiver]

I've seen this response before on other articles. But who else other than a security auditing company is going to do security audits? A companies internal IT may do this, and I say MAY do this, but they're certainly not going to publish their results to the public. Should we discredit companies that do automobile crash tests because they find that cars are inherently unsafe and need crash testing done to make them safer?

It's the Same Everywhere

[derrickh]

You have to realize that somewhere on the net there's a surveillance camera forum with guys saying 'businesses are too cheap to invest in multiple cam setups to cover exploitable deadzones'... and there's a locksmith forum with guys saying 'These companies are still relying on double bolt slide locks, when everyone knows they can be bypassed with a simple Krasner tool!'...and there's a car autosecurity forum wondering why companies still use basic Lo-jack instead of the new XYZ system.. and don't forget the personnel consulting forum where everyone complains that companies don't invest enough in training to recognize grifting attempts on employees.

It's a never ending list and to expect everyone to be on top of all of them at all times is n't realistic.

D

Re:It's the Same Everywhere

[naranek]

It's a valid point, but on the other hand you can't routinely try breaking into random houses or cars with little chance of getting caught, and then use them undetected for your personal gains. Your crappy lock will do unless someone from your neighbourhood personally targets your house. With computer security there is a constant global crime spree trying all the locks all the time. This is why I think that computer security needs to be handled with extra care.

Re:It's the Same Everywhere

[mjr167]

Why isn't private data also a calculated risk, same as physical goods? Both have a cost associated with securing them. Both have a cost associated with losing them. Security is, always has been, and always will a cost/benefit analysis. If losing data costs less than securing it, then why bother? It's cheaper to clean up the mess than prevent it. Until losing data has a higher cost than security, you aren't going to see it treated well. This idea that virtual things are somehow different from real things needs to go away.

Re:It's the Same Everywhere

[mjr167]

Then we need to hold them accountable for losing it. We should not expect other people to safeguard our things out of the goodness of their hearts. When you give your physical goods to another party for safekeeping, you sign a contract stating what they are and are not responsible for. When you give packages to UPS, UPS accepts a certain amount of liability if they should damage or lose the package. When you place things in storage, the storage company accepts a certain amount of liability. Before you entrust your physical goods to another entity, you sign contracts stating what your expectations of their safeguarding the goods are and the penalties you will levy against them should they fail in that trust.

We have failed to impose the same standards on those we trust with virtual goods.

The level of effort taken to secure an item should directly correlate to cost and the consequences of losing/damaging it. Some of the tests for shipping containers for spent nuclear fuel rods include surviving a 100 ft drop onto a 6 inch spike, 12 hours in burning jet fuel, and being shot with an RPG. That is all obviously overkill for a package containing my niece's Christmas present.

I can chose to have a letter sent certified/registered mail to be hand delivered by professional, bonded courier. Or I can chose to ask some kid to drop it off on his way to school. My decision will involve a cost/benefit analysis. Before you can convince people to invest in data security, you need to demonstrate that breaches have real costs that justify the expenses. Network security is really no different than physical security. Some places are going to have broken security cameras that have never actually recorded anything, and others are going to have armed guards monitoring live feeds. The difference between the two is a business decision based on a cost/risk analysis.

One of the biggest problems with data theft is proving who is responsible for the loss. It is easy for me to prove that someone lost my credit card numbers. It's much harder to prove who lost it.

Re:It's the Same Everywhere

[bodangly]

I think you hit the nail on the head with your first sentence. Obviously, companies aren't securing data out of the goodness of their hearts. I'm really not one for adding more laws but it seems to me there needs to be legal repercussions for negligence in regards to customer data. Of course, the issue is far deeper than a merely technical one. The US government isn't exactly known for holding corporations accountable; they much prefer to hold an individual's feet to the fire. So hold the whole damn company accountable. Even I'm not crazy enough to suggest criminal repercussions for negligent managers, after all each company is basically its own government so whos to say which individual is responsible, as you mentioned. So make breaches costly. And I'm not talking fining a multibillion dollar company $100,000. Base the fines on a (sizable) percentage of gross income per data breach. I can't think of any other reasonable way to give a company incentive to invest in security when its not crucial to their own business model.

Yeah but in htis case they're probably right...

[ray-auch]

Where I work, every time we get told to put our details into some new provider system for expenses, business travel or whatever (happens regularly with corporate changes) we see who can hack it first. We're developers, it's our personal data, why wouldn't we check ?

The fraction that are hacked in minutes is probably near 50%, and 32% for SQL injection is probably about right.

I'm not sure which is more depressing - the state of the sites or that even though we have a "security" consultancy practice in house, we get corporate edicts to put our data into sites that we haven't even bothered to audit to the extent of sticking a single quote in a couple of form fields or changing the userid in the url...

What's the real story?

[dreemernj]

The precipitous drop in the "pass" rate for applications was caused by the introduction of new, tougher grading guidelines, including a "zero tolerance" policy on common errors like SQL injection and cross site scripting holes in applications, Veracode said.

Is the story that SQL Injection and XSS are still a problem or that Veracode just recently took a "zero tolerance" stance on SQL Injection and XSS in the applications they test?

I work at Veracode; here's how we test.

[Anonymous Coward]

I work at Veracode, and can share how we test. I'll be brief and technical here, as there's lots of marketing material available other places. In short, we scan web sites and web applications that our customers pay us to scan for them; the "State of Software Security" report is the aggregate sanitized data from all of our customers. We provide two distinct kinds of scans: dynamic and static.

With dynamic scans, we perform a deep, wide array of "simulated attacks" (e.g. SQL Injection, XSS, etc.) on the customer's site, looking for places where the site appears to respond in a vulnerable way. For example, if the customer's site has a form field, then our dynamic scanner might try to send some javascript in that field, and then can detect if the javascript is executed. If so, that's an XSS vulnerability. As you might imagine, the scanner can try literally hundreds of different attack approaches for each potentially vulnerable point on the site.

The static scans are a little fancier. The customer uploads to Veracode a copy of the executable binary build of their application (C/C++, Java, .NET, iPhone app, and a couple of other platforms). From the executable binary, the Veracode systems then create a complete, in-depth model of the program, including control flow, data flow, program structure, stack and heap memory analysis, etc.. This model is then scanned for patterns of vulnerability, which are then reported back to the customer. For example, if the program accepts data from an incoming HTTP request, and then if any portion of that data can somehow find its way into a database query without being cleansed of SQL escape characters, then the application is vulnerable to SQL Injection attacks. There are hundreds of other scans, including buffer overflows, etc.

Personally, I think what we do at Veracode is pretty amazing, particularly the static binary scans. I mean: you upload your executable, and you get back a report telling you where the flaws are and what you need to fix. The technical gee-whiz factor is pretty high, even for a jaded old-timer like me.

Re:I work at Veracode; here's how we test.

[kriegsman]

Oops, I wasn't logged in. The above comment is from me, Mark Kriegsman, Director of Engineering at Veracode.

Re:I work at Veracode; here's how we test.

[kriegsman]

That is a GREAT question, and the full answer is complicated and partially proprietary. But basically, you've touched on the problem of indirect control flow, which exists in C (call through a function pointer), C++ (virtual function calls), and in Java, .NET, ObjC, etc. The general approach is that at each indirect call site, you "solve for" what the actual targets of the call could possibly be, and take it from there. The specific example you gave is actually trivially solved, since there's only one possible answer in the program; in large scale applications it is what we call "hard." And yes, in some cases we (necessarily) lose the trail; see "halting problem" as noted. But we do a remarkably good job on most real world application code. I've been working with this team on this static binary analysis business for eight or nine years, and we still haven't run out of interesting problems to work on, and this is definitely one of them.