DARPA Makes Finding Software Flaws Fun

alphadogg writes "The U.S. Department of Defense may have found a new way to scan millions of lines of software code for vulnerabilities: by turning the practice into a set of video games and puzzles and having volunteers do the work. Having gamers identify potentially problematic chunks of code could help lower the work load of trained vulnerability analysts by 'an order of magnitude or more,' said John Murray, a program director in SRI International's computer science laboratory who helped create one of the games, called Xylem. DARPA has set up a site, called Verigames, that offers five free games that can be played online or, in Xylem's case, on an Apple iPad."

Comments

Fun bugs

[K. S. Kyosuke]

Yeah, but when I exploit a buggy drone and fly it into your own units, the same DoD says "that's not funny". ;/ Make up your minds already!

The real game is finding bugs on their site...

[Anonymous Coward]

Welcome null null ( Logout )

psDOOM anyone?

[netpatriot]

not such a new idea: Doom as an Interface for Process Management: http://www.cs.unm.edu/~dlchao/flake/doom/chi/chi.html

Re:psDOOM anyone?

[ColdWetDog]

Or the method of monitoring the exits of airports in Neal Stephenson's REAMDE.

Greetings, Professor Falken

[MonkeyDancer]

I'm disappointed they do not have the game 'Global Thermonuclear War'.

Re: Greetings, Professor Falken

[runeghost]

Oh, they have it. They're just very selective about who gets to play, or even sit at the table.

Re: Greetings, Professor Falken

[davester666]

you have to find the user name on your own, but the password is '00000000'

Re: Greetings, Professor Falken

[chameleon3]

for those who don't get it

Finding bugs is ALWAYS fun!

[tlambert]

Finding bugs is ALWAYS fun!

What's even more fun is that Tesla Roadster you were able to buy by selling the bugs you find to intelligence agencies, rather than reporting them to the vendor and being sued under the DMCA for reverse engineering their product.

How does it work?

[sideslash]

I actually clicked the link and read the brief writeup. Too lazy to go further, and hoping somebody can tell me this:

How can you make a game out of this? It seems to me that the game can tell when the user wins/loses, then there's no reason to create the game at all -- just make the win/lose logic do the error checking directly. So what's the point of the game?

Or is it the case that their games are not able to tell you when you win or lose, and the player has to determine that himself or herself? That would make sense to me, but seems like it would be really hard to make "fun" for the masses.

Re:How does it work?

[Lewisham]

I worked on Xylem when I was a grad student at UCSC. I was not on the team when it launched, so my info may be out of date.

What players are being asked to do is find loop invariants for code. The invariants are hard for a computer to come up with (and be useful), but are easier to check given certain bounds. So there is no predetermined win state, each answer is checked server-side to see if it holds up within the bounds (or, if the answer is already known, the cache hit is returned). If the invariant is complex and holds, it gets scored highly. If it's trivial and holds, it gets a lower score. If it doesn't hold, the instance where it doesn't hold is returned to the player.

Does this help?

Re:How does it work?

[drinkypoo]

Does this help?

It doesn't help explain how it might be fun for the masses... I would just try playing it, but uh nope. Not for the government. Maybe I'd have played it just hosted at UCSC, running against some useful-to-me code.

Re:How does it work?

[Lewisham]

DARPA funded the project, and DARPA fund lots of projects. I think a debate about whether DARPA is good or bad is pretty out-of-scope for this particular work: we made a game that might show how software verification could be crowdsourced.

The games do try to be fun, that's why none of them are "look at this loop and write an invariant". Xylem dresses up the problem statement as logic puzzles that surround the growth of exotic plants. I don't have an iPad to play the final version of Xylem on, but we tried hard to come up with a compelling game.

I don't believe the expected player base really cares about whether the project was funded by DARPA or not. I understand if you don't, but I think you would also have to stop using the Internet if you have such an issue with DARPA funded projects :)

Re:How does it work?

[neminem]

Perhaps you haven't taken an algorithms class, or you've forgotten it, but go look up NP-Complete problems (you've probably heard of them). I'm not an expert, and also lazy, so I have no idea whether these problems are NP-Complete or not, and I'm sure there are other similar classes of problems that aren't NP-Complete, too. Anyway, the idea is, there are large numbers of computational problems that are astronomically difficult to find solutions to an instance of, but given a potential solution to an instance, it's easy to determine whether the solution is valid. Presumably the problems modeled by these games are among such. You make a move, it checks whether it's a winning one. It doesn't have to check *every* move, only the one you just made.

Re:How does it work?

[sideslash]

Well, OK, but actually it turns out that computers are really well suited to finding "good enough" solutions to NP-Complete problems such as traveling salesman and real world equivalents like optimized circuit printing. So to my mind you still haven't described a situation where crowdsourcing with wetware can improve on a targeted silicon approach.

Single page version of the article

[Fnord666]

Here's a link to a single page version of the article.

It's decent

[eyenot]

These puzzles are definitely interesting. I had a chance to get on and play the preliminaries of the pipe game about two hours ago from a college terminal. I get home to continue my "work" and the site is 505'd. I'm guessing it may have been simply slashdotted. If that's the case, then I've lost a bit of confidence in the project.

It sort of reminds me of that scene in "Sneakers" when the guys roll by to get the box back from the "NSA", and the building is being torn down. Which raises the question, if I can imagine using a site to quickly test a population sample's IQ and then to run like heck with the results, then is there a feasible reason to do so?

Sure

[PopeRatzo]

Next they're gonna make a game out of finding out who's going to those demonstrations and protests.

That one will be a blast. I'm sure there are lots of techies who will gladly play that game.

Or maybe a game where you get to control a mech and use a nerve agent on the protesters.

Re:Sure

[Anonymous Coward]

Maybe for background music they can get it to play "Kill the Boer".

Re:Nothing Fun At All

[J Story]

If you've tried playing any of those "games" then you'd know they are not fun at all. Just a big fail.

I agree that the one "game" I played didn't keep me enthralled once the novelty wore off, but it seems to me that there is the *seed* of something that could be fun, for given definitions of "fun". For example, suppose that these games were games-within-a-game, which one could play to win points or "gold" to use in the larger game. Consider it a form of grinding.

Government websites

[Gary Perkins]

When is the government going to learn to fully test their sites before going public? I heard the user side of healthcare.gov is operational, so I went to check it out this morning. I create an account, get sent a verification email. I open the email using SeaMonkey's email client, and it's blank. I look at the raw source of the email, and the message content has a "Base64 decode error". Nowhere on the site is an option to resend, only a phone number to call (f*ck that). On a hunch, I do something I shouldn't be able to do, and create a second account with the same email address. It works (?!), and I receive a second email...with the same issue. Anyhow... I bring it up, because I go to check out this hot new site, and it's down with an internal server error. LOL

Re:Government websites

[Gary Perkins]

Oh yeah, I also cut and pasted the verification link from the borked email (how many users would think to look at the raw source?), and healthcare.gov refused to verify using their own code they sent. WTF?

OldSpeak translation

[ops2048]

Translation - The NSA needs to find better ways to find more vulnerabilities so that they can compromise more targets more quickly. Idiots deserve to lose their freedom.

It's crap. Don't give them any of your free time.

[eyenot]

I gave it a thorough testing today. Granted, it's still all in BETA stage. But I'm not griping about the stupid bugs.

The whole thing sucks. The five different games are basically five different kinds of problems. There's organic chemistry, atomic chemistry, programming logic, and I didn't play the other two games but they appear to be shrouded versions of real life n-body or other computational problems.

So here's the deal. This shit takes a long time. These games get very complex very quickly. I can see myself playing one game a day, maybe an hour at it. The programming logic game works for that, it doesn't take an hour to solve their largest BETA puzzles. By the way, they don't have real actual DARPA programming troubles being made into puzzles just yet. The puzzles there are static and are meant to test the system and see what user feedback is generated.

But then you go into the folding prion game, and it sucks. The tutorial is incomplete and it's a total side-swipe at Scientology. Why the fuck would you actively seek to alienate Scientologists from your defense industry website? That's stupid as hell. They shouldn't be trying to offend anybody, period, let alone Scientologists.

And the folding prion problem has to run some kind of simulation or something in the background when you choose to eliminate molecular pathways (in the guise of more or less Dianetic engrams). And the wait times can be several minutes. And the combinations of splitting molecular bonds and removing molecular pathways quickly arrives at exponentially large numbers. And you apparently have to get them done in the right order. So you could, yes, spend two hours at one problem and not arrive at a solution. How the hell is that a game?

Furthermore, the time you just spent and/or wasted on the "game" was shrouded in the mysteries of some stupid, silicon-valley wank mythology that was made up from the seat of their ass. So you don't learn anything factual about things like prion folding or variable bit widths or stack leaks or whatever. No, you just learn some made-up Californian crap about "the storms that devastated Aeryth" or "Gee these plugs and gizmos aren't hooking together correctly, get the thingamabobbers all the same color for the point!"

So what are you doing? Wasting your time ten-fold. Don't do it. Fuck these people. It seems like a good premise but they obviously handed the work off to the entirely wrong group of people.

The only people this will be interesting to is disabled children who have real difficulties socializing out of doors and who spend inordinate amounts of time chair-bound in front of the computer, or autistic people, or absolute 100% genuine geeks who are totally oblivious to things like the value of time well spent or what the meaning of "quixotic" is.

Re:Doesn't work

[eyenot]

The games rely on varying layers of compromised or compromiseable browser attachments and plugins. If you are concerned about your system security, then they definitely aren't the games for you. Requirements range from Adobe Flash to Unity Engine.

Re:Does nginx suffer from poor scalability?

[foobar bazbot]

I don't know much about nginx, but any time I hear about it it's usually because of an error message like that.

Even though it isn't used nearly as much as Apache is, I must see an nginx error page like that at least two or three times a month. I can't say the same for Apache, or IIS, or Lighttpd, or any other major web server these days.

This means the opposite of what you seem to think.

Does nginx just suffer from really bad scalability under any sort of significant load? Is poor scalability and load tolerance the reason why it starts giving 500 Internal Server Error responses and error pages like that so commonly?

Short answer: No.

Nginx is very commonly used as a reverse proxy (for caching, encryption, load balancing, etc.) in front of another web server (might be another nginx instance, but it's commonly some other, more popular web server, nudge-nudge-wink-say-no-more). When the proxied web server doesn't respond in time, the proxying nginx returns a "500 Internal Server Error". While this isn't the only reason for a 500, it's far the most common from nginx.

So when you see that error, it doesn't mean the nginx instance that gives you the error fell over, it almost always means that nginx instance is doing fine, but some other web server (which could also be nginx, but is most likely not nginx) fell over.

Re:Well ....... excellent idea for Linux Kernel

[AndrewBuck]

The game you are referring to is Foldit. http://fold.it/portal/ I played it a bit back when it came out and it was an interesting game. It has even been used to find some protein folding solutions that had previously stumped the existing tools used to look for solutions. It doesn't beat the traditional science in every instance (or probably even that many) but having an extra tool in the toolbox never hurts, especially when it is a tool that can be used by thousansds of players with time to kill instead of a handful of highly trained specialists with very limited time.

I think this is a really interesting idea. A poster above who was a grad student who worked on an earlier version of this game pointed out that the goal of these was to find loop invariants in the software (basically a proof by induction that a loop does what you think it does).

With the recent revelations about the NSA backdooring common encryption code I have wanted people to work on something like this to try to 'prove' various software does what it says on the box (PGP, linux kernel, tor, etc). I am glad to hear that there is research being done in this area and hope it succeeds and gets applied to some of the important open source software in use today. Let the unwashed masses do most of the grunt work proving the simple bits like loop invariants and use that to free up the specialist developers to look at the rest of the program.

-AndrewBuck