PunkSPIDER Project Puts Vulnerabilities On (Searchable) Display

First time accepted submitter punk2176 writes "Recently I started a free and open source project known as the PunkSPIDER project and presented it at ShmooCon 2013. If you haven't heard of it, it's at heart, a project with the goal of pushing for improved global website security. In order to do this we built a Hadoop distributed computing cluster along with a website vulnerability scanner that can use the cluster. Once we finished that we open sourced the code to our scanner and unleashed it on the Internet. The results of our scans are provided to the public for free in an easy-to-use search engine. The results so far aren't pretty." The Register has an informative article, too.

Comments

Re:Next - SE for houses without security systems

[Anonymous Coward]

That's not a perfect metaphor and you know it, because if your house is insecure it puts you in danger. If your website is insecure it puts the users in danger. If your house has no security system, you're personally aware. If your website is insecure you likely do not.

I'm not making an argument as to whether or not this is a good idea, but you're over simplifying things on purpose.

Ethics

[gignac.adam]

Funny; my professor just told a networking class recently when discussing vulnerability scanners that it was seriously unethical to scan a system without permission - it would be like walking through a parking lot and checking which cars are unlocked. I think most people would agree with him. This project might have good intentions, trying to encourage the sysadmins to tighten up their security, but I think there's a better way to do it than public shaming.

Re:Ethics

[GrumpySteen]

They're scanning large numbers of websites and putting vulnerability information up on a public website for anyone to view without notifying the website owners, much less giving them a chance to fix the problems before sending hackers their way. There's nothing ethical about that.

Re:Ethics

[GrumpySteen]

I'm fairly sure those don't exist. Even the guarded parking lots have disclaimers that say they aren't responsible for theft and damage.

Re:Ethics

[raymorris]

In the past hackers used to notify the owners and give them a chance to fix problems and the owners would either do nothing or even threaten to sue. Any slashdot reader should know this.

Anyone claiming to know anything about the topic of web security should know the procedure used to remedy ~90% of all vulnerabilities. Those security updates you get each week don't appear out of nowhere. Someone like myself files a security ticket with the vendor or affected party. The vulnerability is confirmed and analyzed, then other vendors who are likely to have similar vulnerabilities are notified. A patch is pushed, THEN a CVE is issued. After that, more mainstream sites like slashdot pick up on, and link to, the CVE which explains what the vulnerability was is links to the update to fix it. That's typically about a week after the vulnerability is teported and 2-3 days after the fix is available. That's how securitu issues are normally handled, they aren't ignored. (If they were ignored we wouldn't average 100 security notices per week, would we?)

When I found the PowerDNS vulnerability I could have come straight to Slashdot with "how to take down wikipedia and millions of other sites". If I were a scumball attention whore I would have done so. Instead, I reported it through proper channels. Wikipedia was patched within 36 hours, then other sites. The next day, the CVE went out, THEN you heard about it. I still get to brag - I just do it AFTER a) wikipedia and other responsive sites are safe and b) I have something worth bragging about, having protected wikipedia from being exploited.
This jackass is merely attention whoting at other people's expense. He hasn't done anything special - just ran Nessus - but is advertising himself via the results rather than handling them responsibly.

Suppose for a moment that some of the sites could leak sensitive information. Suppose also that sites which leak sensitive information should be slapped. Well, the slashvertised site, the cracker's search engine, is most certainly leaking sensitive information, ergo he should be slapped!

Re:Ethics

[Jafafa Hots]

You professor is an idiot.

This is more like going to a public parking lot and testing to find out whether the security cameras are real and working, not working, or fakes, and then telling people they shouldn't park their cars there if they want to park where there are security cameras.

Re:Ethics

[punk2176]

Hmm, a few issues with this...

1) The statement that we "just run Nessus" is incorrect. We wrote our own scanner that works on a Hadoop cluster. Why is this important? It means that we can handle a lot more scans than anyone else (several thousand per day with a small cluster) and it's also specifically made for mass scans. This is important in point 2 below.

2) The process you're describing is for finding a vulnerability in a piece of software in general (e.g. a common CMS), not a specific vulnerability in an implementation of a piece of software (e.g. a specific website). That's a huge difference. You wouldn't put a CVE up for a SQL injection bug in a specific implementation of a site (you would only if it was common to an entire CMS for example). Anyway, what we hope is to build a community of like-minded security folks that can help those website owners fix their *specific issues* first and if applicable go through the process you describe when needed. We also want to provide this for free.

3) What if the vulnerability is in a custom built site that no one cares enough about to do security research on. Who's letting them know their issues? We hope to provide a view of this to the website owner and yes, push them a little to get their security ducks in a row.

4) We're not attention whores or jackasses. Calling people names isn't nice and makes us sad.

Re:Ethics

[Thruen]

Actually, it's less like telling people they shouldn't park there, and more like creating a searchable database of areas with no security cameras. It doesn't take much thought to realize the people looking for a place to park aren't going to search this database, it'll be used by the people looking for safer areas to steal those cars. Or to drop all this stupid analogy crap that never seems to have a positive effect on discussions, this is a searchable database that's only going to be used by people who are looking for vulnerabilities. Is the average user even going to know about this? Nope, but hackers everywhere will definitely know about it. Odds are, site owners aren't going to search it either, which may be a little irresponsible, but is nowhere near as bad as finding the vulnerabilities and trying to make sure everyone knows about it. They claim good intentions, and maybe they really have good intentions, but there's nothing good about this.

Re:Ethics

[BemoanAndMoan]

We hope to provide a view of this to the website owner and yes, push them a little to get their security ducks in a row.

No, you don't. If you did you'd have built your system to make *them* aware first, instead of posting a "don't blame the messenger" shame tool that exposes their vulnerabilities.

The hacking-promotes-security argument is weak sauce, even more so in your case. The vast percentage of people you've exposed (i.e. not anonymous mega-corps, but rather small mom-and-pops set up and left un-managed by unskilled sysadmins, innocuous self-hosting newbies, etc.) will likely never encounter your list, even after it provides scriptkiddies with an easily digestible list of opportunities who wipe their servers and turn them into warez hubs only to be rinse-repeated because they will *never* know any better.

You are merely a new vector for the disease, selling itself as a cure. Where in this is your moment to feel proud?

Re:Next - SE for houses without security systems

[Jah-Wren Ryel]

Well, at least one difference is that when a website gets hacked it is almost always the people visiting the website who are the target because the goal of the hacker is either to grab information about those users from the hacked system or to use the hacked system to distribute exploits to anyone that browses there.

While when a house is broken into, it is basically a problem for the owners of the house and not really anyone else.

So publishing a list of vulnerabilities on websites serves the purpose of shaming the website operators into better protecting their users.

Re:Next - SE for houses without security systems

[tibman]

I view it as a list of dark alleys you shouldn't walk down.

Re:Couldn't find any - the results so far ARE pret

[punk2176]

So one thing that we've been trying to make clear is that the project is *on track* to scan the entire Internet, we haven't scanned everything yet. We have scanned about 70k sites and have under 4 million indexed. Our next version is going to be clearer on what is and is not scanned - currently we just say 0 vulnerabilities if we haven't scanned it, indicating that we have not found vulnerabilities in it yet - not necessarily that it doesn't have any. This was all part of our ShmooCon presentation which just hasn't been released to everyone yet! The system is self-sustaining at this point so these numbers are constantly going up. The "not pretty" comes from the fact that we have over 100,000 vulnerabilities from just scanning about 70,000 sites (some sites have multiple vulnerabilities).

Re:Couldn't find any - the results so far ARE pret

[Sqr(twg)]

Typing * in the search box gets you everything, it seems.

  762 pages (times 10 sites per page) for "bsqli"
  77 pages for "sqli"
  421 pages for "xss"