PunkSPIDER Project Puts Vulnerabilities On (Searchable) Display

← Back to Stories (view on slashdot.org)

PunkSPIDER Project Puts Vulnerabilities On (Searchable) Display

Posted by timothy on Saturday February 23, 2013 @03:11PM from the anarchist's-cookbook dept.

First time accepted submitter punk2176 writes "Recently I started a free and open source project known as the PunkSPIDER project and presented it at ShmooCon 2013. If you haven't heard of it, it's at heart, a project with the goal of pushing for improved global website security. In order to do this we built a Hadoop distributed computing cluster along with a website vulnerability scanner that can use the cluster. Once we finished that we open sourced the code to our scanner and unleashed it on the Internet. The results of our scans are provided to the public for free in an easy-to-use search engine. The results so far aren't pretty." The Register has an informative article, too.

50 of 85 comments (clear)

Min score:

Reason:

Sort:

Next - SE for houses without security systems by raymorris · 2013-02-23 15:58 · Score: 1

Well hours next project be a search engine showing which houses don't have good security systems, or showing the weaknesses in each home's security? What an aweful way to attention whore - by giving criminals a list of defenseless people.
1. Re:Next - SE for houses without security systems by Anonymous Coward · 2013-02-23 16:02 · Score: 2
  
  That's not a perfect metaphor and you know it, because if your house is insecure it puts you in danger. If your website is insecure it puts the users in danger. If your house has no security system, you're personally aware. If your website is insecure you likely do not.
  I'm not making an argument as to whether or not this is a good idea, but you're over simplifying things on purpose.
2. Re:Next - SE for houses without security systems by __aajfby9338 · 2013-02-23 16:15 · Score: 1
  
  I don't see how this is different than publishing a searchable database of unlocked doors that I found in my neighborhood, with the claim that my purpose is to improve my neighborhood's security. I do not see this as oversimplification. A group (gaggle? herd?) of tweakers could use the database to find an unlocked house whose owners are on vacation and then squat there, using it as a base to burgle other houses in my neighborhood, just as malicious hackers could host malware on a vulnerable site. It's still a dick move to publish the list of unlocked doors for all to see.
3. Re:Next - SE for houses without security systems by ArsonSmith · 2013-02-23 17:29 · Score: 1
  
  Yea, let's release a list of known non-gun owners.
  
  --
  Paying taxes to buy civilization is like paying a hooker to buy love.
4. Re:Next - SE for houses without security systems by Jah-Wren+Ryel · 2013-02-23 17:53 · Score: 3, Insightful
  
  Well, at least one difference is that when a website gets hacked it is almost always the people visiting the website who are the target because the goal of the hacker is either to grab information about those users from the hacked system or to use the hacked system to distribute exploits to anyone that browses there.
  While when a house is broken into, it is basically a problem for the owners of the house and not really anyone else.
  So publishing a list of vulnerabilities on websites serves the purpose of shaming the website operators into better protecting their users.
  
  --
  When information is power, privacy is freedom.
5. Re:Next - SE for houses without security systems by tibman · 2013-02-23 18:08 · Score: 2
  
  I view it as a list of dark alleys you shouldn't walk down.
  
  --
  http://soylentnews.org/~tibman
6. Re:Next - SE for houses without security systems by BemoanAndMoan · 2013-02-23 23:08 · Score: 1
  
  So publishing a list of vulnerabilities on websites serves the purpose of shaming the website operators into better protecting their users.
  So by that logic, I assume you rape every woman you pass on a dark street, mug the elderly who don't go out in groups, and commit every other crime of opportunity to shame people into what *you* consider proper, minimum safe behavior. How brave and noble of you.
  I'm so tired of people dressing up shitty behavior under the guise of protecting them when really all they are doing is being selfish, self-satisfying little asshats.
  If this guy wasn't such a douche, he'd be emailing the websites a notice letting them know of the vulnerabilities, not making the list available for everybody. This would have been a good example of how decent behavior could have helped protect both visitors and the site owners, instead of what at best will become a life lesson taught through severe litigation and (if we are lucky) state prosecution.
7. Re:Next - SE for houses without security systems by Kiwikwi · 2013-02-23 23:40 · Score: 1
  
  So by that logic, I assume you rape every woman you pass on a dark street, mug the elderly who don't go out in groups, and commit every other crime of opportunity to shame people into what *you* consider proper, minimum safe behavior. How brave and noble of you.
  Phew. For a moment I was worried this thread would descend into hyperbole and strawman arguments.
Ethics by gignac.adam · 2013-02-23 16:11 · Score: 3, Insightful

Funny; my professor just told a networking class recently when discussing vulnerability scanners that it was seriously unethical to scan a system without permission - it would be like walking through a parking lot and checking which cars are unlocked. I think most people would agree with him. This project might have good intentions, trying to encourage the sysadmins to tighten up their security, but I think there's a better way to do it than public shaming.
1. Re:Ethics by DCisforBoners · 2013-02-23 16:24 · Score: 1
  
  Some of these sites are likely entrusted with sensitive user information. The car analogy is only apt if you borrowed $100 from a couple of your closest friends for rent and left that in the car you forgot to lock while you were getting a taco. As I see it, the benefit of this type of public shaming is it reinforces in end users the idea that you should be careful who you trust with your data. For admins, if the majority of listed sites use web technology "x", maybe if you're designing a new site you look for an alternative.
2. Re:Ethics by kermidge · 2013-02-23 16:46 · Score: 1
  
  "....I think there's a better way to do it than public shaming."
  Ok, such as.... what?
  If someone puts up a web site I have to figure that it might be for people to visit. If that site has vulnerabilities, I have to give the owner benefit-of-doubt that they might likely want to know such, as I also have to figure that they wish for it to be safe from attack - to prevent defacement, hi-jacking for attack app insertion, making off with private infos, etc.
  Therefore I'd hazard a guess that, since everyone's means are limited, by their own knowledge and skill, time available, to do their own testing, or budget to hire it done, that whatever agency is able to easily and quickly point out a few of the more common vulns (and also the same ones used by many of the crims to make money) - that they'd welcome the info so they might fix their site.
  Punkspider seems to fit that bill.
  So, unethical. How so? Is it somehow more ethical to not test and have site open to attack, or is it both more moral and more practical to get information that'll help to protect the site?
3. Re:Ethics by del_diablo · 2013-02-23 17:01 · Score: 1
  
  The analogy falls apart if the parking lot is a guarded parking lot that guarantees the cars safety.
4. Re:Ethics by GrumpySteen · 2013-02-23 17:19 · Score: 2
  
  They're scanning large numbers of websites and putting vulnerability information up on a public website for anyone to view without notifying the website owners, much less giving them a chance to fix the problems before sending hackers their way. There's nothing ethical about that.
5. Re:Ethics by GrumpySteen · 2013-02-23 17:21 · Score: 2
  
  I'm fairly sure those don't exist. Even the guarded parking lots have disclaimers that say they aren't responsible for theft and damage.
6. Re:Ethics by ArsonSmith · 2013-02-23 17:47 · Score: 1
  
  I think a better example may be someone going through stores and seeing which ones are posting people's credit card info on a large board behind the cashier for all to see or the ones that are actually trying to keep it hidden. That is seriously about how stupid many of these web sites are and if this was happening in meat space this list would be uncontested as a supreme public service.
  
  --
  Paying taxes to buy civilization is like paying a hooker to buy love.
7. Re:Ethics by raymorris · 2013-02-23 18:18 · Score: 5, Informative
  
  In the past hackers used to notify the owners and give them a chance to fix problems and the owners would either do nothing or even threaten to sue. Any slashdot reader should know this.
  Anyone claiming to know anything about the topic of web security should know the procedure used to remedy ~90% of all vulnerabilities. Those security updates you get each week don't appear out of nowhere. Someone like myself files a security ticket with the vendor or affected party. The vulnerability is confirmed and analyzed, then other vendors who are likely to have similar vulnerabilities are notified. A patch is pushed, THEN a CVE is issued. After that, more mainstream sites like slashdot pick up on, and link to, the CVE which explains what the vulnerability was is links to the update to fix it. That's typically about a week after the vulnerability is teported and 2-3 days after the fix is available. That's how securitu issues are normally handled, they aren't ignored. (If they were ignored we wouldn't average 100 security notices per week, would we?)
  
  When I found the PowerDNS vulnerability I could have come straight to Slashdot with "how to take down wikipedia and millions of other sites". If I were a scumball attention whore I would have done so. Instead, I reported it through proper channels. Wikipedia was patched within 36 hours, then other sites. The next day, the CVE went out, THEN you heard about it. I still get to brag - I just do it AFTER a) wikipedia and other responsive sites are safe and b) I have something worth bragging about, having protected wikipedia from being exploited.
  This jackass is merely attention whoting at other people's expense. He hasn't done anything special - just ran Nessus - but is advertising himself via the results rather than handling them responsibly.
  
  Suppose for a moment that some of the sites could leak sensitive information. Suppose also that sites which leak sensitive information should be slapped. Well, the slashvertised site, the cracker's search engine, is most certainly leaking sensitive information, ergo he should be slapped!
8. Re:Ethics by kermidge · 2013-02-23 18:25 · Score: 1
  
  Thank you; I learned something. Several somethings, in fact.
9. Re:Ethics by Jafafa+Hots · 2013-02-23 20:18 · Score: 2
  
  You professor is an idiot.
  This is more like going to a public parking lot and testing to find out whether the security cameras are real and working, not working, or fakes, and then telling people they shouldn't park their cars there if they want to park where there are security cameras.
  
  --
  This space available.
10. Re:Ethics by punk2176 · 2013-02-23 20:32 · Score: 2, Informative
  
  Hmm, a few issues with this...
  
  1) The statement that we "just run Nessus" is incorrect. We wrote our own scanner that works on a Hadoop cluster. Why is this important? It means that we can handle a lot more scans than anyone else (several thousand per day with a small cluster) and it's also specifically made for mass scans. This is important in point 2 below.
  
  2) The process you're describing is for finding a vulnerability in a piece of software in general (e.g. a common CMS), not a specific vulnerability in an implementation of a piece of software (e.g. a specific website). That's a huge difference. You wouldn't put a CVE up for a SQL injection bug in a specific implementation of a site (you would only if it was common to an entire CMS for example). Anyway, what we hope is to build a community of like-minded security folks that can help those website owners fix their *specific issues* first and if applicable go through the process you describe when needed. We also want to provide this for free.
  
  3) What if the vulnerability is in a custom built site that no one cares enough about to do security research on. Who's letting them know their issues? We hope to provide a view of this to the website owner and yes, push them a little to get their security ducks in a row.
  
  4) We're not attention whores or jackasses. Calling people names isn't nice and makes us sad.
11. Re:Ethics by Reemi · 2013-02-23 20:50 · Score: 1
  
  How about you borrowed your expensive camera to a friend and you noticed he left it visible inside his car on the parking lot. A parking lot known for many burglaries, in other words he was inviting somebody to steel your camera.
  Would it be ethical to check if he locked his car so you can protect YOUR OWN belongings?
  I agree with your statement, but looking at my log-files I wonder why the good guys should not be allowed to perform one scan while the bad ones are performing hundreds a day. Why should the bad ones have all the information?
12. Re:Ethics by martin-boundary · 2013-02-23 21:41 · Score: 1
  
  The basic car analogy fails to capture that vulnerabilities in computer systems are often used as stepping stones for further attacks on other computer systems. In the car analogy proper, the only person affected by a break-in is the unlocked car's owner, while the other car owners are safe provided their car doors are locked.
  But say the criminal is a joyrider. He picks an unlocked car, and then drives around the parking lot smashing into other locked cars for fun, and then runs away. Now the question: is it wrong to check if some other car in the lot is unlocked and shame its owner? The chance that your car, even if locked, will be damaged due to some other car being unlocked and used as an attack vector is now non negligible.
13. Re:Ethics by blackest_k · 2013-02-23 22:08 · Score: 1
  
  On the other hand you could just check the sites you manage and design with this tool and see if it finds any problems. It is important your website is standards compliant and it is just as important that your site is secure. If you ever got hacked you will soon find your site blacklisted and a pile of work to rebuild the site and more importantly restore your reputation.
  Without this tool you will be hacked eventually if your site has vulnerabilities, it has to be a good thing for you to know beforehand so you can secure your site.
  It would be even better if after scanning a site that it sends an email to the site info@whatever.com is a fairly safe bet. If you haven't used the tool at least you know someone else has and you need to know hopefully before your site gets hacked.
  This tool might lower the bar for script kiddies but there are plenty of people who can hack your site without using this particular tool. You only need one to be successful.
  
  --
  Blarney Quality Restaurant, Plants
14. Re:Ethics by Thruen · 2013-02-24 01:18 · Score: 2
  
  Actually, it's less like telling people they shouldn't park there, and more like creating a searchable database of areas with no security cameras. It doesn't take much thought to realize the people looking for a place to park aren't going to search this database, it'll be used by the people looking for safer areas to steal those cars. Or to drop all this stupid analogy crap that never seems to have a positive effect on discussions, this is a searchable database that's only going to be used by people who are looking for vulnerabilities. Is the average user even going to know about this? Nope, but hackers everywhere will definitely know about it. Odds are, site owners aren't going to search it either, which may be a little irresponsible, but is nowhere near as bad as finding the vulnerabilities and trying to make sure everyone knows about it. They claim good intentions, and maybe they really have good intentions, but there's nothing good about this.
15. Re:Ethics by BemoanAndMoan · 2013-02-24 01:22 · Score: 4, Insightful
  
  We hope to provide a view of this to the website owner and yes, push them a little to get their security ducks in a row.
  
  No, you don't. If you did you'd have built your system to make *them* aware first, instead of posting a "don't blame the messenger" shame tool that exposes their vulnerabilities.
  The hacking-promotes-security argument is weak sauce, even more so in your case. The vast percentage of people you've exposed (i.e. not anonymous mega-corps, but rather small mom-and-pops set up and left un-managed by unskilled sysadmins, innocuous self-hosting newbies, etc.) will likely never encounter your list, even after it provides scriptkiddies with an easily digestible list of opportunities who wipe their servers and turn them into warez hubs only to be rinse-repeated because they will *never* know any better.
  You are merely a new vector for the disease, selling itself as a cure. Where in this is your moment to feel proud?
16. Re:Ethics by Thruen · 2013-02-24 02:01 · Score: 1
  
  I'll help resolve these issues for you.
  
  1) The software used is a very minor part of the point, and as far as the ethics argument goes means literally nothing.
  
  2) The start of the process he's describing, reporting the bug to the people who can deal with it, is the important step that doesn't change. Yes, it is different than dealing directly with software developers. It also means they probably aren't capable of fixing it so quickly. The software developers have a huge edge in that area. It does need to be dealt with differently, but public shaming without giving them a chance to fix it is not dealing with the problem it's exasperating it.
  
  3) If it's a custom site that no one cares to do security research on, chances are nobody's looking to attack that site anyway, until it's posted on a search engine calling it out as a target. As for who's letting them know, adding them to a search engine is NOT letting them know. Odds are these smaller sites aren't out there looking to see if anyone's found a vulnerability in their site. I didn't see anything that states you warn the site owner and even attempt to give them any time to fix it. If you want someone to get their security ducks in a row, step one is telling them everything you know about their problem, and step two is giving them time to fix it. If putting them in a searchable database is any step, it's much further along.
  
  4) I can't say much about this point as it truly depends on your intentions. You may have good intentions and think this is a great thing to do, but you are most certainly going about it all wrong. You shouldn't be advertising this on Slashdot, you should be emailing webmasters everywhere to tell them about their vulnerabilites. And while I'm here, a feature suggestion: an option to search by software used rather than the specific address, that way people can search the software they're using and find out if any specific implementation is vulnerable without relying on their site already being in this database. And before you say it, yes, web developers could and should run these scans on their own, I'm not defending sloppy security at all, but the developers who aren't running the scans also aren't going to go search for their site on yours without being told they have a problem.
17. Re:Ethics by Zapotek · 2013-02-24 02:30 · Score: 1
  
  The prof gave a wrong simile, you are an idiot. WebAppSec scanners can inject harmful payloads (like emptying whole DB tables harmful, a simple string like "or 1=1" in the wrong place can can cause loads of trouble) and should be never run again live/production websites.
  
  Also, those guys are overly excited about their own work to the point or arrogance but give them time. They'll either get to appreciate all the complexities of those types of systems and power on or just give up after a while.
  They got the attention they wanted now anyways...
18. Re:Ethics by WoOS · 2013-02-24 03:28 · Score: 1
  
  Now let's not get too harsh.
  How 'good' or 'bad' this database is depends IMHO partially on its query interface. If one can only ask for single (FQDN) URLs (with a query rate limit) and gets the vulnerabilities of that specific URL as an answer (plus maybe some pointers on what the vulnerable software likely is), it might actually be useful for the somewhat technically-inclined web owner. The proposed list of vulnerable software would probably not help them as they would have to remember which SW their site runs.
  If on the other hand one can simply search the database for "give me all sites suffering from vulnerability X", that is not helping a web site owner at all but a cracker very much.
  The OP's site seems to be slashdotted (seems Hadoop is not applied in the frontend) so I cannot check but from the comments here it seems it implements the second option. That is definitely something that should be changed.
  The other thing any web site owner should take from this article is that Mass vulnerability spidering has reached mainstream. Much better to announce that on slashdot (and as many other news sites, magazines, periodicals as possible) than have people discover it in a year on their own.
  Now the next useful topic on slashdot would be a discussion on: "Should I host my own site, blog, shop, ... or better use a service (doing all the security stuff)." But I have to check on my wordpress software version ;-)
19. Re:Ethics by Thruen · 2013-02-24 05:03 · Score: 1
  
  I'm really not trying to be harsh, sorry for coming off that way. I'm probably a little biased because I have experience being a small business IT guy by default (as in no real training, just better with computers than the other people there), so that's really who I relate to the most. In my position I understood sometimes you need to seek help elsewhere, and I did, but I also learned that problems aren't as easy to fix as people think, nor are they always cheap, and money can be a big issue for a small business. I would've appreciated the help if PunkSPIDER sent me an email describing a vulnerability, and I would've tried to fix it quickly, but if I couldn't figure it out myself I'd have to call someone else in, which gets expensive and might take time to budget it. Instead, PunkSPIDER just puts it in their search engine, unbeknownst to me the lowly inexperienced IT guy, so instead of me being able to fix it before it becomes a problem I still don't find out about it until it's a bigger issue. I know, the heart of the problem is having an IT guy who doesn't have the proper training, but if you've ever worked at a small business during hard times you know spending more isn't always an option, sometimes you need to work with what you have, and that's what they did, I was there so they used me.
  
  On to everything else, it is actually closer to the first implementation you describe, although I don't know about the rate limit as I didn't test it when the site was still loading quickly earlier.
  
  To clarify my suggestion of searching by software, I didn't intend for it to list addresses, only vulnerabilities related to that software. As for the website owner not being able to remember what software they used, without getting into how one sets up and runs a web site without even knowing what software they're using, if they're that technically deficient they almost definitely won't be able to fix it themselves (if they even understand the information they're looking at) and should already be looking for someone to handle the technical end of things.
  
  And as for what they should take from the article, it is mainstream and people should know about it. However, to use a more extreme example, the same can be said for copyright infringement, but would advertising The Pirate Bay or isoHunt really be the right way to alert people to that fact?
  
  As for the discussion about whether you should host your own site or blog, it seems pretty straightforward to me. If you have the technical know-how and understand what you're doing as well as the costs involved, go ahead and run it yourself. But if you lack that knowledge, it's a silly question to ask. It's like anything else, just because you can make something work doesn't mean it's a good idea to do it yourself. I've been in that position, trying to fix something that's over my head, and while I could generally make it work, it was never as good as having the professionals fix it. If you want an analogy (because those are popular here) you can really swap out web development with any other skill in the world. Just because you can figure out how to (fix your car, plumb/wire your house, build any sort of structure, sew your own clothes, stitch your own cut, tow a car) doesn't mean you should do it yourself instead of leaving it to the professionals. And yes, I know I pointed out why that's not always easy or in the budget, it may not always be an option for people, but if it is an option it's the right one.
  
  I drifted a bit off the real topic in the end here, my bad.
20. Re:Ethics by gawdric · 2013-02-24 06:06 · Score: 1
  
  Public shaming has its place. Think back, if you were involved then, to the mid to late 90s. Smurf attacks were the in thing. Places like powertech.no and others began listing networks that had smurfable broadcast addresses. For a while, this did the work for the script kiddies but eventually the networks made it a point to remove themselves from the database and now the problem is nearly gone. I can see this database having a similar impact.
21. Re:Ethics by GrumpySteen · 2013-02-24 07:39 · Score: 1
  
  > Why do I feel that most of the people claiming this is unethical are all the same person.
  Because dismissing all of the people who are telling you that you're wrong as one nutcase with multiple accounts is far easier than acknowledging that you're actually wrong and everyone knows it.
22. Re:Ethics by Dextrously · 2013-02-26 09:43 · Score: 1
  
  I see a note on the punkspider site to opt out of having your site scanned. Is there a specific way to opt in as well? I would be interested in seeing what results could come out of scanning a few of my sites. I've tried using Skipfish in the past, and a few other scanning utilities, and got a lot of false positives, and also a lot of missed positives. Things I knew were vulnerable and just wanted to see if the scanner would pick up on it.
  Thanks for the great work! I look forward to seeing the results, even if some people don't like it. Perhaps sending a notice to "webmaster@domain.tld" would suffice? Possibly even a month or so. Just something along the lines of:
  
  Hey, found some vulnerabilities on your site, this is what they are......
  DOMAIN.TLD is currently in queue to be listed in our database on 00/00/2013: click here to request a time extension (or possibly a removal from the list completely) before listing, or click here to queue up another scan of your site when the vulnerabilities are patched.
  If you would like assistance in fixing these vulnerabilities, feel welcome to come to our forums or read [link to OWASP info] more information.
  
  I know I would personally appreciate such an automated approach.
  Personally, I don't think there should be a removal from the list option if adequate time is given. That's just my opinion though. I feel like if you've done your due diligence to notify the maintainers, after a month or two of time, the public should have a right to know so that they can be avoided.
uncheck some boxes by Bananatree3 · 2013-02-23 16:17 · Score: 1

try "The" or "and" or other basic title searches - and also uncheck some of the vulnerability boxes, and you'll see examples.
Maybe. 99% are not by raymorris · 2013-02-23 17:10 · Score: 1

some of the of sites are likely entrusted

And 99.97% are some guy trying to make ends meet by offering online chemistry lessons or showing you how to hook up your home theatre. IF there were any sites found that held personal information, the right thing to do would be to contact those sites, not encourage people to hack the personal information.

Certainly it does no good whatsoever to give script kiddies a list of sites to deface. The most popular hosting is Godaddy, with their $10 / month hosting account. (35% of sites are Godaddy sites.) The sites with hosting budgets of around $10-$50 month make up 95% of all sites. So that's mostly who is affected - some elementary school art teacher selling used computer parts online in his spare time.
1. Re:Maybe. 99% are not by DCisforBoners · 2013-02-23 17:48 · Score: 1
  
  That's a fine point, but if you were that consumer shopping to implement a prefab site, wouldn't you like to know if the technical foundations are sound? If 1000 GoDaddy sites are hacked in a day maybe that prompts a response from the host.
The truth is... by Anonymous Coward · 2013-02-23 17:34 · Score: 1

It's a tool. Tools can be used for good and evil, it just depends on who's hands the tool is in. Take Metasploit for example -- it's used widely by both whitehat security researchers and blackhat criminals.
As a security researcher, I'll add that PunkSPIDER doesn't shine light on anything that the bad guys don't already know. I'm glad to see another tool that helps enable those who are charged with defending web applications.
Couldn't find any - the results so far ARE pretty by G3ckoG33k · 2013-02-23 17:46 · Score: 1

Tried two dozen sites that I visit regularly. No issues. Most are top 100,000 on alexa but a few below 1,000,000.
Lawsuits and ethics. by sdsucks · 2013-02-23 19:13 · Score: 1

I hope you've got a good lawyer and money to keep him or her happy. The first exploit you publish about a large (organization|government|important person) is going to give you a really, really, really big headache - at best.
Also... ethics - you have none. For this, as someone who has spent past lives working in IS, I hope you rot in a miserable existence.
Fame grasping by a very amateur security "expert".
Re:Couldn't find any - the results so far ARE pret by punk2176 · 2013-02-23 19:45 · Score: 3, Informative

So one thing that we've been trying to make clear is that the project is *on track* to scan the entire Internet, we haven't scanned everything yet. We have scanned about 70k sites and have under 4 million indexed. Our next version is going to be clearer on what is and is not scanned - currently we just say 0 vulnerabilities if we haven't scanned it, indicating that we have not found vulnerabilities in it yet - not necessarily that it doesn't have any. This was all part of our ShmooCon presentation which just hasn't been released to everyone yet! The system is self-sustaining at this point so these numbers are constantly going up. The "not pretty" comes from the fact that we have over 100,000 vulnerabilities from just scanning about 70,000 sites (some sites have multiple vulnerabilities).
90% of everything is crud. by Tenebrousedge · 2013-02-23 19:46 · Score: 1

The most popular hosting is Godaddy, with their $10 / month hosting account. (35% of sites are Godaddy sites.) The sites with hosting budgets of around $10-$50 month make up 95% of all sites.
As a web developer I may say categorically, fuck them. If you put a site on the web, it is your responsibility to make sure that it is secure. If you are not able to do that to a professional standard, you should not do it. In point of fact, there is a need for a licensing organization to prevent amateurs from practicing web development. The problem isn't that this website is exposing poor security practices, it's that it's not promoting professionalism.
We have passed the point where it's okay for the layman to host a site. Even if you're not collecting information about your users, you can still be attacked or used as an attack vector. The era of democratization of the web is over.

--
Those who advocate genocide deserve every protection afforded by law, and none afforded by common human decency.
1. Re:90% of everything is crud. by Anonymous Coward · 2013-02-23 23:38 · Score: 1
  
  Could you please post a list of your client's websites? :)
2. Re:90% of everything is crud. by Zmobie · 2013-02-24 07:55 · Score: 1
  
  Actually I do somewhat agree with the spirit of this post here. Software Engineering is a discipline that can affect large amounts of people and where not many people actually understand it. This is very similar to any other type of engineering (civil, nuclear, electrical, etc.) and to practice those other disciplines generally you have to have a P.E. or at least a P.E. signs off on the work done. Under current models, this is not in any way required for software and while most of your real software engineers don't really need to have this, for every one of them there are 10 or more idiots that picked up a "Complete Idiot's Guide to PHP" and started throwing websites up. The entire point of the license model is to ensure quality of work because engineers are working on things that affect the public tremendously.
  Now, playing devil's advocate to my own point, you can also argue that the no license required is how the web and software grew like it did. There have been plenty of great projects and ideas put out there that would not have been if a P.E. were required to sign off on the work. Indie development would all but die under this kind of model or at best the cost would increase significantly because of needing a P.E. to review everything after the fact and point out the real problems.
  Kind of a double edged sword honestly, but there is a valid point in Tenebrousedge's post.
3. Re:90% of everything is crud. by fast+turtle · 2013-02-24 17:30 · Score: 1
  
  and the god damn license isn't worth the paper it's printed on. What makes the difference is that an Engineer is Legaly Liable for any screwups. In other words, they've got a bit more at risk then Joe Sixpack who's throwing shit at the web to see what in hell is going to stick. Until the liability issue changes for web developers, nothing is going to change
  
  --
  Mod me up/Mod me down: I wont frown as I've no crown
4. Re:90% of everything is crud. by Zmobie · 2013-02-25 14:09 · Score: 1
  
  I'm a little confused here, your post kind of contradicts itself. You say the license isn't worth the paper it is printed on, but then say
  
  an Engineer is Legaly Liable for any screwups
  The license is what allows someone to legally be an engineer for most disciplines. We went over this when I took my engineering ethics course back in college, and there have been numerous (some very frivolous in fact) lawsuits to keep people from using the term or actually practicing any form (of licensed) engineering. The only current exception to this is software engineers can legally use the term without a valid license because one doesn't exist.
  The entire point of what I said in the first half of my post is that a P.E. would in fact make those software engineers legally liable for their work (within reason, even in other engineering disciplines things happen you just have to show that they took reasonable steps and practices to try and prevent it) therefore doing precisely what you said and shifting the liability on to these web developers.
Re:Couldn't find any - the results so far ARE pret by Sqr(twg) · 2013-02-23 20:34 · Score: 2

Typing * in the search box gets you everything, it seems.
762 pages (times 10 sites per page) for "bsqli"
77 pages for "sqli"
421 pages for "xss"
TL;DR by thejynxed · 2013-02-24 00:05 · Score: 1

Most web sites aren't written with security in mind, but pageviews, rankings, and advertising revenue. News at 11.
How about that NASCAR race?

--
@Mindless Drivel: 100% of Twitter posts ever Tweeted.
Re:You found that: Was fixed - this wasn't by Thruen · 2013-02-24 02:19 · Score: 1

So are you saying your argument in favor of the name & shame strategy is pointing out times where companies were named and shamed and still didn't fix it? I see a flaw in your argument... Which is sad because it's a valid point that you're trying to make, sometimes the name & shame strategy does work. But that's not really what this search engine does anyway, it's not as if they're posting on their front page that a site has vulnerabilities, you still need to go out of your way to check a specific site to find the vulnerabilities, which means it's not likely that the general public will hear about the problem, hard to call it the name & shame strategy when they're not doing much to make it publicly known. Beyond that, you point out that the first thing to do is to alert the developers and give them some time to fix it, and while I have looked, I haven't found anything that suggests this site does either of those. You have a very valid point in that in many cases just alerting the developer gets nothing done, but it holds little meaning in regards to this search engine as it doesn't really do any of those things you think should be done.
Re:Couldn't find any - the results so far ARE pret by Zapotek · 2013-02-24 03:46 · Score: 1
There are a few assumptions being made here that should be addressed for people unfamiliar with the field:
- It would be impossible for results of that magnitude to be manually verified in order to weed out false-positives, which are a real problem.
- Just because that scanner hasn't found any vulns it doesn't mean there aren't any.
- As others have pointed out, this is highly unethical. Scanning a site can be disruptive (and even devastating under some circumstances) which is why every such vendor discourages use of their software against live/production websites.
I imagine you saw HD Moore's nmap scan of the internet and thought to yourself "Wow, we got to get us some of that!" but this is a really bad idea and I imagine you already know that. The only way to have gone forward with this is after weighting the bad (ethical issues, fallout from site owners, possible legal troubles, etc.) and the good (getting attention) and here we are.
Re:Couldn't find any - the results so far ARE pret by Anonymous Coward · 2013-02-24 07:29 · Score: 1

Please publish your scanning IP so it can be blocked by people that wish to opt out of this
Java vs .NET by sproketboy · 2013-02-24 12:19 · Score: 1

Java 96, .NET 20247. LOL.
Perhaps a Suggestion by utkonos · 2013-02-24 17:25 · Score: 1

I was at your talk at ShmooCon and was quite impressed. What if for any domains that you discovered vulnerabilities on you were to automatically pull whois data (if the TLD has whois servers or web based whois without a captcha) and send a quick email about your findings to any emails listed? A shameless plug: ruby whois is the best programmatic whois client and parser out there IMHO. It would make the above suggestion quite simple.