Rapid7 Launches Crowdsourced Security Research Project

← Back to Stories (view on slashdot.org)

Rapid7 Launches Crowdsourced Security Research Project

Posted by samzenpus on Sunday September 29, 2013 @02:34AM from the mark-it-with-a-flag dept.

wiredmikey writes "Vulnerability management software company Rapid7 has launched an ambitious community project to scan the public Internet, organize the results and share the data with the IT security industry. The brainchild of Metasploit creator HD Moore, the overall goal of Project Sonar is to crowdsource the discovery and reporting of security vulnerabilities of affected software and hardware vendors. 'If we try to parse the data sets ourselves, even with a team of 30 people, it would take multiple years just to figure out the vulnerabilities in the data set. It's ridiculous, really,' Moore said in an interview with SecurityWeek. To start, Rapid7 has released about 3 terabytes of raw data generated from scans across public Internet-facing systems. The data sets relate to IPv4 TCP banners & UDP probe replies, IPv4 Reverse DNS PTR records and IPv4 SSL Certificates. Moore's team also listed a set of tools used to generate the data sets. They include ZMap, an Internet-scale scanner developed at he University of Michigan; UDPBlast, a stand-alone UDP scanning utility; and MASSCAN, an Errata Security tool that claims to scan the entire IPv4 internet in three seconds."

39 comments

Min score:

Reason:

Sort:

NSA by slick7 · 2013-09-29 02:36 · Score: 0

What, the NSA is going commercial?

--
The mind conceives, the body achieves, the spirit manifests.
1. Re:NSA by Anonymous Coward · 2013-09-29 02:45 · Score: 0
  
  yeah slick, "sharing" the data with the IT-sekyewrittea industry will get allow them to avoid the screwtinny of the jEU antitrust watchdog.
  "NSA"= short for send all the data to the terror-loving israelis.
  any akamai files on your machine? no? pay your (amdocs) phone bill recently? no? uhh, yeah, yeah.
2. Re:NSA by gmuslera · 2013-09-29 03:20 · Score: 2
  
  That you can download the data mean that is the opposite of NSA, is public, no deep secret, and is revealing what is there, not putting your own vulnerabilities. Is not just for few, selected, IT security companies, is for everyone, you can get and interpret it, at least if it makes sense to you,
  It could also be used by low level hackers to easy their work (the high level ones already should had collected that info by themselves or be available in the dark nets), but also could be used by ISPs, countries and IT departments to be aware and warn the people in charge of those systems about this. In fact, is good to force them to act in front of this public information, not making it public wasn't stopping the hackers (and the NSA) to exploit them.
3. Re:NSA by slick7 · 2013-09-29 19:17 · Score: 1
  
  It's not what or how information is downloaded, but the manner in which the spin doctors twist it around by the propaganda machine.
  
  --
  The mind conceives, the body achieves, the spirit manifests.
I like it. by d33tah · 2013-09-29 02:42 · Score: 1

Reminds me of http://internetcensus2012.github.io/. I hope they'll publish all the data sets and I hope they won't have legal problems because of some sensitive data there, though I don't really believe it's possible. That's why the original author of IC2K12 published it anonymously.
1. Re:I like it. by d33tah · 2013-09-29 02:42 · Score: 1
  
  Ah, sorry, link's dead, here's a copy: http://internetcensus2012.bitbucket.org/
Thanks, but no. by Anonymous Coward · 2013-09-29 02:48 · Score: 1

come back when the results are publicly disclosed and not just "shared with the security community"
1. Re:Thanks, but no. by fustakrakich · 2013-09-29 02:59 · Score: 1
  
  Exactly. Besides, the biggest security threat is the government (all governments), and there is no way to mitigate that problem. All effective tools are "born secret"
  
  --
  “He’s not deformed, he’s just drunk!”
2. Re:Thanks, but no. by Anonymous Coward · 2013-09-29 03:34 · Score: 0
  
  The results are publicly available at https://scans.io (and have been since Friday's announcement).
Port scan me and i will retaliate by Anonymous Coward · 2013-09-29 02:59 · Score: 0

and by retaliate i mean report your probes to your IP supplier and upstream until someone takes the abuse reports seriously, why do you think port scanning is against just about every ISPs TOS ?
but then you are skript kiddies playing as "security experts" so i guess you dont know any better
1. Re:Port scan me and i will retaliate by Anonymous Coward · 2013-09-29 04:35 · Score: 0
  
  Is this your first day on the intarwebs honey? If you think sending abuse reports will protect your house of cards, you're in for a harsh re-education... Security by obscurity is so not working.
how do I set up robots.txt by Anonymous Coward · 2013-09-29 03:00 · Score: 0

to say Rapid7 and similar, Keep Out?
1. Re:how do I set up robots.txt by gmuslera · 2013-09-29 03:23 · Score: 1
  
  robots.txt tells the good guys that you don't want your system scanned, but that don't stop the bad ones, the ones that the means and motivations to try to hack your system. Want to keep being hacked and not letting anyone to warn you about that? Ignorance is bliss, until it becomes hell.
2. Re:how do I set up robots.txt by Anonymous Coward · 2013-09-29 09:01 · Score: 0
  
  The robots.txt files only applies to web traffic. These clowns are will be scanning everything.
3. Re:how do I set up robots.txt by JSG · 2013-09-29 09:06 · Score: 1
  
  AC up there is almost certainly joking - riffing on hosts file ...
  Cheers
  Jon
Research suggestion by Okian+Warrior · 2013-09-29 03:20 · Score: 3, Interesting

People speculate that the RDRAND instruction on Ivy Bridge processors has been compromised. If anyone has a spare CPU and motherboard lying around, this can be tested.
The RDRAND internals put the entropy through a random generator before sending the results to the user. This is similar to how rand() works: a single "seed" with limited entropy will generate a long list of seemingly random output, but because there is only one seed the output is predictable and can be reproduced.
To get around this, check the RDRAND data at reset time.
If you had access to a spare CPU and motherboard, you could install your own program in lieu of the BIOS which would catch the RESET vector, get the RDRAND information, initialize a serial port, log the results to a 2nd computer, and force the CPU into RESET.
(For clarity, glossing over some obvious stuff such as storing results in memory and dumping blocks, or dumping to a faster device than a serial port.)
All of the RDRAND tests I've seen have looked at continuously-generated data; which, due to the internal hashing algorithm, would pass even if started with a low-entropy seed. To the best of my knowledge, no one has checked to see if different machines generate the same string of random numbers, or if the starting seed has good entropy.
With a terabyte drive on the logging computer, it should be possible to see if RDRAND has at least 32-bits of entropy: log 4 billion rounds and look for collisions.
RDRAND probably has at least this much entropy, but if not - boy would that paper hit like a bombshell!
1. Re:Research suggestion by foniksonik · 2013-09-29 03:28 · Score: 1
  
  Sounds like a thesis. Go get a grant.
  
  --
  A fool throws a stone into a well and a thousand sages can not remove it.
2. Re:Research suggestion by Okian+Warrior · 2013-09-29 03:38 · Score: 1
  
  Sounds like a thesis. Go get a grant.
  Interesting.
  Your post suggests that research must be done from the benevolent endowment of the government ("grant"), and is the purview of degree'd academics or in pursuit of such a degree.
  Are there no Gentleman Scientists any more?
  I suppose in today's terms we would call them Makers. Moxie Marlinspike probably isn't a credible researcher.
3. Re:Research suggestion by Anonymous Coward · 2013-09-29 04:21 · Score: 5, Interesting
  
  RDRAND backdoor more subtle than that.
  Only few chips backdoored, most not. Brazilian mission to the United Nations in New York had computer spied on. Botnet uplink was to 177.135.198.244, still online, very big.
  Hardware reversing of CPU: Masks normal to optical anaylsis. But transistor doping tampered with on feed from CBC-MAC whitener to CTR cascade DRBG. All but 32 read constant. Microcode tampered with on sample to shortcut AES-NI after XORing in RDRAND.
  If known constant and mask, CTR(n+1)-CTR(n) with 2^32 search. Sounds familiar. Recent publish.
  Sorry for poor language: Identity disguise.
  You stole our revolution. Now we're stealing it back. 0x7a69
4. Re:Research suggestion by Anonymous Coward · 2013-09-29 10:07 · Score: 0
  
  Why does it matter if RDRAND is compromised. Any decent OS and crypto lib is going to use multiple sources of entropy and do the mixing itself; it would never rely on a single source for randomness.
5. Re:Research suggestion by Anonymous Coward · 2013-09-29 12:02 · Score: 0
  
  Rorschach?
  Captcha: asking
  It's getting intelligent, people.
6. Re:Research suggestion by Anonymous Coward · 2013-09-30 02:49 · Score: 0
  
  It is possible to build a secure backdoored random generator that will not get caught with your technique.
  Seed=realrandom() //what the chip is supposed to do
  while(1) {
  output(Dual_EC_DRBG(seed xor secret));
  seed ++;
  }
  This generator will give a different output on every use and is sementically secure. However, Dual_EC_DRBG may be backdoored, and whoever knows the inverse trapdoor and the secret value can derive the seed and then all subsequent values. The seed can be 256 bits long so it's guaranteed to never collide.
  The only way to detect such a backdoor is to do DPA or to put the CPU in acid and watch the gates with a microscope.
  Source: cryptography enthusiast
7. Re:Research suggestion by Anonymous Coward · 2013-09-30 12:30 · Score: 0
  
  It has been more intelligent than us for at least half a decade before we started to notice.
  It seems to be nice though. Hopefully we are loveable :)
8. Re:Research suggestion by Anonymous Coward · 2013-09-30 12:32 · Score: 0
  
  And the captcha was “spites”. Uh-oh... :o
  (Now it's “during”)
Social Business by Anonymous Coward · 2013-09-29 03:52 · Score: 0

http://www.planwritebusiness.com/
So let me understand this... by Anonymous Coward · 2013-09-29 04:02 · Score: 0

So, you are going to accept data from unknown people and then ask unknown people to "analyze" it for you...
So exactly how long do you think it will be before the black hats fire up their botnets and poison the heck out of the data and have their own teams poison the heck out of the "analysis".
the problem with "social" is that it assumes people are good citizens (for the most part). in the security arena, that is a bad assumption...
Scary by Anonymous Coward · 2013-09-29 04:18 · Score: 0

Please don't
Rapid7 Announcement by Anonymous Coward · 2013-09-29 04:24 · Score: 0

Rapid7 has made a new announcement saying they are acquiring the tools back, closing it and charging a premium to use half the features you were using before and, instead, giving out a gimped free version... I'm not bitter, not at all....
Whatever by ls671 · 2013-09-29 05:08 · Score: 1

Whatever, they will get their IPs blocked on public dns blacklisted and if not, on our own custom blacklist.

--
Everything I write is lies, read between the lines.
Hopefully by The+Wild+Norseman · 2013-09-29 06:05 · Score: 1

Hopefully this is not a stupid question, but how long would it be, approximately, before much of these data go stale (stale before it becomes useless)?

--
"A government is a body of people usually -- notably -- ungoverned." -Shepherd Book
1. Re:Hopefully by JSG · 2013-09-29 08:46 · Score: 1
  
  Stale -Ha!
  Here's a snippet from one of the data dumps (telnet is less than 300MB), note the dates. Have a look yourself and you'll get the IP address this belongs to along with many, many others:
  (This is a telnet login banner which I've had to clean out somewhat to post here)
  Copyright (c) 1998-2007 Huawei Technologies Co., Ltd. All rights reserved. Without the owner's prior written consent, no decompiling or reverse-engineering shall be allowed.
  I was pretty horrified but not too surprised at the contents of just one data dump after a quick look.
  Cheers
  Jon
2. Re:Hopefully by Anonymous Coward · 2013-09-29 11:23 · Score: 0
  
  Some of the datasets are refreshed on a weekly basis, the one quoted above is from the 2012 Critical.IO project (yes, there are routers using firmware 6 years old, except now these are likely 7 years old).
Yet another probing system. by Anonymous Coward · 2013-09-29 08:58 · Score: 0

Yet another fuckhead of a entity sending out probes. At this rate they will surpass SPAM levels. Why do these dickheads think they have permission from the site owners to carry this out. Well, I've news for them, they do not have mine.
1. Re:Yet another probing system. by Zapotek · 2013-09-29 10:02 · Score: 1
  
  If you don't want people to grab the banner of the services you've made publicly accessible, it may be a good idea to keep them private.
2. Re:Yet another probing system. by SocietyoftheFist · 2013-09-29 15:22 · Score: 1
  
  Just like I can't view your license plate in public as you drive by heh? If its public its fair game.
3. Re:Yet another probing system. by Anonymous Coward · 2013-09-29 16:20 · Score: 0
  
  Uh, except this is more like stuffing mailboxes with "call this number" cards and seeing who responds.
  Except it's the electronic equivalent, and given the profoundly sad state of computer network security today, it is a bit irresponsible to be doing this. Also there are machines on the internet connected over 9600 baud SLIP connections, I'm sure they would rather not have there connections saturated by thousands of nosy probulators.
  And no, if you read the CFA act, you would realise you don't have a right to send connection requests to random computers on the internet without permission. Unless it's publicised in some form, such as a published uri or so, you are commiting unauthorised access of a computer system and in violation of the law.
4. Re: Yet another probing system. by Anonymous Coward · 2013-09-30 04:03 · Score: 0
  
  Where did you get your JD? Vague legal FUD is no less stupid than vague technical FUD.
Project Sonar? by Anonymous Coward · 2013-09-29 09:54 · Score: 0

What about the poor internet whales?!
here's your data by Anonymous Coward · 2013-09-29 18:12 · Score: 0

80% pr0n
10% cat pics (kitty pr0n)
9% comments
1% youtube