FBI Building App To Scrape Social Media
Trailrunner7 writes "The FBI is in the early stages of developing an application that would monitor sites such as Twitter and Facebook, as well as various news feeds, in order to find information on emerging threats and new events happening at the moment. The tool would give specialists the ability to pull the data into a dashboard that also would include classified information coming in at the same time. One of the key capabilities of the new application, for which the FBI has sent out a solicitation, would be to 'provide an automated search and scrape capability for social networking sites and open source news sites for breaking events, crisis and threats that meet the search parameters/keywords defined by FBI/SIOC.'"
You can already assume that your 'public' posts are being seen by people you wouldn't want to see it, but now you know that it is automatic. Depending on data sharing agreements these companies come up with with the FBI, they might even get access to private information.
Hopefully, the latter isn't an issue and they're just scraping public information, but even then, any hopes of not being carefully monitored are dashed. Assume that everything public (and most things private) will be read by people other than the intended recipients. Privacy? What privacy?
I recently inactivated my account, but I know they still have all of that data. I looked, but found no obvious place to request that your data be deleted. Anyone have any first hand experience with getting them to actually erase your data?
I know for a number of things, using the application Splunk has been great for monitoring various types of security and administrative logs... perhaps it can be adapted to look at social media sites in real time?
I have seen people use Splunk to try to correlate intrusion attempts with various things, be it weather, or even sports scores. It is a very useful tool.
Downside is cost... They generally license by the amount of data entered per 24 hour period, but the licenses can be negotiated.
Believe me, I understand why we should all be nervous about projects like this being carried out by the Three Letter folks.
But thinking about another aspect, this sort of thing is always outsourced to some defense contractor who in turn lakes way too long to soak the taxpayer for software that ultimately either fails or does itâ(TM)s job poorly.
Why donâ(TM)t agencies like the FBI recruit âoeSpecial Agentsâ to work in âoehacker labsâ to turn this stuff out in-house? I mean, such a job description is âoemade for TVâ, totally sexy! A bunch of hot young geeks in the latest styles sitting around with holstered Glocks, in a hacker lab with the latest toys?
Who WOULDNâ(TM)T want that job?
If you want news from today, you have to come back tomorrow.
As in, the bottom of the barrel.
They will know all of your dirty secrets because even if you don't spill, your friends will.
"Lame" - Galaxar
Do we really think that terrorists are going to be coordinating things via FB?
It's like that old joke, what was Bin Laden's last FB entry?
"BRB, someone's at the door"
*Seal Team 6 likes this
Hope they've changed how they do IT.
I thought we already knew that law enforcement agencies were watching social networking websites? They have caught people because of pictures posted online in the past:
http://www.guardian.co.uk/technology/2009/oct/14/mexico-fugitive-facebook-arrest
Palm trees and 8
How many articles are we going to have about this?
giggity
If you believe the whack jobs out there, Google is run by the NSA so you would think they would just share what they have :)
The FBI and other three letter acronyms have been using programs for years that monitor the internet.
Echelon, carnivore, et all. Why do they need a whole new system just to include social networking sites?
Don't know something? Look it up. Still don't know? Then ask.
Now trending: #FBIbastards
Al-Zaida Terror1sts develop tricobalt salted antimatter bomb. Antardic ice sheet vaporized. Millions of penguins now homeless.
should read: .... likelihood of social media sites can NOT get legally shutdown ....sorry about that
I've always said English was my second language. Had Romeo and Juliet been written in C, I might have understood it.
Who thinks Facebook is private? The whole point is to *not* be private, right? Otherwise... what is the point of Facebook?
If the FBI was going to start monitoring encrypted email, VPNs, and other things where you are *trying* to be private, I would be concerned (yes, I know-- whole 'nuther can o' worms). But Facebook? You are giving the info away as a user, that is the purpose of having a Facebook account.
Spook BackDoors In Cisco Routers
- Older news, but still relevant!!
Please save this story and repost it everywhere
Especially in Security Discussion Forum Sites
- You should use OpenBSD or a hardened Linux distro
For a router, NOT these blackboxes offered with
proprietary hardware & firmware!
http://www.forbes.com/2010/02/03/hackers-networking-equipment-technology-security-cisco.html
"Special Report
Cisco's Backdoor For Hackers
Andy Greenberg, 02.03.10, 01:45 PM EST
The methods networking companies use to let the Feds watch suspects also expose the rest of us.
ARLINGTON, Va. -- Activists have long grumbled about the privacy implications of the legal "backdoors" that networking companies like Cisco build into their equipment--functions that let law enforcement quietly track the Internet activities of criminal suspects. Now an IBM researcher has revealed a more serious problem with those backdoors: They don't have particularly strong locks, and consumers are at risk.
In a presentation at the Black Hat security conference Wednesday, IBM ( IBM - news - people ) Internet Security Systems researcher Tom Cross unveiled research on how easily the "lawful intercept" function in Cisco's ( CSCO - news - people ) IOS operating system can be exploited by cybercriminals or cyberspies to pull data out of the routers belonging to an Internet service provider (ISP) and watch innocent victims' online behavior.
But the result, Cross says, is that any credentialed employee can implement the intercept to watch users, and the ISP has no method of tracking those privacy violations. "An insider who knows the password can use it without an audit trail and send the data to anywhere on the Internet," Cross says.
Cross told Cisco about his findings in December 2008, but with the exception of the patch Cisco released following the revelation of its router bug in 2008, the security flaws he discussed haven't been fixed. In an interview following Cross' talk, Cisco spokeswoman Jennifer Greeson said that the company is "confident in its framework." "We recognize that security is complicated," she said. "We're looking at [Cross'] findings and we'll take them into account."
Cisco isn't actually the primary target of Cross' critique. He points out that all networking companies are legally required to build lawful intercepts into their equipment.
Special Report
Cisco's Backdoor For Hackers
Andy Greenberg, 02.03.10, 01:45 PM EST
The methods networking companies use to let the Feds watch suspects also expose the rest of us.
ARLINGTON, Va. -- Cisco, in fact, is the only networking company that follows the recommendations of the Internet Engineering Task Force standards body and makes its lawful intercept architecture public, exposing it to peer review and security scrutiny. The other companies keep theirs in the dark, and they likely suffer from the same security flaws or worse. "Cisco did the right thing by publishing this," says Cross. "Although I found some weaknesses, at least we know what they are and how to mitigate them."
The exploitation of lawful intercept is more than theoretical. Security and privacy guru Bruce Schneier wrote last month that the Google ( GOOG - news - people ) hackings in China were enabled by Google's procedures for sharing information with U.S. law enforcement officials. And in 2004 and 2005, a group of hackers used intercept vulnerabilities in Ericsson ( ERIC - news - people ) network switches to spy on a wide range of political targets including the cellphone of Greece's prime minister.
All of that, argues IBM's Cross, means that Internet-related companies need to be more transparent about their lawful intercept procedures or risk exposing all of their users. "There are a lot of other technology companies out there that haven't published their architecture
I thought Google is ran by the LiBeRaL mEdIa!!!
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
What data will the FBI have access to? If it's just what's marked public which anyone can see then more power to them. If Facebook is giving them wholesale access to private data that's a matter of concern.
Or is this the FBI saying everyone is a suspect in their books?
In that results returned by Google are at Google's discretion. Should Google change their policy or cease to exist, or be brought down by a foreign cyber attack, FBI probably would like to keep operating.
Any guest worker system is indistinguishable from indentured servitude.
We have have highly credible reports that Farmville is planning a sneak attack on Washington. Air Force One is fueled and ready.
Suggesting that you're not a criminal, tells the FBI you are a criminal.
It'll be a hundred million dollar application that searches Twitter for the "#bomb" hashtag.
I know right, all they hire are elite intellectuals.
Don't know something? Look it up. Still don't know? Then ask.
...make sure all threats are using social networks......
The only thing I'm surprised about is that they weren't already doing this. It's not like Facebook is new or anything.
As long as they are only scraping public posts, I don't see any problem with this. I'd even be fine with Facebook providing them with an API to make it easier to scrape public posts.
Misspelled 'scrap' again.
Have gnu, will travel.
Everyone knows the Female Body Inspectors have been monitoring social media for years.
Sure, we COULD all take an hour to read and digest and understand every word of every user agreement of every web site we experiment with.
Just like mankind COULD have chosen to avoid the Dark Ages, and be 1000 years more technologically advanced. We should have been on Alpha Centauri by now. As Prostetnic Vogon Jeltz of the Galactic Hyperspace Planning Council correctly stated, and he could have been speaking about EULAs, endless laws, or any other triumph of beauracracy:
"There’s no point acting all surprised about it. All the planning charts and demolition orders have been on display in your local planning department in Alpha Centauri for fifty of your Earth years, so you’ve had plenty of time to lodge any formal complaint and it’s far too late to start making a fuss about it now."
why does this site even exist anymore? reddit posts everything first, with less bias, and without all the self-loathing commentators screaming shill/troll/astroturf/mccarthyist label of the day.
We're here specifically to annoy you, AC. Looks like we're on top of our game again.
Faster! Faster! Faster would be better!
I keep getting Facebook notifications about people attempting to acquire more cell phones and baseball bats, as well as coordinating with other people who need help with fighting crime bosses.
I deleted all pictures, I deleted all posts, I de-friended everyone, I changed the name, contact info, everything... after that I disabled the account. Why... let's just say some things should stay private. TMI. Less is More. I think everyone should just delete their fb account like I did. Trust me.
That this app will be bloated and waste tons of resources.
The FBI keeps showing up in my "people you might know" list. Same with everyone else, I assume?
http://en.wikipedia.org/wiki/Virtual_Case_File
2001 Projected started just after 911. ...
2009 The FBI is years behind and millions over budget ... only half of the project's four-phase development had been completed
2010 The FBI is $100 million over budget on the
2011 The FBI's upgrade of its computerized case file system has hit another snag
All it looks like to me is that the FBI is finally joining the other TLAs in putting out an RFP that specifically is tailored to the makers of Palantir getting a contract.
It's gotten pretty darned crusty by this time, let me tell you!
Please do not read this sig. Thank you.
I would guess that commercial interests are already raping the internet for info now, just buy off-the-shelf software.
There was an unknown error in the submission.
Okay, so the FBI want's to drop major coin on what's really just expensive news aggregation.
.. as much as I hate Twitter, fast happening or highly localized events often are posted there tens of minutes before they make any sort of major media.
.. this is basically like surveillance cameras, but with automatic (and free) interpretation.
.. don't put private shit on the Internet. Period.
Don't immediately assume that because the government is doing it, that it's bad
There's millions of people running around with smartphones that constantly yap
And before anyone complains about the privacy "implications"
Okay, so v1 and v2 were fairly quiet, but now were about to be higher profile with the results of our database scraping... so now we publicly state we're working on it.
I can configure my low end 'smart' switch to duplicate all traffic coming in on a port to another. If I can do this, then you know that it's trivial for them to. No longer a need for the 'secret' room at the telco. Now we can just duplicate and redirect all traffic to a huge server farm. Raptor or velociraptor, or my balls... whatever you want to call it... they can filter sift and re-filter. It's being done already.
This is just a pointless opinion push article.
I just heard a really loud boom, like an explosion or a bomb going off! - my new sig
We need someone to make sure no one is engaging in un-American activities like free speech. Civil rights are the slippery slope to godless communism!
I thought Google was ticked off that they couldn't get this, thus Google+.
As for govt monitoring and especially doing it by subcontracting to defense contractors, I have a long list of govt agencies and defense contractors that have spidered my little site through the years, and I mean a long list.
They might be tweaking to better assimilate random social site comments but they've been mining the web for years.
Don't make the mistake of confusing "the world I want" with "the world that is". Facebook is a private enterprise, it is free, and it is deeply flawed.
I wish I could park my fancy convertible outside the liquor store with the engine running... and it still be there when I get back. Alas, I must suffer through the world that is. And someone is going to steal my car if I make it too easy.
Yeah, I find I can't start my day without a goatse link.
I want to know where the cookie recipe guy went, I liked that troll.
---
ECHELON is a government program to find words like bomb, jihad, plutonium, assassinate, and anarchy.
All of you who scoffed at people like me about preserving and protecting your privacy? Who pointed and laughed at the "privacy freak"? Who said "If you're not doing anything wrong, you have nothing to hide"? Who then proceeded to put their entire lives on Facebook? I'm pointing and laughing at you for being so utterly short-sighted and stupid. How do you like me now, hmm? Be sure to enjoy having government law enforcement pawing through all your personal posts, maybe deciding through profiling methods that you're a potential terrorist -- or just dropping by to arrest you and toss your house because you had harsh words to say about an elected official, or made an unfortunate joke. Also, enjoy your Police State -- because that's what we're living in now, morons, because you didn't give a fuck about it when it counted.
Better be sure your papers are in order, Comrade, or the Stassi will not be pleased.
Are YOU using the TOOL, or is the TOOL using YOU? Think about it!
You mean the government is going to know what kind of awesome life I have?
Right on!
We will know when the new scrape system has been implemented, the "Hate" button will be added right next to "Like"
-KI
#include bier;
Why has no one here mentioned Anonymous, HBGary, or Aaron Barr?
Isn't this exactly what Aaron Barr was up to? Wasn't he trying to sell his social media scraping tool to the FBI? Isn't that why Anonymous took him down?
C'mon Slashdot!
Right. And as we all recall, Facebook was initially funded by the NSA's venture capitol arm too. Same for twitter.
This signature has Super Cow Powers
if you have read the Anonymous dump of the HBGary emails, they already have tools that will scrape facebook. Then you can internetwork that with Palantir Technologies and Berico Technologies stuff to make your own "Team Themis".
NSA most probably is doing this for a long time now.
They will take out every viable way of communicating. Sure there is nothing to hide, I have nothing to hide but at the very least my taxes are going to some bloated agency and private corp. I do not want the gov to defend me, let me defend myself and make my own mind up about who is a threat to me and my house! And if you do not like it then move or do not live by me. Its America, right? Come on, face book, twiter, google... when will it stop? They are controlling all areas of free speech and communication. Its the old saying "you can do just what you want as long as its what I say it is and how I say it should be done". The sad thing is no matter how many people say they do not like it our gove, congress and special agencies keep doing it. So we vote them out? Right? Then money and power corrupts the next batch of elected officials and the cycle continues. And all we are left with are posts to other posted comments on what we think is an educated guess on how to fix things while the gears of the machine keep going and we get pissed off about it but still NOTHING HAPPENS to fix what the gov is doing. Don't protect me, use my taxes, support me by telling me what I can eat and what is or isn't good for me or regulate my life! I think we can make our own informed guesses on most of what life brings our way, I don't need the FDA, FCC, CIA, FBI, DOD, Etc.... telling me how to live and what I can or can't read on the internet or how much I should or shouldn't eat just so the cow farmers can make a buck and give votes to the person who wrote the bill (for example). That was never the role of the gov, sure to protect America, keep local and national peace but all the external over bloated agencies where not on the agenda back then. I get it, allot has changed and its not the same. Still that is no excuse.