FBI Building App To Scrape Social Media

← Back to Stories (view on slashdot.org)

FBI Building App To Scrape Social Media

Posted by Soulskill on Friday January 27, 2012 @08:08AM from the they-know-you-read-everything-you-can-about-bieber dept.

Trailrunner7 writes "The FBI is in the early stages of developing an application that would monitor sites such as Twitter and Facebook, as well as various news feeds, in order to find information on emerging threats and new events happening at the moment. The tool would give specialists the ability to pull the data into a dashboard that also would include classified information coming in at the same time. One of the key capabilities of the new application, for which the FBI has sent out a solicitation, would be to 'provide an automated search and scrape capability for social networking sites and open source news sites for breaking events, crisis and threats that meet the search parameters/keywords defined by FBI/SIOC.'"

28 of 133 comments (clear)

Min score:

Reason:

Sort:

So. It begins. by willaien · 2012-01-27 08:11 · Score: 5, Insightful

You can already assume that your 'public' posts are being seen by people you wouldn't want to see it, but now you know that it is automatic. Depending on data sharing agreements these companies come up with with the FBI, they might even get access to private information.
Hopefully, the latter isn't an issue and they're just scraping public information, but even then, any hopes of not being carefully monitored are dashed. Assume that everything public (and most things private) will be read by people other than the intended recipients. Privacy? What privacy?
1. Re:So. It begins. by kiwimate · 2012-01-27 08:24 · Score: 4, Insightful
  
  So what?
  Yes, I sound cavalier, but I see so many people on /. blithely affirming that people should just know that what they put on the internet stays there forever, and should just know that their SSID is being broadcast and it's a good thing that it can be tracked and stored, and should be fine with people capturing anything whatsoever that's done outside the house, or in the house with the curtains open...
  So I can't see that anyone on Slashdot has anything to complain about here. Or is it different because it's not Google doing it?
2. Re:So. It begins. by cpu6502 · 2012-01-27 08:42 · Score: 2, Interesting
  
  Google doesn't have the power to arrest me and send me to Gitmo without trial (NDAA). That's why I hate corporations but don't fear them. I only fear the government.
  As for permanence, I can find my posts going all the way back to 1988. Who knew in 1988 that posts would be archived forever??? Anyway I don't use my real name anymore, and try to rotate my fake name every 1-2 years.
  For the FBI:
  terrorist
  Ron Paul
  militia
  Constitution
  gun
  Liberty
  airplane
  MIAC REPORT
  Yeah that sound catch their attention. ;-)
  
  --
  My AC stalker: " I personally agree with your posts most of the time, but that won't keep me from modding you troll"
3. Re:So. It begins. by Jeng · 2012-01-27 08:54 · Score: 3, Informative
  
  List of supposed trigger words as of @ 2005
  http://www.rense.com/general66/scgh.htm
  
  --
  Don't know something? Look it up. Still don't know? Then ask.
4. Re:So. It begins. by VortexCortex · 2012-01-27 14:59 · Score: 3, Funny
  
  List of supposed trigger words as of @ 2005
  http://www.rense.com/general66/scgh.htm
  Well then, let me just target myself by creating a story from choice words on that list... bolded for your convenience.
  --------
  "MITM won't work, they have firewalls", the portly nerd stated flatly, "Only way in is SSH over SSL, but brute forcing the RSA key may take eons."
  "Let me try," said Skully as she sat at the terminal displaying the remote supercomputer's login prompt.
  Despite entering many spook words, the correct phrase remained an enigma; They had all been incorrect passwords and the secure shell remained so.
  The Infowar program's conspiracy theories blabbed over the radio. Posters on the wall proved the basement dwelling nerd who summoned her clearly had a fetish for redheads and Skully became uncomfortably aware of him now staring at her rack.
  "Do we have to listen to this crap, Bubba?" said Skully in an effort to divert the nerd's attention from her.
  The radio clicked off, and Bubba adjusted an old TV's UHF dial, past Bugs Bunny and paused momentarily on a news broadcast:
  
  Police say the hacker group Anonymous ran into a speedbump of sorts after targeting a Unix Security Firm--
  "Lame," remarked Bubba as he resumed his search, "Pseudonyms should be cool or at least ironic... like Verisign ."
  Bubba chortled gelatinously as he arrived at his favorite show and said, "Firefly is great, ever seen it? Used to be on another Chan..."
  Skully wasn't listening, she was intently staring at the debugging output on one of the various screens. Surrounded by miscellaneous glyphs and hexadecimal numbers was a single coherent word, "sardine".
  Skully attempted the password, and immediately gained access. "That executive's fish breath wasn't exactly top secret."
  --------
  I hope the spooks find it an entertaining read.
Can you get Facebook to delete your info? by Beelzebud · 2012-01-27 08:11 · Score: 4, Interesting

I recently inactivated my account, but I know they still have all of that data. I looked, but found no obvious place to request that your data be deleted. Anyone have any first hand experience with getting them to actually erase your data?
1. Re:Can you get Facebook to delete your info? by NIN1385 · 2012-01-27 08:16 · Score: 3, Interesting
  
  If you are referring to Facebook, I don't think there is a damn thing you can do to get them to erase or not retain that data. I never started a Facebook account which would be the best advice I could give.
  
  Wiki-leaks releases tons of information on the government and banks and they get punished for it, Mark Zuckerberg does the same thing to people and he gets praised.
  
  On another note, my spell-check that Firefox uses wants to change "Zuckerberg" to "Rubbernecker". Interesting...
  
  --
  
  If carrots got you drunk, rabbits would be fucked up. - Comedian Mitch Hedberg R.I.P. 03/30/68-2/24/05
2. Re:Can you get Facebook to delete your info? by M4n · 2012-01-27 08:21 · Score: 2
  
  You can with this www.zdnet.com/photos/how-to-delete-every-facebook-wall-post-wipe-your-timeline/6335458
  
  --
  In space no-one can hear your vuvuzela.
3. Re:Can you get Facebook to delete your info? by mr1911 · 2012-01-27 08:22 · Score: 4, Informative
  
  It isn't your data. It is their data about you. Read the TOS you agreed to when you made the account.
  
  Scream about it all you want, but you accepted the terms.
  
  --
  This post comes with a double-your-money-back guarantee!
  Any offense taken to this post is at your sole discretion.
4. Re:Can you get Facebook to delete your info? by kiwimate · 2012-01-27 08:43 · Score: 2
  
  Wiki-leaks releases tons of information on the government and banks and they get punished for it, Mark Zuckerberg does the same thing to people and he gets praised.
  REALLY? You're comparing a group that publishes illegally obtained secret documents that discuss high level sensitive international operations with a group that has information that people willingly give to them, signing an agreement that explicitly allows this, and that information consisting most of the time of innocuous Farmville status updates.
5. Re:Can you get Facebook to delete your info? by Anonymous Coward · 2012-01-27 08:47 · Score: 2, Insightful
  
  Well, you know, not everyone lives in the States where you can put any kind of garbage on paper. In other countries the data still remains under your control as this is a very basic and fundamental *right* and if you want to be removed, companies need to comply no matter what is written in a contract.
  Now, you may start saying that this is a US company after all. Then again, since US arrogates itself the right to enforce US laws in other countries, I hope EU will have enough balls to do the same and black-out Facebook until they start complying with EU privacy laws . It's not that Facebook will be happy to loose such a huge number of users.
  As a side note, and back on topic, I'd like to point out that the main problem is not the FBI snooping around. The real problem is that, in the long term, they will start pretending to be able to model social behavior; following a very long and prolific populist tradition of treating correlation as if it was causation. Then, after a period brain-washing the sheeple, they may simply start using such models to go after the populace.
  What do you expect from a country that actively uses Polygraphs?
Made For TV? by Frosty+Piss · 2012-01-27 08:12 · Score: 2

Believe me, I understand why we should all be nervous about projects like this being carried out by the Three Letter folks.
But thinking about another aspect, this sort of thing is always outsourced to some defense contractor who in turn lakes way too long to soak the taxpayer for software that ultimately either fails or does itâ(TM)s job poorly.
Why donâ(TM)t agencies like the FBI recruit âoeSpecial Agentsâ to work in âoehacker labsâ to turn this stuff out in-house? I mean, such a job description is âoemade for TVâ, totally sexy! A bunch of hot young geeks in the latest styles sitting around with holstered Glocks, in a hacker lab with the latest toys?
Who WOULDNâ(TM)T want that job?

--
If you want news from today, you have to come back tomorrow.
1. Re:Made For TV? by HomelessInLaJolla · 2012-01-27 09:21 · Score: 2, Interesting
  
  The copy and paste buffer, even if you believe it to be plain text, contains bits per byte which are part of the overlying OS. Some people know where those bits are and manipulate them.
  For example, when editing in MS-Word, open a plain text document--a real one (such as is generated by notepad). If you allow autospell to correct something, even if you save the document again as plain text, there will be an artefact of the autospell replacing text and some of it may be marked as UTF-8 (or other standard encoding in your OS). You think it's plain text but, until you really sanitize it with notepad or a true plain text buffer, those little tags will be preserved even if MS-Word tells you that the file is being saved plain text. The copy and paste buffer and HTML form buffers are susceptible to that sort of thing.
  Networks are exploited in a similar manner. Every packet and frame transmitted has bits wrapped around at the beginning and end--those bits are electronic pulse timings which fill the hardware as the electronic signal goes from wire to chips. A router is a bank of repeaters with bitmasks used to control which packets go where. If you, for example, use a BASIC interpreter to operate on a TCP/IP stack (such as the uIP stack inside of Contiki-OS), you will be able to manipulate those bit timings (eg. peruse uip.c for "add arch timings") which you are unable to see using standard TCP/IP libraries on your modern day OS. Careful manipulation of PEEK, POKE, and READ (and using BASIC's open hole of deliberately mixing string and numeric variables) will allow you to generate packets which, when POKE'd back into the overlying OS network stack, will hit a router and have a priority to activate the repeater circuit because a priority higher than the network bitmask was used.
  Precise knowledge of the packet generating process and the nature of the hardware to be targeted will allow the attacker to practically load an entire OS (such as the tiny ones listed on the wikipedia page for Contiki) onto the target hardware and, because it's all about nothing more than pulse timings, the "rootkit" will run concomittant to the code running in that hardware; concomittant meaning that it may or may not even be able to access the processes running in the cycles on the other side of the pulse timings.
  
  --
  the NPG electrode was replaced with carbon blac
Terrorists putting their plots on FB? by HockeyPuck · 2012-01-27 08:13 · Score: 5, Funny

Do we really think that terrorists are going to be coordinating things via FB?
It's like that old joke, what was Bin Laden's last FB entry?
"BRB, someone's at the door"
*Seal Team 6 likes this
1. Re:Terrorists putting their plots on FB? by willaien · 2012-01-27 08:15 · Score: 4, Insightful
  
  Terrorist or Freedom Fighter?
  Egypt and Libya's uprisings were greatly facilitated by twitter and other social networking sites.
  Not to say that we should overthrow the government, but, what about them using it to keep tabs on, say, the Occupiers and then using threatening, but legal, actions to undermine them?
2. Re:Terrorists putting their plots on FB? by tatman · 2012-01-27 08:17 · Score: 5, Interesting
  
  I read a few articles during and after the mayhem in London the police where having trouble coordinating their efforts because the rioters were using social media to communicate their plans. If I remember correctly, someone in British government structure has proposed a law to allow them to shut down social media sites in "emergencies". So I would assume the FBI motivation is more on these lines as the likelihood of social media sites can get legally shutdown. (btw I am in no way advocating shutting down social media or validating gov attempts to monitor it--just offering my thought as to motivations behind the FBI project)
  
  --
  I've always said English was my second language. Had Romeo and Juliet been written in C, I might have understood it.
3. Re:Terrorists putting their plots on FB? by Anonymous Coward · 2012-01-27 09:18 · Score: 2, Insightful
  
  This isn't about terrorists. This is about Occupy Wall Street and the FBI being able to feed real-time intel to local police departments.
4. Re:Terrorists putting their plots on FB? by tlhIngan · 2012-01-27 10:21 · Score: 2
  
  Maybe the FBI doesn't want to shut it down, though- by passively monitoring, they can determine where the next gathering point will be. If they were to shut it down, another method of communication (not as readily intercepted) would be used rather quickly.
  Exactly.
  Reframe it this way - did people stop pirating music once the RIAA sued Napster? No, they moved onwards to Gnutella, Bittorrent and other P2P mechanisms. In fact, killing Napster make the alternatives more popular and even harder to stop. Taking down Bittorrent trackers just made more private ones spring up.
  So only idiots shut down stuff that they can easily monitor. If people are rioting, don't shut down Facebook, instead glean intel from it. Shut down facebook and that intel would get posted elsewhere, Don't shut it down and people will post it right in front of you.
Not news? by betterunixthanunix · 2012-01-27 08:15 · Score: 2

I thought we already knew that law enforcement agencies were watching social networking websites? They have caught people because of pictures posted online in the past:

http://www.guardian.co.uk/technology/2009/oct/14/mexico-fugitive-facebook-arrest

--
Palm trees and 8
Privacy? by liquidhokie · 2012-01-27 08:24 · Score: 5, Interesting

Who thinks Facebook is private? The whole point is to *not* be private, right? Otherwise... what is the point of Facebook?
If the FBI was going to start monitoring encrypted email, VPNs, and other things where you are *trying* to be private, I would be concerned (yes, I know-- whole 'nuther can o' worms). But Facebook? You are giving the info away as a user, that is the purpose of having a Facebook account.
1. Re:Privacy? by webheaded · 2012-01-27 08:51 · Score: 3, Funny
  
  I'm not giving away all my information on Facebook to the public. I keep things contained to people I actually know for a reason. Of course even then, I barely post much of anything because there are too many people on there. The fact of the matter though is that I didn't add the FBI to my friends list so they'd damn well better not be viewing my private information. It's not for them to have. I've specifically taken the steps MULTIPLE TIMES (thanks for constantly changing that Facebook, you assholes) to keep the information private. There's nothing particularly sensitive there but that really isn't the point. I said it was private and it had damn well better stay that way. I don't give a shit if it's out on the internet...it was understood that any random person should not be able to see my information when I marked all my profile fields and posts as FRIENDS ONLY.
  
  --
  "Those who would sacrifice essential liberties for a little temporary safety deserve neither liberty nor safety." - BenF
2. Re:Privacy? by willaien · 2012-01-27 09:10 · Score: 2
  
  I'm saying: don't assume it stops at 'public' stuff. That's stuff you could already assume would be consumed by people outside of your obscure circle.
  After the whole NSA AT&T traffic sniffing stuff, I assume that anything that goes over the wire that isn't encrypted (and even some that is) can be seen by the government.
3. Re:Privacy? by tlhIngan · 2012-01-27 10:18 · Score: 2
  
  I'm not giving away all my information on Facebook to the public. I keep things contained to people I actually know for a reason. Of course even then, I barely post much of anything because there are too many people on there. The fact of the matter though is that I didn't add the FBI to my friends list so they'd damn well better not be viewing my private information. It's not for them to have. I've specifically taken the steps MULTIPLE TIMES (thanks for constantly changing that Facebook, you assholes) to keep the information private.
  
  The problem is, you posted the information. An old adage from the early days goes "never put online what you don't want the world to know".
  You fell for the "privacy" settings. There is no such thing, because honestly everyone else can get easy access to that information.
  Think about it - who benefits from people posting information on their website? Facebook/Google/etc. all benefit because it's useful for mining and marketing. How do you get the public to post their whole lives online? You implement "privacy" settings, giving them privacy theatre - because once it's online, it can't be taken off.
  Hell, I'm sure Facebook, Google, etc. would hand over lots of information if the FBI posed as advertisers and bought a few ads.
  And as long as someone else can see it, they can post it. Expecting Facebook to keep stuff private from reposts/retweets is just as good as emails that can only be read by one person.
  There is only one "privacy" setting. And it's "everything you post is public". If you mark your post as "Only Me" - why bother putting it online? If it's "Only friends" then you're counting on them to not broadcast it to THEIR friends and so on (i.e., it's public).
Re:Duplication of efforts? by betterunixthanunix · 2012-01-27 08:30 · Score: 2
1. The FBI cannot use NSA resources like Echelon.
2. Carnivore is very limited in what it can do
3. New communication systems can be exploited in different ways, and so new technologies need to be developed. The value of Facebook is not just in what people are typing, but in what their friends, friends of friends, etc. are typing as well.
--
Palm trees and 8
Wake Up Mr. President! by tgeek · 2012-01-27 08:41 · Score: 3, Funny

We have have highly credible reports that Farmville is planning a sneak attack on Washington. Air Force One is fueled and ready.
Re:and here comes slashdot, late again by ColdWetDog · 2012-01-27 08:59 · Score: 3, Insightful

why does this site even exist anymore? reddit posts everything first, with less bias, and without all the self-loathing commentators screaming shill/troll/astroturf/mccarthyist label of the day.
We're here specifically to annoy you, AC. Looks like we're on top of our game again.

--
Faster! Faster! Faster would be better!
Well That Explains It... by JeanCroix · 2012-01-27 09:12 · Score: 2

The FBI keeps showing up in my "people you might know" list. Same with everyone else, I assume?
Track record by trolman · 2012-01-27 09:14 · Score: 2

If this 'app' goes the way of the other FBI IT projects then we have no worries.
http://en.wikipedia.org/wiki/Virtual_Case_File
2001 Projected started just after 911. ... 2009 The FBI is years behind and millions over budget
2010 The FBI is $100 million over budget on the ... only half of the project's four-phase development had been completed
2011 The FBI's upgrade of its computerized case file system has hit another snag