Slashdot Mirror

← Back to Stories (view on slashdot.org)

A Look Into the FBI's "Everything Bucket"

Posted by kdawson on from the what-they-know-and-when-they-know-it dept.

Death Metal notes an EFF report on information wrested from the FBI over the last three years via Freedom of Information requests. The report characterizes what Ars Technica calls the FBI's "Everything Bucket" — its Investigative Data Warehouse. (Here's the EFF's introduction and the report itself.) The warehouse, at least 7 years in the making, "...appears to be something like a combination of Google and a university's slightly out-of-date custom card catalog with a front-end written for Windows 2000 that uses cartoon icons that some work-study student made in Microsoft Paint. I guess I'm supposed to fear the IDW as an invasion of privacy, and indeed I do, but given the report's description of it and my experiences with the internal-facing software products of large, sprawling, unaccountable bureaucracies, I mostly just fear for our collective safety."

21 of 31 comments (clear)

  1. EFF's Use of the FOIA to Acquire Details by eldavojohn · · Score: 5, Informative

    Well, I'm glad someone out there with lawyers is taking advantage of the Freedom of Information Act (FOIA). To see how the EFF has taken advantage of it, their main FOIA with the subpage on this entire DOJ Investigative Data Warehouse topic and all the documents they've collected (some are linked in main story).

    If you are a US citizen, you yourself are able to make a FOIA request.

    --
    My work here is dung.
    1. Re:EFF's Use of the FOIA to Acquire Details by Smidge207 · · Score: 5, Informative

      *sigh* My dear frined eldavojohn is suffering from a case of naivety, I fear.

      In fact most of my FOIA requests have been with the FBI. To date, I've filed 57 requests with them. Of these, 8 have resulted in documents, 18 were "no records" (including cases where records had been destroyed), and the remainder are still open. The most important thing to know about the FBI is that their records system is decentralized. By this I mean that every FBI field office keeps its own records, and there is no centralized database for searching all records at once. Yes, FBI Headquarters (HQ) in Washington, D.C. keep lots of files, but just because FBI HQ doesn't have any records on a topic doesn't mean that the field offices don't. For example, I requested information on an organization called the Youth International Party Line, or YIPL, which was based in New York City during the 1970s. FBI HQ had no records on YIPL, but the New York field office did.

      What this means for you is that you need to send FOIA requests to both FBI HQ and to any FBI field office that you think might have records responsive to your request. This is a pain, but it's not the end of the world. There are about 56 FBI field offices and you can use the same FOIA request letter for each one, so we're talking about at most $22 in postage to spam all of their field offices with your request. This link provides a list of all the FBI field offices, and this other link provides the same list in Microsoft Word mailing label format. The second important thing to know about the FBI is that their records system is broken up into "automated" (i.e., computerized) and "manual" indices. The manual indices are for criminal investigations prior to 1973 and security investigations prior to 1958. By default, the FBI generally searches only the automated indices, since it's much easier for them to do so. If your request covers times prior to 1973, you should add a big bold note specifically asking them to search both the automated and manual indices. Wise up or shut up, eldavojohn, my dear twisted sister.

      =Smidge=

      --
      Is it just my observation, or is eldavojohn an idiot?
    2. Re:EFF's Use of the FOIA to Acquire Details by eldavojohn · · Score: 4, Interesting
      Well, that's unfortunate but with the massive failure of Virtual Case File (and at extreme taxpayer expense), you can understand why you have to mail to every field office.

      In fact most of my FOIA requests have been with the FBI. To date, I've filed 57 requests with them. Of these, 8 have resulted in documents, 18 were "no records" ...

      I would consider your story a success story. It seems you don't but you recieved what documents they could provide to you.

      Here's my own anecdotal worthless history of FOIA. I was a junior in high school and was dissatisfied with lunch prices of Aramark (the same people who rape you at arenas and stadia) in our cafeteria. Every month they would systematically increase prices on all products by five or ten cents and it got to be ridiculous not long after. Then they "locked down" the campus so we couldn't leave for lunch. Which really really pissed me off. Yes I could have brought my own lunch but I didn't really like doing that.

      So I asked my friend to ask his dad (lawyer) for a template FOIA and filled it out with three other kids. We signed our names requesting the public high school release all details on their contract with Aramark. Instead, they brought us into an office room and gave us everything. I think that was an attempt to dissuade us but instead we were there late into the night. We had records on everything. What ever teacher was paid, what every contract had been made with an external business, everything. So we looked into the lunch provider history. The school had made some sort of several year contract with Aramark (not uncommon I guess) but that made them the only purchasable food.

      The rest of the story is pretty offtopic. But I found that to be a highly successful and satisfying use of the FOIA on the local level. I'm sorry Federal cases don't sound as profitable and I don't mean to sound naive but it is the Federal Government. You have to expect bullshit bureaucracy there--I'm sure field offices requesting documents from other field office experience the same problems.

      --
      My work here is dung.
    3. Re:EFF's Use of the FOIA to Acquire Details by DragonWriter · · Score: 1

      The rest of the story is pretty offtopic. But I found that to be a highly successful and satisfying use of the FOIA on the local level. I'm sorry Federal cases don't sound as profitable and I don't mean to sound naive but it is the Federal Government.

      You do know that the FOIA only applies to the Federal Government to start with, right? Many states have sunshine laws, which may apply to local governments within the state, and local governments may have their own sunshine laws as well, but those are not the FOIA. Whether your friend's dad actually gave you a template for a request under the actual records law applicable to the school, whether the school treated the request as one under the applicable records law, or whether the school felt it was easier to dump a bunch of paper on you rather than even bother to evaluate your request, the one thing that can be stated with a fair degree of certainty is that the actual federal Freedom of Information Act had little substantive to do with your experience.

  2. More like an heterogenous soup of data... by Smidge207 · · Score: 1

    I got the impression (remember, this is a kdawson green light *sigh*) that it was more of a super gigantic database rather than a data mining system. That is, any information derived from the data is done by humans. Data mining is incredibly difficult even WITH perfectly clean data, much less the gigantic mess that seems to be the everything bucket. Meh, color me unimpressed.

    =Smidge=

    --
    Is it just my observation, or is eldavojohn an idiot?
    1. Re:More like an heterogenous soup of data... by HTH+NE1 · · Score: 1

      Soooo, it's The Intersect?

      --
      Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
  3. The Truth is Out There... by Drakkenmensch · · Score: 4, Funny

    ... way out there, and it's done in MS Paint.

  4. Summary is biased... by iamhigh · · Score: 3, Funny

    I mean first of all Windows 2000 is currently still the most rock solid, impenetrable OS you can use. There is no reason to think a more secure OS has been released since that time.

    And everyone knows that hardcore web designers use MS Paint.

    --
    No comprende? Let me type that a little slower for you...
  5. Maybe by yerktoader · · Score: 1

    Something cool will be born out of this "sea of information"...

  6. Oblig. xkcd reference by bunratty · · Score: 4, Funny

    Today's xkcd is apropos.

    --
    What a fool believes, he sees, no wise man has the power to reason away.
  7. Fear the giant anonymous database by netwiz · · Score: 1

    Here's a concern. Given the degree to which morons are let enter data to run-of-the-mill DBs like the one in the article, what's the level of accuracy? How many times has some nimrod phoned you up to generate harassment due to bad data? How many times have you yourself called to correct something only to be told "well, the computer says so, it must be right!"

    People will make serious policy decisions based on this bad data.

    Voluminous quantities of stupid and failure are certain to follow.

  8. Re:Summary is biased... &, you're not? by Anonymous Coward · · Score: 2, Insightful

    "I mean first of all Windows 2000 is currently still the most rock solid, impenetrable OS you can use. There is no reason to think a more secure OS has been released since that time." - by iamhigh (1252742) on Friday May 01, @10:25AM (#27786703)

    Windows 2000 (&, later Windows versions) can be made to be VERY secure, by security-hardening them, & secured based on "industry best practices"!

    (Per the multiplatform CIS Tool, which also mind you, runs on various *NIX OS', such as Solaris, quite a few BSD variants (since you noted it in your 2nd url (no MacOS X though afaik)), & yes - Linux too), quite easily - heck, the CIS Tool makes it actually sort of "FUN" to do (almost like running a PC performance benchmark test).

    In fact, Windows 2000 Pro, specifically, can be made to CIS Tool score to the tune of a 99.058/100 score on this test:

    http://www.xtremepccentral.com/forums/showthread.php?s=7de5812b7341873cc5e6ee9582f21bf9&t=28430&page=3

    & the HIGHEST *NIX score I have seen, to date, came from Bert64 (a member here) ->

    http://www.xtremepccentral.com/forums/showthread.php?s=7de5812b7341873cc5e6ee9582f21bf9&t=28430

    That was done on SuSE Linux @ 90/100 on CIS Tool, AND, like Windows? It came up from its default score of 46/100 (just as Windows typically does, same range of score initially by default).

    (The makers of the test say not to compare "apples to oranges" (in other words, OS-to-OS score comparisons), but, my point IS there - they all can be further secured above the "norm" & that probably goes for things like SeLinux bearing Linux distros as well)).

    APK

    P.S.=> You *NIX guys often fail to note that market share of desktop & server markets matters in terms of how often these OS' are attacked - & that's obvious as to WHY Windows is the most attacked: Today's malware authors aren't after only making mischief & getting "bragging rights": They're about getting INFORMATION, that leads to monies from YOUR WALLET/BANK ACCOUNTS, etc. et al... &, they're going to target the LARGEST SINGLE BODY OF USERS OUT THERE, today, in order to do so... & guess what? Yes, that's right - that's Windows users! Make MacOS X or Linux (or even another BSD variant) the "top dog" out there, market-share-wise?? That'd then become "the most attacked" from a single codebase for attack, because they are out to "hit the largest mass they can with a single shot"... apk

  9. "I mostly just fear for our collective safety." by John+Hasler · · Score: 1

    Why? It sounds like it is too mucked up to be very dangerous.

    --
    Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
  10. Re:Summary is biased... &, you're not? by Gr8Apes · · Score: 1

    Disabling the server and browser services, among others, are the first things I do with a windows system. Yes, this means no file sharing from a windows machine, but how often do you do that anyways on most home single system setups?

    It's true that this will block most worms, as there will be few open ports left. It still doesn't mean that the system is secure. The fact that much code runs through DLLs with system level access allows for buffer overflows to basically run anything they want at root level, so if you hit anything external like a web page, you're vulnerable. Even third party software like Firefox that used to use the image processing included with the OS was vulnerable to this attack. (I believe they now ship with their own GIF/JPG renderer, but don't recall, as I no longer run windows anywhere so don't really care)

    --
    The cesspool just got a check and balance.
  11. Re:Hey Faggots by pwfffff · · Score: 1

    0/10: it didn't make sense, isn't going to anger anyone, and you didn't even bother to take out the 'pic related' crap. At least make your trolling site-specific.

  12. Re:Hey Faggots by earlymon · · Score: 1

    Dear Gary McKinnon,

    Many of today's operating systems afford the user with the ability to invoke a screensaver when you're not using your computer. In addition, many of those allow for an optional password entry, dissuading passersby from using your account to post drivel in your name.

    If you're a beginner or student in a public computer lab, and therefore don't have access to this feature, talk to your instructor - you'll find them sympathetic and helpful.

    Finally, if this wasn't how the above post was made, let me assure you that you don't have to worry about loss of face here. In this forum, they're called Anonymous Cowards. We pay about as much attention to these posts as we do to graffiti when driving down the road- not much.

    Study hard, have a nice day,
    The EarlyMon

    --
    Pathological kinda promises Path + Logical - but instead, you get stuck with pathetic.
  13. This is your receipt for your husband... by stinkydog · · Score: 1

    Jack Lint:
    It's not my fault that Buttle's heart condition didn't appear on Tuttle's file!

    It's Brazil! all over again...

    --
    âoeWho knew something as harmless as willful ignorance could end up having real consequences?â
    1. Re:This is your receipt for your husband... by Jaazaniah · · Score: 1

      Mod parent up! I can't wait till someone gets wise and publishes a story similar to this.

      On the other hand, I wonder if there's anything to gaming decentralized records, such as attracting attention in the northeast states, slipping off through the border, circumvent the globe, and arrive in LA to do the same shit all over again with reduced risk.

      Of course, I've probably landed on at least one of their watch-lists for simply mentioning it, but hey, what does that say about their organization if some lay-person can see vulnerabilities in their practices?

  14. Put it over there by SEWilco · · Score: 1

    Is this a bucket or a "Raiders of the Lost Ark" warehouse?

  15. Yet another case of... by russotto · · Score: 1

    "Tyranny, tempered by incompetence"

  16. Feds and Tech by no1home · · Score: 1

    I guess I'm supposed to fear the IDW as an invasion of privacy, and indeed I do, but given the report's description of it and my experiences with the internal-facing software products of large, sprawling, unaccountable bureaucracies, I mostly just fear for our collective safety.

    After the experience I just had today with an agent from a Federal agency, I have to agree. He needed to use a computer that couldn't be traced back to his agency for undercover reasons, and that agency is remodeling and rewiring their system, so no such luck. He came to my facility for access. The Feds are so very much behind the times it's a wonder they ever catch a criminal. The public makes HUGE demands (mostly rightfully so) on what is right/wrong, what is evidence, etc, but then freaks out over the cash needed to give them the tools. Result? Minimal advancement in technology. To make matters worse (or better if you want to hide from the Feds), bureaucracies are amazingly slow to change, so even if the money was available to get better technology for crime-fighting, the big bosses and their underlings would mostly resist having to work with it.

    Then again, even without all of this, I fear for our collective safety. The more I work with the public, the more I like my cat!

    --
    I hope this comment is well received... I could have moderated instead!

    Persecutors will be violated!