Data Mining And The CIA

Brotha Z writes "It seems that the CIA has developed a piece of software labeled "Oasis" that can convert the audio from television and radio broadcasts in to text. This software is stated to be able to determine the sex of the speaker, if the speaker is a different person than the original speaker - and if one of the speakers is named, it will continue to place the name next to the correct speaker from that point on. More information on this multi-faceted piece of software can be found here." Hmmm. Sounds like some nice speech recognition technology ("perfect demo" alert!), but as a taxpayer, something rings badly about it. If they're going to use my money to spy on me, can't they at least open source the code so I can dictate a letter?

Comments

Re:Uhm, yes.

[rgmoore]

Actually what it sounds like the CIA is working on is trying to mine data out of public sources. There's good reason to think that you can discover a lot of what governments want to keep hidden if you can just go through enough publically available data and correlate it. For instance, you can probably get a good idea of a government's secret spending by figuring out how much money they're taking in taxes and borrowing and subtracting out expenditures- provided that you can actually track both of those things. It looks hopeless because there's so much data to go through, but with good computers it should be possible, especially if the other guys have a lot of secret spending. Or you can figure out what the inner circle of the government really thinks by looking at all of the news leaks from highly placed government officials.

This stuff scares the crap out of governments that are both required to be open but interested in hiding things from other countries. You simply can't hide everything, especially not anything big enough to be really interesting, because it has to interface with the world somehow. The CIA obviously wants to get really good at this kind of thing, and monitoring vast quantities of mundane stuff like TV news programs, budgets, and corporate annual reports is part of the process. The best part is that if you can do this effectively, you don't need spies as much, but you do need a lot of drones to go through huge piles of paper and TV to enter the raw data into the computers to process. There's probably some filtering out the interesting stuff from listening in on videoconferences, too, but it's amazing how many paper pushing drones wind up working in a sexy sounding business like spying.