Slashdot Mirror
Data Mining Rescues Investigative Journalism
John Mecklin sends in word of initiatives through which the digital revolution that has been undermining in-depth reportage may be ready to give something back, through a new academic and professional discipline known as "computational journalism." "James Hamilton, director of the DeWitt Wallace Center for Media and Democracy at Duke University, is in the process of filling an endowed chair with a professor who will develop sophisticated computing tools that enhance the capabilities — and, perhaps more important in this economic climate, the efficiency — of journalists and other citizens who are trying to hold public officials and institutions accountable. The goal: Computer algorithms that can sort through the huge amounts of databased information available on the Internet, providing public-interest reporters with sets of potential story leads they otherwise might never have found. Or, in short, data mining in the public interest."
It doesn't matter how efficient journalistic gum-shoeing becomes, because the end product will still be subject to a certain amount of spin by the publisher.
But as it is, we can't get local news media to perform their "watchdog" role in most cases. I can't even begin to count the number of times when I've seen a case that looked suspicious as hell based on the reporting of it, but the local media just parroted the police/prosecutor's story and moved on. Alternatively, when they do get involved, it's often in cases like the Jena 6 where you end up finding out that the media was spreading disinformation and building up a narrative to make more profit.
Most news media have become a combination of an AP outlet and a source of editorials and classifieds. They're like a primitive RSS feed with some mashed up content thrown in there for local flair.
No, most reporters will continue to copy PR releases into articles.
SELECT *
FROM advertising_revenue_table, list_of_local_business_table
WHERE advertising_revenue_table.business_name = list_of_local_business_table.business_name
AND advertising_revenue_table.cost_of_ad_space_purchased = 100
AND list_of_local_business_table.owners NOT IN (select names from list_of_publishers_buddies)
ORDER BY cost_of_ad_space_purchased ASC
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
The journalists groom their resources and need to keep in their sources good books to keep up access. Play ball and you get indented with a patrol so you can send back gripping combat footage. Piss off the brass and you get indented with the guys washing trucks at the transport park.
It is no wonder that editors and TV execs are quick to fire and distance themselves from any journalists that forget this and start snooping too deeply. Just look at http://en.wikipedia.org/wiki/Peter_Arnett
Engineering is the art of compromise.
It's not what journalists don't know. It's what they don't report.
And basically people just don't care. Have we decided who to blame for the economy collapse yet? But bathroom foot tapping, wow, that's the shit we have to get to the bottom of it.
As someone who does investigative journalism for a living, data mining won't get you squat. Having done it for a living for 5+ years, and being very familiar with data mining, the two so rarely cross paths that it rounds to zero.
Why? Because if it is in minable form, it doesn't take any digging to find. If you can run a google search and get even a tidbit about what you need, you don't need investigative journalism.
Of the stories I have gotten, little ones like the P4 going 64 bits, it never reaching 4GHz, Dell exploding laptops (an assist on that one), and more recently the Nvidia bump cracking problem(s), none of that would have been possible through data mining.
If it is out there, it doesn't need an investigative journalist. If it isn't, than data mining won't help. The end.
-Charlie