Slashdot Mirror
Open Data and a Critical Citizenry
Last week we discussed news that the UK government had released a treasure trove of public spending data. Charles Arthur, the Guardian's technology editor, wrote at the time how crucial it was for citizens to find ways to examine and interpret the data; otherwise it would be useless. Now, an anonymous reader sends in a response from open data activist David Eaves, who takes it a step further. He writes,
"We need a data-literate citizenry, not just a small elite of hackers and policy wonks. And the best way to cultivate that broad-based literacy is not to release in small or measured quantities, but to flood us with data. To provide thousands of niches that will interest people in learning, playing and working with open data. ... It is worth remembering: We didn’t build libraries for an already literate citizenry. We built libraries to help citizens become literate. Today we build open data portals not because we have a data or public policy literate citizenry, we build them so that citizens may become literate in data, visualization, coding and public policy."
All true, but we didn't publish books in sanskrit or any other proprietary language! So, the information should be in open format, available for all, not just for some with a, usually very expensive, proprietary tool or toy!
We need a data-literate citizenry, not just a small elite [group] of hackers...
You aren't going to get that, because what you're talking about is citizen-to-citizen education. Hackers do that already amongst themselves, and as a result they are constantly watched by the government and often viewed as a threat to the state. The government only wants people educated to a certain level -- that level being whatever is necessary so they can become a wage slave. Anything more than that, and you're abnormal, and therefore a threat.
You'll never have a data-literate citizentry, because you'll never have a government that wants citizens to be capable of independent thought, critical thinking, and access to the facts and circumstances in realtime (or even a reasonable time) because that's a security risk. All that data being released without form or processing ability is not to help you, it's to overwhelmn you and divide you to the point where no criticism can be effectively leveraged against the government because it's simply too large, too entrenched, to make any form of protest useful.
#fuckbeta #iamslashdot #dicemustdie
Today we build open data portals not because we have a data or public policy literate citizenry, we build them so that citizens may become literate in data, visualization, coding and public policy.
Good luck with that, because the painful truth is that the average pop-media-junky, marketing-spoon-fed citizen is resistant to learning as a general course. I've known so many people - adults, mind you - who honestly believe they simply can't learn new things, especially things of any sort of technical nature, be it computer-related, math-related, or what have you. It makes me sad, but there it is.
It's impossible to ever have a data literate citizenship because data is interpreted by religion. It's interpreted by religion because people aren't naturally reasonable and don't do whats in their individual self interest.
An example, if the sexual behavior of every human being in the USA were released what would be the response by the general public? If you want to predict the response look at the response to homosexuality.
Another example, if nobody in the USA had any secret from anyone else what would be the result?
If you are a jew are you comfortable with neo-nazi's having your medical records? If you are muslim are you comfortable with jews having access to your medical history? If you are anyone are you comfortable with everyone having access to every dumb thing you've said or done in your entire life?
Sure they wont be as much blackmail going on, instead people will be getting killed for who they are and with unlimited information it's much easier to make the lists.
If we knew for sure the data being datamined on facebook were anonymous and would forever remain anonymous thats one thing but when you have peoples names and identities attached to this data it moves beyond the realm of statistical analysis and scientific research and into the realm of religious judgement and criminal persecutions.
What you end up with is witch-hunting, banning of certain behaviors, hatred of larges groups of individuals who think a certain way, and general cultural bigotry.
How can you trust with your life, liberty and honor that the data being collected about you wont someday be used to destroy your life, liberty and honor?
Obviously people refuse to learn the truth because the truth is too scary. They want to believe in the fantasy illusion created by Disney and by religious systems, to convinced themselves that God is protecting them or that the good guys always win, or that the government is protecting them, or that some savior, aliens, robots from cybertron, will come and save them.
NOBODY IS COMING TO SAVE YOU. First learn to save yourself and you wont tolerate being ignorant. Free yourself from your own ignorance before you try to save someone else.
Thats the only message that will allow people to truly WTFU(Wake the f*ck up). And even this isn't enough because even if you know the truth about reality you won't be able to change it, because the majority of sheep are comfortable.
Too bad most of the population doesn't even grasp basic tactics used to manipulate the representation of data. The most common ones being:
-cherry picking
-disproportionately small samples sizes
-no idea of what a confidence interval is
-the fallacy of appealing to tradition/religion/celebrity-endorsement/belief
-the inability to distinguish between science and psuedo/junk-science
-or that, GASP, correlation does not imply causation
This is a simple combination of common sense and critical thinking, which we were taught in fifth grade elementary school. Still, such a disturbing number of people seem to be unable to grasp such basic concepts.
How exactly will they be able to interpret this data beyond dumping parts of it in excel, extracting an average (without outlier removal or consideration, because, "why would we do that? Won't it make it less accurate?"), and making another ridiculous policy or tax as a result? I'm all for making this data available for analysis by the public, but it is the responsibility of individuals to educate themselves beyond what they did (or did not) learn in school as a kid so that policy makers cannot bully them into submission due to an inability to interpret data or understand certain concepts.
And before someones cries out "academic elitist", let me note, again, that that the above topics are taught to us as children, not in university.
Quit being so damned negative!
Many people teach themselves all kinds of extraordinary talents these day. Have you looked around youtube recently? You don't have to be a geek to be interested in learning after college age.
Examples from just one subject:
http://www.youtube.com/watch?v=lLCsKGRniGY&feature=fvst
http://www.youtube.com/watch?v=_WkEKb4Ukzs&feature=fvst
I for one, assumed all of this data would be owned and sold by companies, making it scarce. Today's business runs on information, though.
Where is this free data? Any sources?
Unfortunately, here in the US 99% of the population isn't interested in becoming really "literate". They are happy to watch NASCAR events and American Idol on TV, read People magazine and the Weekly World News and not much else.
The Internet is specifically for playing games, forwarding hoax emails and for building up a self-reinforcing attitude of superiority by reading blogs from people suffering from "Extreme Ego Syndrome" or EES. It used to be that in person when encountering someone with EES you would excuse yourself quickly - today's EES sufferers now have a forum and an audience, two things guaranteed to build a simple case of EES into the much more severe EFES (Extreme Fatous Ego Syndrome). EFES sufferers have been known to rise to high public office (like Grand Dragon or the equivalent in Mensa) and sometimes run for President.
Sadly, there is no know cure for EFES and the cure for EES is to isolate the sufferer until their ego will fit in a hatbox. The US Army used to have a solution for this - Alaska - but this is no longer a reasonable solution. Too many people in Alaska these days.
Sorry, the IPCC deleted the data.
If you lack logic and mathematics skills and have no real understanding of scientific method, to say nothing of broader and deeper general education than you are likely to get in American schools, all the data in the world won't help you. We are decades past the point when an average person with an average education and no particular motivation to constantly self-educate can understand the world around them, much less make coherent policy decisions.
We have reached the point when we must either commit to much higher educational standards or disenfranchise the under-educated. Or just stand by helplessly while professional rabble rousers manipulate an increasingly ignorant general population and turn democracy into a mockery of itself.
Proud member of the Weirdo-American community.
The more data you flood people with, the less likely it is that they will look at any of it. At best, they will pick an agency (one that agrees with their prejudices and preconceived notions) to aggregate, collate, and boil down that data into short bites that have little relation to the meaning of the original data.
At worst, they will be overwhelmed to the point where they simply ignore most of it.
Example of the latter: at my job, I receive about 150 work-related emails a day, even on my weekends. I delete all but two or three without even reading them, because the vast majority are informational emails that I don't have to read - and if I read them all, I'd have no time to actually do my job.
Could we please have the same flood of open data for China, in the hope that we might reveal the secrets of their blooming economy?
Cool post! Mind if I share it with my friends on facebook?
(And goddammit, why do I use facebook? ... sigh ...)
Big talk, basement boy.
Here's one source -
http://www.guardian.co.uk/world-government-data
And while I think of it
EUROSTAT EU statistics - http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/
Just to chip in with the perspective of someone who has been dealing in health care data for the past several years: Most of the analyses of this open data will be done by people and groups with an agenda. Which will be good because they'll be motivated to actually do the analyses, and will be bad because their analyses will be at risk of intended and unintended bias in practice and reporting. Doing good data analysis is hard, as the health care field has learned to its cost. See, eg, Gary Taubes on epidemiology, PLoS Medicine on writing and reporting practices in pharma, Ioannidis on "Why most published research findings are false." (Google the title). Results of open data analysis may be released on the Internet with minimal review or quality control, and misinformation has a prodigious ability to spread and persist. Even those who don't feel qualified to critique the analysis itself can still insist that analysts are open about their intentions and disclose potential conflicts of interest. Those able to do a technical critique should insist that we are told enough about the methods to reproduce the analysis in full.
Of course I meant quality, but it was a typo. Anyone searching for "quality" in the subject field would not find it. There is a huge amount of "dirty data" out there. In addition 2 organization, even in the same building; part of the same department or work group; Ofen record things differently or have conflicting information.
How do we find the inforation and insure it is correct?Quality must
be addressed before releasing data.
putting the 'B' in LGBTQ+
Take Facebook.. due to the flood of usage and forcing people to use what is a *web app* as opposed to just a web site/blog, I think it's made a lot of people becomes more web-savvy and aware of things like "settings".
In that way, I think FB has, however inadvertently, done the general public a great service. It's also, inadvertently, made people think more about "technology issues" like privacy and security, and what "data" actually means.
with informative article titles like Bing Gets A Foursquare Badge For The World Cup With Thrillist Tips
I read the article and still don't know what it means.
To most people, money works kind of like this:
$1,$10,$100,$1000,$10,000,$100,000,$gazillions and gazillions
This leads to insane election results, where an incumbent government is kicked out
because the documentation was missing on $ 3 million (i.e. $ 3 gazillion) in government spending, on some
insignificant program, when the government was managing a $150 billion annual budget.
The unaccounted for amount was 1 / 50,000 (=0.002%) of the budget being managed.
(This actually happened in Canada).
Or, when asked what a government should do to cut costs, many voters say:
"The bastards should cut their MP (representative) salaries in half!"
(Thereby probably saving something like a whopping 1 / 10,000 (0.01%) of the annual budget.)
Anything we can do to make people start being able to at least figure in
rough orders of magnitude, and to understand what "in the noise" means,
has my vote!
Where are we going and why are we in a handbasket?