Measuring Real Time Public Opinion With Twitter

Hugh Pickens writes "The NY Times reports that statisticians from the University of Vermont are hoping to harness the stream of messages flowing through Twitter to read public opinion and sentiment in real time. '"Twitter is a reflection of what people are interested in right now," says Peter Dodds, adding that the goal is to establish an index, akin to the Dow Jones industrial average, that can "give an overall sense of how a collective body of people are feeling at any given point in time.' Dodds says he and his colleagues are analyzing about 1,000 tweets each minute, or about a million a day, looking for trends in descriptive words and phrases that indicate moods and emotions. In addition, the two can monitor the public reaction to news or policy announcement and track it over time. The tool is still in its early stages, but eventually Dodds hopes that it could work similarly to Google Flu Trends, a Web tool that doubles as an early-warning system for flu outbreaks by detecting spikes in certain search terms. Since relationships and conversations are so intrinsic to how people communicate on Twitter, the researchers hope that observing how one user's mood is affected by another might shed some light on crowd behavior and emotional contagion. 'All of this data serves as a remote sensor of well-being,' Dodds says."

Comments

Won't work for long

[jonbryce]

Wait until the astroturfers and lobyists discover this.

Re:Won't work for long

[Frosty Piss]

Wait until the astroturfers and lobyists discover this

They already have. But it's irrelevent, Twitter has or is near peaking. As soon as "the next big thing" hits the scene, Twitter will fade, like ICQ and all the rest of the chat "communities".

I just wrote on twitter

[ls671]

I just wrote on twitter about my opinion about harnessing the stream of messages flowing through twitter to read public opinion and sentiment in real time.

Public Twitter opinion only 98% inane

[David Gerard]

Only 98% of Twitter updates are "pointless babble," says a new report that studied 2,000 tweets over a period of two weeks.

The top category was "pointless babble" tweets, with nearly 98% of tweets being inanity no sane person could want to read, retweets of inanity, links to inanity, retweets of links to inanity and retweets of retweets of links to links to the reretweet itself. And camera phone pictures of bowel movements on Twitpic.

Almost 2% was Stephen Fry, Neil Gaiman or retweets thereof and the rest was Warren Ellis posting scatological abuse of his fans.

Botnet command messages were becoming more popular, many disguised as combinations of the syllables "lol" "wtf" "d00d" "RT" and "#fb" or scatological abuse of Warren Ellis's fans.

Twitter's demographics as of June 2009 were 55% female, 43% ages 18 to 34, 78% white, and 99.5% of such short attention spans that Facebook might as well be War and Peace. Botnet readership was considered likely to rise as soon nothing with organic intelligence would be able to cope.

Twitter recently redesigned its homepage, changing the tag "What are you doing now?" to "Post tomorrow's CNN headlines, particularly about #goatse."

sample selection bias

[drDugan]

"Twitter is a reflection of what people are interested in right now"
=>
"Twitter is a reflection of what the twits are twatting in right now"

Can you see the problem?

"public" opinion?

[AliasMarlowe]

Or the collected opinions of twits, er twitterers, twats, or whatever. It's a self-selected group, whose collective opinions are no more representative of the general public (or voters or any other subset of the general public) than, say, the opinions of slashdotters. And although there is much drivel on slashdot, I suspect it is nothing compared to the twaddle on twitter.

Re:"public" opinion?

[freedomlinux]

Is anyone else thinking "selection bias"?

How can Twitter users be a representative sample of the public as a whole? And I don't even want to think about issues with geographical context...

Re:"public" opinion?

[negRo_slim]

I suspect it is nothing compared to the twaddle on twitter.

Depends on who you follow try SciAm's current recommendations: science writer carlzimmer, evolutionary biologist Jonathan Eisen phylogenomics, theoretical physicist seanmcarroll, science writer RebeccaSkloot, NASA astronaut Mike Massimino Astro_Mike, or astronomer Phil Plait BadAstronomer

Re:"public" opinion?

[AliasMarlowe]

I suspect it is nothing compared to the twaddle on twitter.

Depends on who you follow try SciAm's current recommendations: science writer carlzimmer, evolutionary biologist Jonathan Eisen phylogenomics, theoretical physicist seanmcarroll, science writer RebeccaSkloot, NASA astronaut Mike Massimino Astro_Mike, or astronomer Phil Plait BadAstronomer

Actually, I visit Phil Plait's sites regularly, and really like his material (http://www.badastronomy.com/index.html and http://blogs.discovermagazine.com/badastronomy).
His tweets are a different matter entirely - twaddle, and inescapably so due to the limitations of tweeting (what a pathetic term). You can find his twaddle at http://twitter.com/BadAstronomer. I encourage you to contemplate how poor a communication medium twitter really is, when it reduces someone with the aptitude and knowledge of Phil Plait to producing twaddle.
Now contemplate how intensely boring/stupid/inane the twaddle of a less gifted twitterer must be. Collectively, it's electronic flatulence - an outpouring of brainfarts.

Rigged sample

[gmuslera]

Assuming that "normal" people behaves and think about subjects like the average high activity twitter user in all cases is a somewhat a risky choice, throwing dices could be more exact.

Re:Won't work when twitter goes down

[ls671]

> I cannot post at this moment.

We are deeply sorry about this, we have just launched a test run of our experimental software and we did not think it could affect Twitter's performance. We have now reduced the threshold of our requests so everything should be back to normal now.

Thanks,

Department of Statistics
University of Vermont

Not representative?

[clarkkent09]

Twitter is a reflection of what people are interested in right now

Correction: Twitter is a reflection of what morons are interested in right now. Still, useful marketing information.

The Onion

[Jamamala]

Another case of life imitating Onion.

Open to Gaming

[Nom du Keyboard]

This seems open to gaming the system since there are no controls on the users and Twit-bots are easily possible. Given that politicians are so d@mn poll-driven these days, this idea seems dangerous.

ABC already did this

[kramulous]

The ABC (Australian Broadcast Commission) did this for a live state -vs- state football match we have here. It was quite good. Measuring sentiment of who would win based on the positiveness or negativeness of 'tweets' (or whatever the fuck they're called).

Here