Web Log 'Word Bursts' Could Identify New Crazes

Zorgatron writes "New Scientist reports that a researcher from Cornell University has come up with clever method of identifying what's cool by automatically searching weblogs. Sudden increases or "bursts" in the usage of particular words may reflect a new craze, according to Jon Kleinberg. He has demonstrated the technique by searching through state of the union addresses given since 1790." I wonder how long before this can be done real time enough to really make this useful.

Comments

Not Real Useful

[foo fighter]

This isn't going to show what's popular in our Culture. It's going to show what's popular in the web logging sub-culture.

I doubt there's that much of an overlap that the latter is an indicator for the former.

Google's Zeitgeist is a much better tool since it's analyzing what the general web populace is looking for. I think that would be a more accurate indicator of current web culture trends.

Still, neither are good indicators for our culture as a whole since they only reflect the interests of those who have ready access to a computer. The Zeitgeist is getting better all the time as more and more come online.

Well duh... research uncovers the obvious.

[dpbsmith]

The example given is totally unimpressive: analyzing State of the Union messages, "In the years that immediately followed the American Revolution, for example, sudden bursts in the use of words such as "militia", "British" and "savages" are found. From 1930 to 1937 a spike in the use of the word "depression" is seen. And from 1949 to 1959 "atomic" is the word with the greatest "burstiness". Later in the 20th century, words such as "Vietnam", "Soviet", "communist" and "Afghanistan" increase sharply in usage." What a surprise. No tenth-grade history student would have ever guessed.

Equally appalling is the suggested "use:"

"For example, identifying word bursts in the hundreds of thousands of personal diaries now on the web could help advertisers quickly spot an emerging craze."

See Shurtape's web site for a letter explaining to distributors that they have shifted all production to three popular consumer products that lead times on every other kind of duct tape is increasing to eight weeks... If only Shurtape had analyzed the burstiness of the words "duct tape" in blogs would they have been able to anticipate the spike in demand?