Slashdot Mirror


Web Log 'Word Bursts' Could Identify New Crazes

Zorgatron writes "New Scientist reports that a researcher from Cornell University has come up with clever method of identifying what's cool by automatically searching weblogs. Sudden increases or "bursts" in the usage of particular words may reflect a new craze, according to Jon Kleinberg. He has demonstrated the technique by searching through state of the union addresses given since 1790." I wonder how long before this can be done real time enough to really make this useful.

239 comments

  1. Google? by irc.goatse.cx+troll · · Score: 4, Insightful

    Could this be what Google wants with Blogger?
    They have the capacity to do this, I don't see why they wouldnt.

    --
    Pain lasts, kid. Its how you know you're alive. Sometimes I think this growing up thing is just pain management-TheMaxx
    1. Re:Google? by tmark · · Score: 3, Insightful

      Except that Google already has the de-facto capability of rapidly searching as many weblogs as they care to. Sure, it takes long to spider them across the web, but it takes a long time to spider damn well near every single page in the world.

      As for how long it will be before we can do this in "real time", this all depends on what your definition of "real time" is. If you're happy with doing a few thousand blogs and getting results back in a few minutes, since at most only a few pages change on the aveage blog a day, I'd say any decent Perl guy could do that for you now.

    2. Re:Google? by t · · Score: 1
      There is a huge difference between getting updates pushed to you versus having to poll for changes, especially with a server-challenged service such as blogger was/is. Not to mention that you can get much more accurate timining information.

      You also must not read blogs very often, when the action is hot and heavy like it has been with iraq/inspections/etc.. many blogs start having conversations that jump from blog to blog to blog. If you and a couple hundred "perl guys" were to try track the action in "real time" I'm sure you would collectively cause enough bandwidth abuse to harm the very bloggers you are trying to read.

    3. Re:Google? by zeno_2 · · Score: 2, Informative

      Although its not really what the story is about, I always had thought that the Google Zeitgeist was a good indication of "new crazes".

  2. Hopefully they don't read slashdot for this by Rush+Limbaugh · · Score: 0, Troll

    or else they'll think that goatse.cx is now considered cool.

    1. Re:Hopefully they don't read slashdot for this by mshiltonj · · Score: 4, Insightful

      they'll think that goatse.cx is now considered cool.

      Which begs the observation: once poeple know the rules that determine what a "word burst" is and when it's happening, then tools will be developed to artificially inflate desired word burts

      Create a few hundred shill accounts across thousands of blogs, then each accounts on each blob will make a couple posts with the pre-determined phrase, and you have a manufactured word burst.

      Like a few years ago, when poeple sold the ability to seed search engines so your site is in the top of the results list based on certain keywords.

      Google makes that harder now, but it's always a contest between those who develop the rules (or algorithm) and those who seek to manipulate the data or the rules of the game.

      A manufactured word burst I can remember from before the 2000 election was 'gravitas'. That word came out of nowhere, and was suddenly all over the media, used to describe a quality that Dubya was lacking. There was a talking points memo somewhere that was very widely distributed -- which is the analog version of what I am describing.

      Look it up.

    2. Re:Hopefully they don't read slashdot for this by Brainboy · · Score: 2, Funny

      If Slashdot were used these would be the word that burst:

      "Natalie" "Portman" "Soviet" "Russia" "1337" and "Dell"

      --
      Just a guy with an opinion
    3. Re:Hopefully they don't read slashdot for this by anethema · · Score: 1

      Lucky this thing didnt happen in the kill the commies days...

      "Soviet and Russia are very popular words on this particular news discussion site. Start recording IP's and dispatch the SWAT teams!"

      --


      It's easier to fight for one's principles than to live up to them.
    4. Re:Hopefully they don't read slashdot for this by Anonymous Coward · · Score: 0

      Nice try, except that it is in fact an existing word. It wasn't manufactured during the 2000 elections, it didn't come out of nowhere. It's been here all along, for those with the intelligence to crack a fucking dictionary and VERIFY IT.

      http://dictionary.reference.com/search?q=gravita s

    5. Re:Hopefully they don't read slashdot for this by metlin · · Score: 1


      Talking of manufactured words, there are a lot of such words which float around, some of which are created ones, like Medireview or common mistakes, like anyways as well as lingos like wanna/gonna which are part of the langauge.

      It gets particularly interesting after a while to watch the stats. I'd infact written a primitive paper on such behaviour long ago, it can be found here at my site. I'd also written an agent based on this, details of the agent can be found here.

    6. Re:Hopefully they don't read slashdot for this by The+Troll+Catcher · · Score: 1

      How could you possibly forget "First Post!" and "Hot grits down my pants"? Shame, shame.

      Not to mention "All Your Base"... ;)

    7. Re:Hopefully they don't read slashdot for this by JDWTopGuy · · Score: 1

      You forgot "grits"!

      --
      Ron Paul 2012
    8. Re:Hopefully they don't read slashdot for this by Brainboy · · Score: 1

      Well the thing they measure the change in words. All your base has been around forever. If it hasn't changed much, it won't show up. Same with first post. Though i did miss grits so I beg forgiveness.

      --
      Just a guy with an opinion
    9. Re:Hopefully they don't read slashdot for this by Xenographic · · Score: 1

      Well, a "burst" implies that they don't consider just how often it's used but a *change* in how often it's used; unless we had a massive troll attack, they *shouldn't* notice any change therein & even if there was, the burst would be 'localized' to slashdot [rather than appearing across all the blogs] and therefore not a very broad trend... :]

  3. Blogdex by nob · · Score: 4, Informative

    Theres another "what's popular on blogs" webpage at Blogdex. It tracks links, showing which pages are most linked to.

    --
    daed si luap
  4. Nukular weapons by flokemon · · Score: 5, Funny

    In a simple historical test of the technique, Kleinberg analysed all the annual State of the Union addresses given by US Presidents since 1790. He found that particular word "bursts" could indeed be linked to important events at the time the speeches were delivered.

    Has an important increase of the use of the word "nukular" been reported in the last few weeks then?

    1. Re:Nukular weapons by tmark · · Score: 3, Insightful

      He found that particular word "bursts" could indeed be linked to important events at the time the speeches were delivered.

      Does anyone else find this painfully obvious ? Certainly you wouldn't expect to hear the word "computer" much in FDR's state of the union addresses; just as you wouldn't expect to hear "icebox" in GWB's addresses.

      The idea isn't as revolutionary as the author makes it out to be. People have been searching for terms in literature and using counts as indices of "importance" for a long time. Just to cite one example, researchers commonly use citation indexes to find out which fields are/were "hot".

    2. Re:Nukular weapons by hagardtroll · · Score: 1

      I bet you'll find the word "Pot" to be popular in both administrations.

      As a chicken in every... vs I didn't inhale.

    3. Re:Nukular weapons by coopaq · · Score: 0
      I believe it's actually "nukyalar".

      You're not from Texas huh?

    4. Re:Nukular weapons by Anonymous Coward · · Score: 0

      BOTH??
      Last I checked there were over 40 administrations

  5. Google by Citizen+of+Earth · · Score: 3, Informative

    Google can do much the same thing, on a real-time basis, by examining what phrases are searched for.

    1. Re:Google by ccweigle · · Score: 4, Informative
      Google can do much the same thing, on a real-time basis, by examining what phrases are searched for.

      And they do that much already ... on their Zeitgeist page: http://google.com/zeitgeist

      But this is different. The article is about monitoring the blogs, not the searches. As suggested in another comment, this may be related to Google's acquisition of Blogger.

    2. Re:Google by XCondE · · Score: 3, Insightful

      I'm eager to see what will come up next with Google's recent entry in weblog world.

      It's just what I thought when someone said " Blogs are like dreams; they're only interesting to the people they belong to".

    3. Re:Google by gmuslera · · Score: 3, Insightful

      Is more subtle than that, is not what you are searching for, but it tracks how you (or society) changes it way to express itself based in current trends, news, etc. That can be related or not with what you are currently searching in google.

      In a way, it should track even how languages evolve, how new meanings are given to existing words (i.e. in the past would anyone think that defensive attack were not opposite words? :)

      I wonder if this kind of analysis can be affected by people like me that without proper knowledge of english write in it :)

    4. Re:Google by CvD · · Score: 1

      Actually, their news page is even more up to date. The zeitgeist is updated infrequently compared to the news page. Of course, the news page is biased, since it only gets its info from news sites which have been defined already.

      Of course they could do something similar with weblogs.

      Cheers!

      Costyn.

    5. Re:Google by Anonymous Coward · · Score: 0

      please dont post that link again.
      dear god the stats on that page are depressing page (yes i know there not perfect)

  6. Great.... by xtermz · · Score: 4, Funny

    ..Now we're going to see Pepsi add's slinging "in soviet russia, you drink pepsi' , and Nike yelling about "all your sports belong to us..."...

    --


    I lost my concept of community when my community lost all concept of me.
    1. Re:Great.... by Anonymous Coward · · Score: 1, Informative

      that's "IN SOVIET RUSSIA, PEPSI DRINKS YOU !!!" and "All your sport are belong to us". Just crossing the i's and dotting the t's.

    2. Re:Great.... by Surak · · Score: 2, Funny

      The Dell Dimension, with the powerful Intel Pentium 4 processor.

      "Dude, imagine if you had a Beowulf cluster of these things!"

    3. Re:Great.... by Hard_Code · · Score: 1

      "in soviet russia, you drink pepsi"

      No, that would be misleading advertising, because obviously:

      In Soviet Russia, Pepsi drinks YOU!

      --

      It's 10 PM. Do you know if you're un-American?
    4. Re:Great.... by Anonymous Coward · · Score: 0

      No no no no no, you have it all wrong!

      In Soviet Russia, pepsi drinks YOU

      get your memes straight! :-)

    5. Re:Great.... by nigelthellama · · Score: 1

      I thought it was "in Soviet Russia, Pepsi drinks you"?

    6. Re:Great.... by jafac · · Score: 1

      Actually, I was out at a retirement lunch the other day, at a chinese restaurant, and the new thing is to read the fortune cookie thusly:

      "In Soviet Russia. . ." (text of fortune cookie).

      Which is a refreshing change, but often not as funny as:
      (text of fortune cookie) ". . . in bed."

      --

      These are my friends, See how they glisten. See this one shine, how he smiles in the light.
    7. Re:Great.... by Anonymous Coward · · Score: 0


      $$$$$exyGal [slashdot.org] is a MAN, baby!


      How d'ya know? Any evidence to suggest so? Other than some female pr0n links that is?

      Just out of curiosity :-)

      ~metlin

    8. Re:Great.... by Anonymous Coward · · Score: 0

      What? Afraid I'm going to expose you for the fraud you are, $$$$$SexyGal? :-P

  7. Conspiracy by Scott+Hussey · · Score: 3, Funny

    I can already see the collusion of weblog editors.

    "Okay, everyone write about polka dot socks tomorrow. And throw in something about drinking rotten milk. I bet we can start a new fad..."

    --
    Scott, Keeper of the Crystal Flame
  8. "What's cool"? by ites · · Score: 4, Insightful

    By my definition "cool" is that which most people have not yet discovered. Example: that... ah, but I'm not going to tell you. Perhaps this method can tell you what just became cool, but it's hard to track something that is by definition under the radar. Otherwise, just track Google searches. You'll soon see what's popular.

    --
    Sig for sale or rent. One previous user. Inquire within.
    1. Re:"What's cool"? by Anonymous Coward · · Score: 1, Insightful

      Actually, for most Americans, cool is not
      what they "discover" but what television
      and the media tell's them is cool.

    2. Re:"What's cool"? by deanc · · Score: 4, Insightful

      That's what the researchers seem to track. Not the commonality of a phrase, but the "burstiness" of a certain word or phrase... ie, the delta of the word use over time. High delta values indicate something is starting to take off, though it may not yet have become popular or mainstream. That's a decent metric of "coolness."

    3. Re:"What's cool"? by tekunokurato · · Score: 2, Insightful

      But Marketable is what is becoming cool, and things stay marketable for longer than a few instants. This is not really about cool, it's about marketably cool.

    4. Re:"What's cool"? by Fishstick · · Score: 4, Insightful

      Ever see Merchants of Cool on Frontline?

      A Report on the Creators & Marketers of Popular Culture for Teenagers

      Yeah, that's right. Popular Culture is manufactured -- everything the teenies think is "cool" or "hot" is identified months in advance by a highly sophisticated machine that probes the minds of kids to predict what will be the next trend so that the marketing establishment can gear up to take advantage of the short window where the "thing" is "cool" and can be sold to teens in such a way that they don't even realize what is going on.

      --

      There is much cruelty in the universe, John.
      Yeah, we seem to have the tour map.

  9. Applications by benjiboo · · Score: 2, Interesting
    This work has been around for a long time in the data mining literature. For instance, searching the logs of customer service calls to identify common problems etc.

    These techniques could easily be expanded to searching weblogs - I imagine the findings could be very interesting for content providers - eg a simple measure of what people want to read about.

    --
    Vacancy for signature. Apply within.
  10. Don't you know? by Anonymous Coward · · Score: 0

    It's all part of Google's master plan! Daily indexes of the web including blogs to find new crazes so they can buy stock in companies involved in those crazes. They'll all be rich!!! Muahahahaha.

  11. Apache Logs too by josephgrossberg · · Score: 3, Interesting

    "Joe Millionaire winner" and "Bubb Rubb" have generated most of my personal blog's hits.

    I, myself, am a distant third.

    Write about enough things and then check your referral logs for Google and Yahoo searches (which include the query in the URL), and you get an imperfect idea of what people are interested in this week.

  12. Useful? by gpinzone · · Score: 1

    I wonder how long before this can be done real time enough to really make this useful.

    Define "useful."

    1. Re:Useful? by feldsteins · · Score: 1

      It might give us the ability to make intelligent statements about what people are thinking and talking about - a cultural barometer, if you will. One that's tied to something other more valid and more immediate than the Letters section of People magazine.

      Someone once wrote (I'm really sorry, I forgot who - public thank you to the person who knows) something like "individually, nobody knows what's going on but collectively, we know exactly what's going on." This kind of meta-information is a social scientists wet dream, I bet. I admit I'm fascinated. It's very...William Gibson? Was he the guy who wrote that line? Damn it.

      --
      You like your Macintosh better than me, don't you Dave? Dave? Can you hear me Dave?
    2. Re:Useful? by Anonymous Coward · · Score: 0
      "individually, nobody knows what's going on but collectively, we know exactly what's going on."

      The paraphrase is from John Brunner, used to describe the success of Delphi betting pools in The Shockwave Rider.

      It's very...William Gibson? Was he the guy who wrote that line?

      Wrong, and not a good guess either. Brunner was writing about cyberspace before Gibson got there with his narcoleptic Neuromancer. For some reason, Gibson gets all the credit among the disaffected geek youth, while Brunner (whose work featured interesting characters and actual points to the stories of disaster and self-induced woe) is ignored.

  13. Useful? by Longjmp · · Score: 4, Insightful

    I wonder how long before this can be done real time enough to really make this useful.

    Yes, I bet the spammers can't wait until they can use it...

    --
    There are fewer illiterates than people who can't read.
  14. Imagine by jos091 · · Score: 3, Insightful

    Imagine the feedback loop that could develop...

    1. Re:Imagine by kalidasa · · Score: 2, Funny

      Re: Imagine the feedback loop that could develop...

  15. God Help Us by Oculus+Habent · · Score: 3, Funny

    And think, the DMCA will become the most popular piece of legislation in existance - at least on slashdot.

    And CowboyNeal is the most popular man alive!

    --
    That what was all this school was for... to teach us how to solve our own problems. -- janeowit
    1. Re:God Help Us by kalidasa · · Score: 1

      "All your base are belong to us" is already being used in marketing. Don't you guys listen to NPR?

    2. Re:God Help Us by gsfprez · · Score: 2, Funny

      asshat asshat asshat
      asshat asshat asshat
      asshat asshat asshat
      asshat asshat asshat

      just doing my part...

      --
      guns kill people like spoons make Rosie O'Donnell fat.
  16. pathetic by Joe+the+Lesser · · Score: 1

    'has come up with clever method of identifying what's cool'

    So is this guy like Screech in Saved by the Bell, constantly looking for a way to impress Zack and the guys?

    --
    "I only speak the truth"
    Karma: null(Mostly affected by an unassigned variable)
    1. Re:pathetic by flokemon · · Score: 1

      It reminds me a bit of how everyone on UK TV used to say "wicked" constantly at the time of BB3, while I never heard it in real life.

    2. Re:pathetic by Joe+the+Lesser · · Score: 1

      lol, and how I've never heard anyone in the US say 'wazzup' like they do in the Bud commercials.

      --
      "I only speak the truth"
      Karma: null(Mostly affected by an unassigned variable)
    3. Re:pathetic by flamingnight · · Score: 1

      I've heard people say 'wazzup' in the US. Only after the Bud commercials, though.
      I thought of suing Bud for creating noise pollution, but realized they don't have the money to reward me for my suffering.

  17. Identifying new crazes is useless by Anonymous Coward · · Score: 0

    unless your IQ is so low that you're taking pleasure being the first to quote Zerowing AYBABTU in your office.

  18. Useful for... by mcleland · · Score: 1, Funny
    Great! Now I'll know sooner what the latest pop culture craze is so I can "be different" and follow everyone else to stay popular! Then I can put that information in my blog and let everyone else know I'm following the latest trend!

    Seriously, just read /. if you want to know the important stuff of the day. :)

    1. Re:Useful for... by Duds · · Score: 4, Funny

      Seriously, just read /. if you want to know the important stuff of the day. :)

      Twice usually.

    2. Re:Useful for... by skillet-thief · · Score: 2, Funny
      Great! Now I'll know sooner what the latest pop culture craze is so I can "be different" and follow everyone else to stay popular!

      Except now popularity will last about 6 hours, tops, before some new wave of pop culture replaces it. By the time craze "X" hits the craze detector, all the really cool people will already be onto craze "Y", which will be detected a few hours later.

      It's like the whole "avant-garde/in-style/out-of-style/retro/back-in-s tyle" cycle managed by a Perl script in an infinite loop.

      --

      Congratulations! Now we are the Evil Empire

    3. Re:Useful for... by Mard · · Score: 1

      Thrice if you're lucky.

      --
      DRM = Digitally Restricted Media. This is a viral sig, pass it on.
  19. Too late by then by Anonymous Coward · · Score: 0

    by the time it's talked about on the internet, us real trendsetters have moved on to something else :o)

  20. Microsoft! by t_allardyce · · Score: 1

    This system should be run over all speech's too. ".Net" "Drag & Drop" "Point & Click" they are full of it.

    --
    This comment does not represent the views or opinions of the user.
  21. algorithm? by Jrod5000+at+RPI · · Score: 1

    my big question is about the algorithm... it doesn't seem difficult to parse a document, store the words in (let's say) a tree, then compare it other documents. simple keep counters for each word to track how often it's mentioned.
    i'd really like to know what this guy did.

  22. Daypop by Apreche · · Score: 4, Informative

    http://www.daypop.com

    Its got the top 40 every day. Doing it some other way would only catch memes sooner. And if the system doesn't catch it until its popular, it really doesn't help. What we need is a large and complete database of all meme type things.

    --
    The GeekNights podcast is going strong. Listen!
  23. Oh, great by ZoneGray · · Score: 3, Funny

    Whoopeee. The marketers will start using this to identify trends, and next thing you know, we'll have some fast food named "Cheese-Eating Surrender Monkeys."

    Not In Our Brand Name, say I.

    1. Re:Oh, great by Anonymous Coward · · Score: 0

      On the subject of cheese (ie, US anti-French sentiment),

      I would rather be eating cheese and reading Sartre on the banks of the river Seine than eating popcorn with a born again bible-belt fundamentalist Republican administration in Crawford, Texas, execution capital of the world.

      George Galloway, at the London anti-war demonstration 15 Feb 2003.

    2. Re:Oh, great by Anonymous Coward · · Score: 0

      >>execution capital of the world

      No, that would be Baghdad.

      I suppose, though, if I made some wisecrack about "Qu'ran Belt Fundamentalists," it would be considered insensitive. But hate speech about the "bible-belt" is fashionable.

      Cheerio. Peace In Our Time.

    3. Re:Oh, great by Anonymous Coward · · Score: 0

      I suppose, though, if I made some wisecrack about "Qu'ran Belt Fundamentalists," it would be considered insensitive

      No, that would be CNN

      Peace In Our Time

      Not in this century

    4. Re:Oh, great by gughunter · · Score: 1
      I would rather be eating cheese and reading Sartre on the banks of the river Seine than eating popcorn with a born again bible-belt fundamentalist Republican administration in Crawford, Texas, execution capital of the world.

      Well, diff'rent strokes for diff'rent folks. Since you asked, I'd rather be sitting down with a sack of Rusty Brown's Ring Donuts at the Front Page Cafe in Vice City. Now that's the execution capital of the world!

    5. Re:Oh, great by blair1q · · Score: 1

      "Inanimate Carbon Rod"

      New! From Mattel!

  24. Let the webloggers determine what's cool? Heh. by rubberpaw · · Score: 4, Insightful

    Of course, since there is only a very specific socioeconomic subset of the world population weblogging, what real usefulness does this give us? Honestly, even if you did ranking based on the most popular weblogs, that wouldn't help you very much.

    Furthermore, this thing isn't telling me anything I don't know. So it finds the word "Vietnam" during the Vietnam years. Hooray. I bet it finds the word Iraq today, or the phrase "Bin Ladin" last year.

    Whoopdie-do. I'm impressed :P. Unless this thing actually can find out the things that people are excited about that aren't well-known, it's pretty much just another search tool limited to blogs.

    1. Re:Let the webloggers determine what's cool? Heh. by barnaclebarnes · · Score: 4, Insightful
      Unless this thing actually can find out the things that people are excited about that aren't well-known, it's pretty much just another search tool limited to blogs.


      Thats the whole point. Weblogs are not the mainstream media so he is betting that a new craze (or refresh of an old one) will show up there beofore the mainstream sites get a hold of. Face it, once it has hit CNN it is already past its sell by date.


      Take the whole potato gun thing for instance. if this was appearing on peoples weblogs 6 months ago and an underground following had started then it would pick this up. Could be a perfect time for one of the toy companies to start producing a parent friendly version (Not sure how...but hey!). By the time the craze hits CNN Toys 'R Us is stocked with a version that fires water ballons, only uses compressed air and comes in 10 different plastic colours. Then they would have the advantage before the other companies jump on the bandwagon.


      Of course, since there is only a very specific socioeconomic subset of the world population weblogging, what real usefulness does this give us?


      A lot! Let me see, I have a large group of people who are rich, computer owning, and probably middle /Upper Middle Class all saying they want X. Now who is your target audience again? Not low income, no disposible cash types.

      /b

      --
      [Please type your sig here.]
    2. Re:Let the webloggers determine what's cool? Heh. by Repran · · Score: 1

      Your missing the point. He used it on the state off the union adresses to have an easy test for the algorythm. It spit out the right words so he knew it would work in cases where the result will be far from certain.

      --

      -- Contradictions only exist in thought - not in reality.

    3. Re:Let the webloggers determine what's cool? Heh. by tekunokurato · · Score: 1

      Well, I don't think anyone's going to start a marketing campaign based solely on the information retrieved from blogs. However, it can act as a nice supplement to other information one might have, in that it can 1. reinforce, 2. show a rise in popularity of a specific item, and 3. show a decline in popularity of another item.

      Viral marketing is something marketing officers have been trying to tap into forever, and this might help them determine how that sort of thing works; how information and trends are passed between people.

      Also, maybe they could pinpoint characteristics of market leaders, i.e. those who talk about major trends before those trends get major.

    4. Re:Let the webloggers determine what's cool? Heh. by JPelorat · · Score: 1

      Yah, no kidding.. you're much more likely to find "Turd Bursts" instead of "Word Bursts" on a weblog...

      --
      Hokey statistics and ancient misconceptions are no match for a good thought in your head, kid!
  25. News Flash by zorglubxx · · Score: 1

    New method discovered by Cornell scientist discovers that Slashdot is not cool anymore due to the same words (Linux and Open Source) being repeated over and over again.

    News at 11.

    1. Re:News Flash by grub · · Score: 1


      New method discovered by Cornell scientist discovers that Slashdot is not cool anymore [...]

      "anymore" suggests that slashdot was once cool. This has never been the case.

      --
      Trolling is a art,
  26. It's useful *now* by backlonthethird · · Score: 3, Insightful
    I wonder how long before this can be done real time enough to really make this useful.

    Why have to wait until it's realtime? Historical analysis is very useful, and not just to historians. Linguists, anthropologists, social scientists, etc.. Taking such a body of texts is called studying a "corpus," and such studies often yield surprising and interesting results (better than "atomic" showing up in the ocld war). A new method like this would be very useful to nearly every discipline in the humanities I can think of

    Not all geeks are computer geeks. Not all nerds care only about the future.

  27. The state of the World from google by MrBlic · · Score: 2, Interesting

    The ultimate way of watching trends on a month-to-month basis has to be Zeitgeist from Google.

    --
    Celebrate Excellence!
    1. Re:The state of the World from google by caluml · · Score: 1

      Yeah, but those are the things that everyone is ALREADY asking questions about.
      This study is to find the things that the "fashionable" people are talking about BEFORE they get big.

    2. Re:The state of the World from google by Anonymous Coward · · Score: 0

      Top 10 Gaining Queries
      Week Ending Feb. 11, 2003:

      5. "cricket world cup"

      That's a bit of a slap in the face to all of you who think that cricket is boring.

  28. No Kidding? by BuBu_ · · Score: 2, Insightful

    Did anyone read the article? Amazingly enough this wonderful software with its POWERFUL algorithms proved a true point of "no shit". While running this gem of coding genius, the authors managed to find reoccuring references to "Depression" while scanning texts from the 1930's. Imagine that, finding the word depression from a time period thats been nicknamed "The Great Depression" I would of never linked the word "Depression" with "The Great Depression". Have we really reached the point where we can just do the same shit over and over again and it's magically a new invention?

    MS is bringing out 3 Degrees which is reinventing IRC, this guy is telling us the painfully obvious, and I've been working on this little trick thats gonna really change the way we think of food, get this guys: I take two pieces of bread, a piece of cheese, and a piece of meat and stack it together.. I call this wonderful new life shaping discovery "The meat-and-cheese-on-bread" I really think it's gonna change how we eat!

    1. Re:No Kidding? by Anonymous Coward · · Score: 1, Insightful

      Don't be silly.

      He thought he'd invented a method to show emerging trends, by analysing how a word's frequency of use increases. So, you want to test that method. So you pick something like the great depression, and hey, sure enough, the word "depression" is used more as the great depression starts. It's entirely possible that it might not have been (this is science, you know), in which case you'd think that the method didn't work. But it passes the test, so you can be more confident about using it to spot trends you didn't previously know about.

      It's like when Newton invented his laws of motion and gravity: they predict that the Moon should stay in the sky, and that's important (and divots who say "no shit!" should be pilloried and ignored) because if they said otherwise, they'd be wrong.

    2. Re:No Kidding? by epsen · · Score: 1
      "The key is to find unexpected changes in the frequency of the appearance of words," he told New Scientist.
      He looks at changes in the word frequency, not the frequencies itself. The period has been nicknamed after the period took place. If this algo had been run at that time it would probably have detected that the times were changing to worse and caught the trend at that time.
    3. Re:No Kidding? by BuBu_ · · Score: 1

      How about you don't be silly?

      Do you really need to have a guy go out and research and spend money/time on figuring out that world events shape what we talk about? Are you really that socially inept that you couldn't determine this on your own using a bit of logical reasoning? I mean come on now, what am I really showing about trends in word usage when it's almost painfully obvious to note that people are shaped by our world events. You notice we all don't stomp around babbling about how much we hate the Japanese, but 65 years ago you can bet your ass people did. When it really comes right down to it, how can you believe that research of this nature is showing us something we couldn't of seen before. Do we really need algorithms to prove social theory?

    4. Re:No Kidding? by BuBu_ · · Score: 0, Offtopic

      How can you even guess that? I mean look at what you're saying to me. "I think that I should be able to predict the future based off what people are saying". Hey buddy, ESP isn't real, and don't you worry.. you aren't the only one who can't predict the future. The point I'm driving at is: It's no shocking relvelation that someone would find that we're using words to describe our world events. Those are the things that actively shape our lives and our views, so why is it some kind of big discovery to find out that "Hey! They were talking about the depression as it was going on! Imagine that!"

    5. Re:No Kidding? by Anonymous Coward · · Score: 0

      Off the top of my head, one use for this technology is that advertisers always want to know what's "hot", and will pay money to learn. This tech spots that we talk about world events, yes, but that's the test he ran to prove the tech worked, not the final end use of the tech; a fact you seem to have missed completely.

      I think you haven't understood the first word of this article, or any of what I said to you. Read it again.

    6. Re:No Kidding? by nfk · · Score: 1

      The examples may seem obvious but imagine for a second that you didn't know anything about History. You just run the algorithm and get the trends for different periods, that's a useful thing. They just ran it on State of the Union addresses to test it, when they do it with more diverse texts, spanning the whole human history and different cultures, interesting results may emerge. And the point is, you get a summary, or a timeline, in a fraction of the time it would take to actually study the issues. It won't replace in-depth analysis, but it gives you some clues before you start.

      Then there's the interesting possibility of following current trends in real time, using it on blogs, like they suggest, or on newspapers, letters, etc. Like you said in another comment, it doesn't "predict the future", it's not ESP, but having that kind of information (some of which will not be obvious by just glancing at data) in real time can be a great advantage.

    7. Re:No Kidding? by nicodaemos · · Score: 1

      By your reasoning, the war taking place in Europe from 1914-1918 was already named World War I by the participants. They were aware that this naming scheme was easily extended to incorporate future conflicts such was WWII, WWIII, etc.

      But, if memory serves me correctly, this war was actually called "the big war" or "the war to end all wars" by its participants. It was only years later, when WW II erupted that they renamed the earlier conflict to WW I.

      Over 100 years hindsight is 20/20. I think the goal of this technology is to provide hindsight over a span of days/weeks/months.

    8. Re:No Kidding? by critter_hunter · · Score: 1

      That's "The Great War". "The war to end all wars" is also correct, even though it's a pretty damn ridiculous name for WWI - unless they hoped to eradicate war by wiping each other out for no reason whatsoever

      --
      Karma: Could be worse (could be raining)
  29. A new apache module...? by stroudie · · Score: 2, Interesting

    I can see a nice distributed implementation for burst-searching - a "mod_ephemera" module for apache.

    The module would count words/phrases most commonly served (less tags and the top-n most common words in the language-encoding), then serves out the top-10 as HTTP header messages. That way, the results are unobtrusive and easy to recover.

    Of course, this approach would inevitably be easy to skew/cheat. Anyway, that's my sixpeneth :)

  30. And in other news... by djupedal · · Score: 3, Funny

    ...Yahoo, today, was accused of seeding 2.5 million user blogs with keywords designed to influence/fool/skew robots that attempt to identify what's cool by automatically searching weblogs for so called 'word bursts'.

  31. so much for that theory by stinky+wizzleteats · · Score: 2, Funny

    I guess this pretty much lays to rest the article about how nerds don't work to be popular. We automate it!

  32. Bwahahahahaha by Anonymous Coward · · Score: 0
    So is this guy like Screech in Saved by the Bell, constantly looking for a way to impress Zack and the guys?

    To answer your question: yes. He even looks surprisingly like Screech (he went into pornos, didn't he?).

  33. Relegence (~eNow) already does this in realtime by lieutenant · · Score: 2, Informative

    They have a realtime search mechanism that can search within Chat rooms also , and TV and radios streams. (Kevin Kelly is on the Board). Used to be a downloadable personal edition. there is a free trial. Not a plug !!! , they became a corporate (financial and others) company , turning back on "Free Information Now" roots. but at least it works :)
    http://www.relegence.com

  34. Caution should be taken, and research valued by deadsaijinx* · · Score: 1

    It would be foolish to look at whats on blogs as being whats popular. If there was a sudden burst of people blogging about palladium, it would be foolish to mistake it for being popular. Instead, a burst of mentionings really means more along the lines of what is a hot topic. Well, at least IMHO.

    --
    YOU SUCK BALLS!
  35. Word Bursts? by ackthpt · · Score: 1

    Word Bursts? Back in my day we called these cliches, I guess that was the old craze.

    --

    A feeling of having made the same mistake before: Deja Foobar
    1. Re:Word Bursts? by ianscot · · Score: 1
      Cliches, yes, or really just fads. Crunching through business speeches of the last 20 years, you could sure see the jargon trends come and go. That'd make a fun bar graph for an all hands meeting -- not that it makes any sense as a bar graph, you understand, those are the only kind of chart we're capable of following, supposedly...

      When I saw the term, I thought of Tourette's syndrome. Consider it: wouldn't it be nice if presidents did suffer from Tourettes "word bursts" in their State of the Unions? It'd be the only words they said that weren't written on the teleprompter by paid lackeys -- and we'd know more about the state of the union, no question.

      --
      "Fundamentalism" isn't about divine morality. It's about human authority.
  36. Zeitgeist and Memes by mrmiasma · · Score: 3, Informative

    Sounds like a combination of Google's Zeitgeist and LiveJournal's MemeTracker. In other words, nothing that new.

    It's also the basis for Computational Lexicography. Doing analysis on large corpora. One of the interests people have in this field is introduction of new words in society. The field used to use corpora such as the British National Corpus, but since the explosion of the Web, sites such as Google can far exceed that size. Weblogs are simply a good example of a more natural form of language. The interesting thing would be not so much to find new trends through words... but if we can truly solve the whole natural language parsing problem and use such information to extract higher-level knowledge

  37. Feedback loop and dotcom crash by skillet-thief · · Score: 4, Interesting
    It is kind of like the stock market craze and the theory that "all the information you need to know about a stock is contained in the market itself" (ie. in the stock's chart). Enough people start believing that theory, and the stocks quit behaving rationally.

    The analysis only works if your tool doesn't start modifying the data you are analyzing. If this thing ever caught on, it would quickly become meaningless, because everybody wants to be part of whatever craze is going on. Every morning you check which words are hip, you put them on your website... etc. etc.

    You are right about feedback: the buzz would become a terrible din. That said, it is a cool idea.

    --

    Congratulations! Now we are the Evil Empire

  38. This isn't new by Nick+Arnett · · Score: 1

    I've been using this technique for years... We partially developed it at Opion, now part of Intelliseek, and I'm currently using it as one of a number of methods to find out what's going on in developer communities. It's not necessarily even all that clever, since the data are readily available by comparing the statistics that search engines inherently create.

    Nick

  39. The Inevitable Result... by Root+Down · · Score: 1

    It's no secret that the most commonly searched item on the internet is pronography. Only once has this top ranking ever been dethroned - September 11, 2001. It returned to the top spot shortly thereafter. So, by examining web logs, we will find that - year after year - we are all interested in pornography. While this is likely the case, it is also a trapping of the medium through which the research is being conducted. You'll excuse my complete lack of surprise.

    1. Re:The Inevitable Result... by Anonymous Coward · · Score: 0

      No. This tracks increases and decreases in word usage; the first derivative of word usage with respect to time. So it would spot sudden massive changes in the popularity of porn (eg if a new wave of puritanism swept the world), but wouldn't be interested in something that remained about as popular as it ever was.

    2. Re:The Inevitable Result... by Anonymous Coward · · Score: 0

      Only once has this top ranking ever been dethroned - September 11, 2001.

      For a few fleeting days a few years back, "MP3" was the most searched-for term. Of course, pr0n rapidly became the top term shortly thereafter.

    3. Re:The Inevitable Result... by OrientalGuitarpick · · Score: 0

      No! Ping say you wrong. Ping say you not know what you talking about. Pornagraphy offends Ping, has chinese girls in it. No like. Ping tired of seeing chinese girls, Ping want korean. Where Ping's piece of pie?

    4. Re:The Inevitable Result... by Anonymous Coward · · Score: 0

      Shut the fuck up you dirty Chink piece of shit. It's too bad faggots like you weren't run over at Tiananmen Square because the world certainly needs less dirty communist faggots. Take your hammer and sickle and shove it up your ass, you dirty pink-skirted cum-guzzling shitstain. Go take a shower, gouge your slanty eyes out, shove a fucking Linux CD up your ass, and stand in front of a Chinese tank. The world would be a better fucking place without China, Korea, and all the fucking sand niggers. I fucking wish Bush would nuke all the fucking bastards. Chinese fucks like you are the new niggers here in America. Japan should fucking invade China again and put all the lousy bastards to work as slaves. Same with fucking Korea. Then send all the kikes and sand niggers in too. And fucking blow up a couple of nukes and all the world's problems would be solved. No more fucksticks like you. Now go back to raping little boys and taking it up the ass, you fucking Chink.

    5. Re:The Inevitable Result... by OrientalGuitarpick · · Score: 0

      You piss Ping! Ping gonna kill you now, Ping says to go shove your dildo up your arabian ass. Ping says ALLAH CONDONES THE ANAL SEX! Ping no like you towelheads no more. World would be better place without you, Ping make suggestion you crash plane into own tower. Or throw stone at mud hut, same difference to Ping. Ping hope Americans kill you.

  40. Nope.. by -+FuckingNerds+- · · Score: 1
    Not going to work. Everything popular!=cool.

    All you need is an advertising blitz that costs a few million dollars and you have created cool!

    It's shameful what makes the news these days.

  41. I did this on Slashdot by daves · · Score: 1

    It came up with the words "dup" and "speling".

    --
    People who disagree with you are not automatically evil, greedy, or stupid.
  42. New article title by Samus · · Score: 2, Insightful

    Should have been entitled "Nerds Find Automatic Method to Enable Them to Talk to Other People." I have this picture in my head of some poor guy who is a social outcast that wants to figure out a way to be able to talk to a girl about things she might be interested in.

    --
    In Republican America phones tap you.
  43. May just find buggy software with lousy support by Mostly+a+lurker · · Score: 1

    Many of the Google searches I do are just trying to find out if anyone has found a fix for bizarre problems in software products. This is not quite the same as finding out what I find to be popular!

  44. Art Exhibit by ACNeal · · Score: 1

    I can't think of any of the details, so if anyone has them, please post links. There is an art exhibit talked about in the latest issue of Linux Journal that describes an art exhibit that is a wall of tiny little lcd displays.

    This wall changes often. The LCD's have messages culled from the internet in real time. I haven't seen the exhibit, just the picture on the cover of the magazine, but it seems like an interesting endeavor that is slightly akin to the artcile.

    1. Re:Art Exhibit by ACNeal · · Score: 2, Interesting

      Found it, after some digging over my lunch hour.

      The listening post is an art exhbit that more or less lives. It monitors certain chat rooms, and posts messages from those chat rooms to a wall of small lcd displays.

  45. Intresting approach. by IncarnationTwo · · Score: 1

    There are ads that are based on very idea of anti-coolness or repulsivenes. Like old benetton ads that were there to shock people.

    I find the idea of using the forementioned website as an ad figure as repulsing... but let me ask who of us does not remember the address of forementioned site.

    If a product could sell even if its ads were repulsive (ie. Benetton sold clothes with pictures of shirts used by people who were violently slain), using web address named like tagline for the product with content as repulsive as in forementioned site, and using the adress as un-responsibly as the trolls here do, the campaign could yield really good results.

    --
    In dream society, people could be given the ability to mod replies. In real life, it would be disaster.
    1. Re:Intresting approach. by JDWTopGuy · · Score: 1

      Uhh, ok. So what are they selling? You've obviously been there, so I'm certainly not going.

      --
      Ron Paul 2012
  46. Better start shopping now... by rustycage · · Score: 1, Funny

    to get the kids that "Beowulf cluster" i keep hearing about

    --
    No Sig For You
  47. Re:Oops by All+Your+Base+Nazi · · Score: 0

    Of course I meant, "all your sport are belong to us." How embarrassing.

    --

    Keeping All Your Base parodies correct since AD 2002.
  48. This is great for customer support! by G.+W.+Bush+Junior · · Score: 2, Insightful

    The approach could also be applied to sifting through other types of information. Identifying word bursts within email messages sent to a company's customer support address might help maintenance staff spot a major new problem.

    I'm sure customer support employees are going to love this idea... This way you can keep up an appearance of actually having read the customer emails, while really just redirecting to /dev/null (through the filter of course).

    --
    "I don't know that Atheists should be considered as citizens, nor should they be considered patriots." -George H.W. Bush
  49. Interesting use in science research by nfk · · Score: 2, Interesting

    I attended a conference last year, where they proposed a similar method to find trends in scientific fields, and more importantly, link them and predict future connections. For instance, when words from two unrelated fields start showing up associated in many papers, there is possibly a trend for those fields to meet and merge in the near future. Of course Informatics doesn't replace traditional methods, because it needs the input data, but it's a helpful tool.

  50. State of the Union Address? by nycsubway · · Score: 1

    So, words like evildoers, misunderestimated, and axis of evil are now cool?

  51. Market department's dream.. by BuR4N · · Score: 1

    If done in real-time it could be very valuable to a company to probe current trends and alter commercial plan and demographic target for their products...

    Its a cool thing from a tech. aspect , but I'm not quite sure I like it.... kind of 1984:ish feeling all over it...

    --
    http://www.intellipool.se/ - Intellipool Network Monitor
  52. Geeks finally figure out what is cool? by pcraven · · Score: 3, Funny

    Oh great. Just what we need. "Well, after careful analysis computer analysis with my powerful algorithms, I have concluded that break-dancing is now cool. I will be the first nerd in history to be atop this new trend."

  53. All your base are belong to us by Zog+The+Undeniable · · Score: 1

    Now we have the facts to say, "Dude, that's so 2001".

    --
    When I am king, you will be first against the wall.
    1. Re:All your base are belong to us by Anonymous Coward · · Score: 0

      It could also tell us how many jackasses started
      saying "Dude" after the Dell commercials came out.

      Surf's up, DUDE.

  54. Examples by kahei · · Score: 1

    So, in this article, the examples are:

    The words 'militia', 'British' and 'savages' were used a lot around the time the American 'militia' tended to fight the 'British' and what they called the 'savages'.

    The word 'depression' was used a lot during the 'depression'.

    The word 'atomic' was used a lot during the cold war, and 'Vietnam' was used a lot during the Vietnam war.

    I am utterly at a loss as to how such a seemingly interesting field as tracking word usage (well, it interests me) could possibly yield such stupefyingly, numbingly, almost frighteningly obvious and dull results.

    I can only assume the true significance of Dr. Kleinberg's results was simply too terrifying to be revealed...

    --
    Whence? Hence. Whither? Thither.
  55. Individual words? by gmuslera · · Score: 1

    Individual words will be useful showing some trends, but maybe counting phrases or how n-tuples of words could be better (AND harder). Sometimes "what's cool" is not a single isolated word.

    With common words the language or the way society express itself could change in a way that doing simple word counting not show, at least, not show clearly.

  56. strange... by dotgod · · Score: 2, Interesting

    Our definition of "cool" is the output of a computer analysis of weblogs then sit there wondering why nerds are so unpopular?!?

  57. Cool by algorithm? No, just common. by The+Babbler · · Score: 1

    Cool doesn't necessarily have to be unknown, but how could it be determined by an algorithmic formula? Common does not mean cool.

  58. dont they... by tx_mgm · · Score: 1

    ...do something similiar on yahoo already? how do they figure out their buzz index? because i thought it was based on buzzwords. mind you, its buzz words in hyperlink format, but its still the same concept i think

    --
    Gentlemen...BEHOLD!
    -Dr. Weird
  59. Not Real Useful by foo+fighter · · Score: 0, Redundant

    This isn't going to show what's popular in our Culture. It's going to show what's popular in the web logging sub-culture.

    I doubt there's that much of an overlap that the latter is an indicator for the former.

    Google's Zeitgeist is a much better tool since it's analyzing what the general web populace is looking for. I think that would be a more accurate indicator of current web culture trends.

    Still, neither are good indicators for our culture as a whole since they only reflect the interests of those who have ready access to a computer. The Zeitgeist is getting better all the time as more and more come online.

    --
    obviously no deficiencies vs. no obvious deficiencies
  60. Amazon.com by DarylBeattie · · Score: 2, Interesting

    Is this news considered "new"? This is exactly what Amazon did in order to forecast what book titles would sell the most money. They became the biggest web retailer because of this very same idea -- but many years ago. And now somebody at Cornell copies the idea but uses weblogs instead of IRC and newsgroups and suddenly he's "clever"? I know lots of people are complaining that the information gleamed from this is not useful; but it is! It's an amazing way to forecast what will sell.

  61. Well duh... research uncovers the obvious. by dpbsmith · · Score: 0, Redundant

    The example given is totally unimpressive: analyzing State of the Union messages, "In the years that immediately followed the American Revolution, for example, sudden bursts in the use of words such as "militia", "British" and "savages" are found. From 1930 to 1937 a spike in the use of the word "depression" is seen. And from 1949 to 1959 "atomic" is the word with the greatest "burstiness". Later in the 20th century, words such as "Vietnam", "Soviet", "communist" and "Afghanistan" increase sharply in usage." What a surprise. No tenth-grade history student would have ever guessed.

    Equally appalling is the suggested "use:"

    "For example, identifying word bursts in the hundreds of thousands of personal diaries now on the web could help advertisers quickly spot an emerging craze."

    See Shurtape's web site for a letter explaining to distributors that they have shifted all production to three popular consumer products that lead times on every other kind of duct tape is increasing to eight weeks... If only Shurtape had analyzed the burstiness of the words "duct tape" in blogs would they have been able to anticipate the spike in demand?

  62. Slashdot topics are twice as cool ! by MarkoNo5 · · Score: 1

    They'll find that most things that hit the slashdot frontpage are now twice as cool as they were before.

  63. Prior Art -- The Economist's "Recession Index" by JPMH · · Score: 2, Interesting
    Since the early '90s, the Economist has from time to time published occasional tongue-in-cheek articles about its "Recession Index", a useful leading indicator of the state of the US economy -- namely, the number of times the 'R-word' appears per month in the New York Times and the Washington Post. This appears to correlate strongly with the future state of the economy...

    eg:

    Dec 10, 1998

    Nov 21, 2002

  64. whats george michel by oliverthered · · Score: 1

    what's george michel and a pair of wellies got in common?

    They both get sucked off in blogs.

    --
    thank God the internet isn't a human right.
  65. Re:Cool by algorithm? No, just common. by Anonymous Coward · · Score: 0

    Remember the story a couple days ago about how /. geeks always got the tar beat out of them in high school? Your post is a good example of why.

  66. Stamp consumer on my forehead... by tazochai · · Score: 4, Interesting

    .... one more time why don't you. And I quote,

    "For example, identifying word bursts in the hundreds of thousands of personal diaries now on the web could help advertisers quickly spot an emerging craze."

    Gonfonit!!! Why does cool new social technology have to be related to ways to help people sell things to Americans! Why is it okay for us to be considered a nation of consumers, otherwise basically useless biological skinsacks?!

    I'll just strap my wallet to my chest with duct tape now and write my social security number in huge numbers on the back of my t-shirt for fast credit checks.

    1. Re:Stamp consumer on my forehead... by Hard_Code · · Score: 1

      So, consumer angst is the next trend! This is great, I'll just set up a Cafe Press store and sell t-shirts, mugs and mousepads:

      "I was brainwashed into becoming a mindless consumer and all I got was this high quality Beefy-T (tm) shirt"

      The product is you!

      --

      It's 10 PM. Do you know if you're un-American?
    2. Re:Stamp consumer on my forehead... by Frank+T.+Lofaro+Jr. · · Score: 1

      I'll just strap my wallet to my chest with duct tape now and write my social security number in huge numbers on the back of my t-shirt for fast credit checks.

      No need to go through all that trouble, just sign up for Micro$oft Passport.

      --
      Just because it CAN be done, doesn't mean it should!
  67. Tell me something original! by david_e_v · · Score: 1

    This, in the state it is presented in the article, is absolutely useless and obvious. I mean, proving that during the beginning of the Atomic Era, the word "atomic" was very common, or that during the war in Vietnam, "Vietnam" was a common word, is just making the evident more evident. It only proves that people tend to talk about what is happening at their time.
    If they were able to preview any of this, say, trends, it could mean something. But, don0t you think that, if you are able to find many occurences of a word, it means that it is already a trend?
    This is way it is difficult to foresee the future, because you don't have lots of newspapers talking about it (just joking).

  68. Neat idea, but what good is it? by Anonymous Coward · · Score: 0


    The burst could be something like: war has broke out in Iraq. Hey, just turn on radio or go to cnn and you get it.

    One of the use I can think is that the source of information come from individual. For example, if someone in a location A suddenly has stomachache and dizziness, and they complain about it, this could alert the official of something serious going on.

    Another is the terorist activity, which the US gov't already use this method to find out if there's possible teriorist attack. So, it turns out that the idea is nothing new.

  69. this is so old hat by Anonymous Coward · · Score: 1, Interesting

    I can think of two now defunct internet startups that did this like four years ago. One was a financial analysis tool that looked for stock symbols on particular financial chat boards. The other was based on usenet posts.

    If I wasn't going senile I would remember their names.

  70. Great by Dark+Lord+Seth · · Score: 1

    Like harvesting the info about some (rand(10)+15) year old person writing bullshit about (boyfriends|girlfriends|music|movies|stupid online quizes|webrings) with a site design that's usually so horrible they could be succesfully sued for crimes against huminity. All the marketing companies would encounter is the hype they created a few days before the harvest. So it might work to check if hypes/trends work out, but looking at "blogs" (the very word disgusts me) for something new an innovative is about as futile as trying to comprehend Bush' ramblings. The few remaining web logs or journals, as I prefer to call them without retching, are mainly technical. What trends are they going to squeeze out of the journal of a team of developers who want to keep the outside world up to date about what has happened lately? That such and so compiler sucks? That the network admin is a bitch? That the coffee tastes like sewage waste?

    Heck, if any of those marketing companies are GOOD, they'll MAKE their own trends, not ride around on the succes of others.

  71. link to real buying & selling by Anonymous Coward · · Score: 0

    I ran a system from my home computer that searched for 130 key words over major news sites. It came up with some interesting results. A variety of words such showed clear relationship to rise & fall of markets (Treasury bonds, EUR/US$). It was fun while it lasted.

  72. Paper is here by Isamu+Noguchi · · Score: 2, Informative
    J. Kleinberg. Bursty and Hierarchical Structure in Streams. Proc. 8th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining, 2002.


    Data from state of the union addresses here.

  73. To paraphrase Stevenson by xenocide2 · · Score: 2, Insightful

    Out of the six billion people on the planet, only 3 percent can afford one. Of those that can afford one, half decide they actually want one. Combine that half with the lonely few in cyber cafes and markets and you have the world's top spenders in one place, perfect for advertisers.

    --
    I Browse at +4 Flamebait

    Open Source Sysadmin

  74. That's too easy.. by clickety6 · · Score: 1
    I read the heading as:


    Web Log 'Word Bursts' Could Identify New


    and thought it must just go through blogs looking for long rambling outburts about black helicopters, FBI, greys and aluminium beanies. Blimey, that's half the bloggers out there - you don't need a program to identify the crazies!

    --
    ----------------------------------- My Other Sig Is Hilarious -----------------------------------
  75. Isn't that Google News... by dargaud · · Score: 1

    Well, that idea was my entry attempt for the google programming contest, inpired by the Google Zeitgeist which I personally find was too infrequent (and not to say static).

    But finally they've put exactly that system for use in Google news. Keywords that suddenly appear in many news sources get sent to the top of the front page. That's where I learnt of Columbia, a few minutes after it happened, and the first headlines didn't make sense at first.

    So as usual, just search google !

    --
    Non-Linux Penguins ?
  76. Well.. by Mike+Hicks · · Score: 1

    I've noticed that certain words and phrases come and go in the news media, and sometimes other areas. For a while a few years ago, I was seeing the phrase vis-a-vis (I forgot an accent somewhere) all over the place. I even had a history professor at the time who couldn't use the word often enough. But I haven't seen it used for years now.

    One thing I've been noticing recently is `N.B.' I don't really know what it means, but people use it to insert extra comments when writing or updating something.

    1. Re:Well.. by fgb · · Score: 1

      N.B. (Nota Bene) Note Well

  77. Bah... Opera already has this technology! by Xaroth · · Score: 1

    It readily recognizes bursts of "Bork! Bork! Bork!" on important news sites.

    Read more on this technological development!

  78. Re:Set off the ninnle linux craze! by Anonymous Coward · · Score: 0

    Why did this get modded as a troll? Ninnle Linux really is the next big craze! After all, Linus himself has endorsed the kernal! Linux is ALWAYS on topic for /., especially Ninnle!

  79. Does this Benefit Bernnie Shifman? by deeLo57 · · Score: 1

    The last thing we need is to make him 'popular' again, I lost 3 hours of productivity to that piece of spam

  80. How Long? by Bob9113 · · Score: 1

    I wonder how long before this can be done real time enough to really make this useful.

    About 3 weeks after the patent expires.

    * Note: I don't actually know if the guy patented the idea, this is a joke.

    1. Re:How Long? by pkcs11 · · Score: 0

      The NSA has been using this concept using Bayesian Inference for years. Also, Autonomy has been doing this for years as well.

      --
      "I have an odd craving to whisper about those few frightful hours in that ill-rumored and evilly shadowed seaport of dea
  81. Pre-emptive strikes? by McSpew · · Score: 1

    Perhaps this emerging trend early warning system could be used to prevent such tragedies as the chronic overuse of the word "uber."

    The first time I remember seeing "uber" being used was in the days when Microsoft's plan for world domination was described as "Windows uber alles." Since then, it's snowballed and these days, the word has been so overused it's simply become an annoying cliche.

    If only we'd had an early warning system back then, we might have been able to prevent the uber-ification of Slashdot.

    1. Re:Pre-emptive strikes? by Luke-Jr · · Score: 1

      Never heard the word 'uber'. What's it mean?

      --
      Luke-Jr
    2. Re:Pre-emptive strikes? by Anonymous Coward · · Score: 0

      It's German for "super" or "higher" (as in Übermensch--literally superman), and is written über, with the u-umlaut.

  82. Nothing New by pkcs11 · · Score: 0

    The NSA has used this very concept for years to do forensics. Roll back the histogram enough to find the first instance of a topic and chances are, you have found a suspect. For instance, on 9-12 roll back the histogram of gathered intel regarding planes and NYC and you have a few good suspects.

    --
    "I have an odd craving to whisper about those few frightful hours in that ill-rumored and evilly shadowed seaport of dea
  83. Wasn't calculus invented long ago? by knewman · · Score: 1
    This seems to be nothing more than plotting the popularity of certain words or phrases and then taking the first derivative. This is like building a speedometer for your car with a GPS and TI-82.


    Of course this is spun as something new for marketers. I doubt that they will ever find an algorithm for 'cool'. If you are marketing to kids, just hire some kids to do your marketing. duh.

    1. Re:Wasn't calculus invented long ago? by pkcs11 · · Score: 0

      I assume he's using a spin-off of Bayesian Inference or some concoction based on Shannon's Law of Info Theory.

      --
      "I have an odd craving to whisper about those few frightful hours in that ill-rumored and evilly shadowed seaport of dea
  84. kleinbergs annoted course outline a good resource by rawdirt · · Score: 1
    the outline includes a list of -linked- references and links to other research on the topic The Structure of Information Networks.
    Information networks such as the World Wide Web are characterized by the interplay between heterogeneous content and a complex underlying link structure. This course covers recent research on algorithms for analyzing such networks, and models that capture their basic properties. Topics include methods for link analysis, centralized and decentralized search algorithms, probabilistic models for networks, and connections with work in the areas of social networks and citation analysis. The course pre-requisites include background in algorithms and graphs at the level of CS 482, as well as some familiarity with probability and linear algebra. The work for the course will consist of a mixture of reaction papers and a few problem sets, concluding with a project. The coursework is discussed in more detail here.
  85. Onligitory Simpsons Reference by Nathan+Ramella · · Score: 1

    Milhouse: Maybe we should put it on the internet?
    Bart: No! we need to get it to people with opinions that really matter!

    --
    http://www.remix.net/
  86. Consultant-speak by t0ny · · Score: 1
    Wonderful! Now we can see which bloggers are using the most fashionable buzzwords.

    How many people are talking about a 'paradigm shift' this week?

    --

    Manipulate the moderator system! Mod someone as "overrated" today.

  87. "No shit" code is a good thing. by Anonymous Coward · · Score: 0
    the authors managed to find reoccuring references to "Depression" while scanning texts from the 1930's. Imagine that, finding the word depression from a time period thats been nicknamed "The Great Depression"

    In our industry, we have a practice called "testing". This is where we run our code on "test cases" and check that it performs as we expect it to perform. The fact that it does indeed find "Depression" during the 1930's is a good thing; it implies that the code extracts useful markers from unstructured text. We should be unimpressed if it failed to do so, not when it's succeeded.

    "No shit" code is a good thing. Just think about the all-too-common alternative.
  88. Weblogs != Cool. by Anonymous Coward · · Score: 0

    However, blogging is cool, because regular blogging may prevent colon cancer. Frankly, there's nothing I enjoy more in the morning than sitting down with the newspaper and a cigarette, proceeding to take a lengthy blog.

  89. Interesting...but. by callipygian-showsyst · · Score: 1
    Are people who have "blogs" really a good barometer? They are the un-coolest people around.

    Face it, the Internet is dead. The remaining core users are hardly leveragable.

    1. Re:Interesting...but. by Anonymous Coward · · Score: 0
      Wait a minute!

      You're a guy with a 3D Web Cam and you think bloggers are un-cool?

  90. Useful? by mwood · · Score: 1

    Why do you think this would be useful. Enough time is already wasted on the "latest thing". We need some gadget which tends to start people thinking, not to identify what is making them stop thinking.

  91. Economist by bperkins · · Score: 1
    The Economist has an index of sort that has a similar idea to this. Here's an article that describes it.

    It looks for the occurance of the word recession in major newspapers, and it's a pretty good predictor (better than most economists).

    Unfortunately, a lot of the related articles are subscribed content.

  92. I'll just strap my wallet to my chest... by Hubert_Shrump · · Score: 1

    I'll just strap my wallet to my chest with duct tape now

    So that's where all the duct tape went.

    Man, it's hard to finish a good homebrew anything without duct tape.

    --
    Keep your packets off my GNU/Girlfriend!
  93. Too much metablogging going on for this to work by Anonymous Coward · · Score: 0

    Given the current craze for "trackback"/"pingback" type mechanisms, the vast majority of blogs have primarily become metablogs, linking back and forth to each other, blogging on the blog entries of others in a huge ring, creating a misleading and misrepresentative perception of mutual support for lowest common denominator viewpoints.

    Which is to say that the vast majority of blogs have become a lot like /. : Very little new to discover - lots of rallying around existing views which are well-known, not necessarily correct, and often not even objective.

    I suspect a tool such as the one this thread is discussing could do no better than a standard search tool, and might even do worse, artifically inflating its impressions of what's new and hot because of its special knowledge of blog structure and content. A standard search engine, faced with the same web pages, would just come up with a lot of hits. A "craze finding" tool would draw regularly invalid conclusions from the supposed patterns it finds within those hits. Worse yet, its conclusions, if published widely enough, would likely only serve to further inflate certain viewpoints. Again, much like /. itself.

  94. I have been doing this for 3 years by 3trunk · · Score: 1

    My news sites, Infobreakfast and newsQuakes have been using a similar algorithm for the last 3 years to produce a useful summary view of world news.

  95. File this under the "no shit" drawer. by RatBastard · · Score: 1

    And next week he discovers that water is wet, death sucks and Republicans are evil.

    --
    Boobies never hurt anyone. - Sherry Glaser.
  96. Kleinberg by Anonymous Coward · · Score: 0

    Don't forget that Kleinberg was the one who came up with the hubs and authorities algorithm (essentially the Google model) at the same time as Page and Brin.

  97. O_B_ligitory by Nathan+Ramella · · Score: 1

    Before any dictionaries in tennis shoes pipe up.

    --
    http://www.remix.net/
  98. But just think! by jabber01 · · Score: 2, Insightful

    What will future searchers make of Slashdot (and by extension, the net as a whole), what with the waxing and waning in the popularity of Natalie Portman, Hot Grits, Soviet Russia, All your base, gonads and strife, MEEEEEEPT!, and the ever-present FIST PROST.

    This is a significant tool for the post-information age. It could reliable guage the effectiveness of viral marketing. It could also intercept sub-culture developments before they become popular, and introduce them to the general population in association with a corporate brand.

    Imagine if Nike or Pepsi, or *shudder* Microsoft, had caught the "All Your Base" thing on the upswing. They'd have a better slogan than the top down "Dude, you're gettin a Dell".

    --

    The REAL jabber has the user id: 13196
    What you do today will cost you a day of your life

    1. Re:But just think! by Sdrawcab · · Score: 1

      Man, you need to get a job in marketing, you have some great ideas!

    2. Re:But just think! by fognugen · · Score: 1

      Can someone tell me the origin of 'Hot Grits'? Seriously, I've always enjoyed Slashdot zeitgeist\fads and have seen 'Hot Grits' endure but can't place the origin.

      Thanks....

  99. Most popular topic : why nerds are unpopular !!!! by OneInEveryCrowd · · Score: 1

    The last time I checked blogdex the most popular topics by far were blogs themselves. I was kind of surprised to see that bloggers are now wondering whether people think they are nerds or not.

    Actually if you filter stuff like the above out blogdex really does look like its telling you what people are thinking about these days.

  100. The death of Fads by tchdab1 · · Score: 1

    It seems inevitable that as information technology, anthropology, marketing and research technologies (and everything else) are brought to bear on the tasks of predicting, identifying, and capitalizing on an emerging fad ASAP, this will inspire the creative forces that generate new culture to avoid generating it. Creative forces (artists, "the hip", those who do this kind of stuff) want to be differently expressive. If ideas are co-opted for mass exposure and profit as soon as they begin to emerge, those ideas will stop emerging. Those creative forces will inevitably learn to generate anti-fads (new, different, difficult to co-opt in the current culture), whatever that turns out to be.

    The requirement to exploit emerging creative difference will change those fads to something else.

    Let's figure out what it will be and sell it!

  101. Reminds me of Asimov's Foundation by AntonyBartlett · · Score: 3, Insightful

    once poeple know the rules that determine what a "word burst" is and when it's happening, then tools will be developed to artificially inflate desired word burts

    The Three Theorems of Psychohistorical Quantitivity:

    1. The population under scrutiny is oblivious to the existence of the science of Psychohistory.

    2. The time periods dealt with are in the region of 3 generations.

    3. The population must be in the billions (±75 billions) for a statistical probability to have a psychohistorical validity.

    1. Re:Reminds me of Asimov's Foundation by Anonymous Coward · · Score: 0

      By 'billions' you mean 1,000,000's (the British reverse the usage of million/billion) right? Because to us US types, billion is 1,000,000,000 and there aren't 75 billion people within any 3 generations I know of... :)

    2. Re:Reminds me of Asimov's Foundation by AntonyBartlett · · Score: 1

      I think we *are* talking thousands of millions here.
      Psychohistory is a fictional science employed upon a galatic population.

  102. FBI has been doing this for years now... by Hubert_Shrump · · Score: 1

    Carnivore was so they'd have the edge on being cool.

    As you can see, even with advanced technology and a huge corpus of email and search requests, coolness is all about the mirrored shades and gold embroidery.

    --
    Keep your packets off my GNU/Girlfriend!
    1. Re:FBI has been doing this for years now... by Anonymous Coward · · Score: 0

      Indeed. I once evaluated a piece of software from Los Alamos which was set up to look for correlations in micro-array data. Millions of points of data which vary up or down varying on gene expression levels, and you are looking for sets of genes which having matching expressions levels. The rumour was the the software originated from the security services who used it to scan emails for trigger words. This software then provided you with a nice GUI which showed you what "chatter" was about.

  103. this is already being done by Anonymous Coward · · Score: 0

    This is already being done:

    http://www.daypop.com
    http://www.popdex.com
    ht tp://blogdex.media.mit.edu/
    http://groups.google. com

    And we all already know what it can be used for:

    http://www.pbs.org/wgbh/pages/frontline/shows/co ol /

  104. Britiah Savages! by supersnail · · Score: 1

    Damn we been sussed!

    --
    Old COBOL programmers never die. They just code in C.
  105. blogspamming by Anonymous Coward · · Score: 0

    I wonder how long before 5000 or so scientology blogs spontaneously appear overnight at and we discover that the latest craze is Dianetics, the Modern Science of Mental Health(tm)

  106. Buzzword bingo by Julian+Morrison · · Score: 1

    Doesn't Dilbert already have a copyright on that one?

  107. Waypath Project by Anonymous Coward · · Score: 0

    Hello???

    Haven't you /. nerds ever heard of the Waypath Project?

  108. culture jamming by Parsec · · Score: 2, Funny

    What a great opportunity for culture jamming! We just need a few thousand webloggers to start using weird words designed to repel "normal" people.

    Obviously this could backfire and we could actually start a real trend. So, I propose that the first words we need to put out are ( geek || nerd ) && sexy. (And if you understood that, you must be hot stuff.) I'm willing to take this risk if you are.

  109. Bigger question... by UncleGizmo · · Score: 1

    I wonder how long this would be able to be measured accurately before marketers begin building blogs that tout a 'new' fad, to get noticed? Perhaps it'll become just another tool of viral marketing for the big folks?

    --
    Who put this thing together? Me, that's who.
  110. infobreakfast by atabotix · · Score: 1

    See http://www.infobreakfast.com/ which does something similar for news items - and has a very terse, wireless-friendly, interface.

    --
    Problems worthy of attack prove their worth by hitting back.
  111. How about non-word bursts? by nani+popoki · · Score: 1

    Lindon Johnson "invented" NEW-QUEUE-LAR weapons. And when was the last time any politician pronounced it NEW-KLEE-ARE? (Maybe that's not such a bad thing -- now you can tell the politicans from the physicists... no wait: you always could -- because the politicians made sense.)

  112. Because being Cool by blair1q · · Score: 1

    ...is all about bragging about your method of searching online blogs... ...cribbing memes... ...and knowing where to find all of the State of the Union addresses since 1790...

    --Blair
    "AAAAaaaaaaaayyyyyyyyyyyyyy..."

  113. Kinda sad... by SmlFreshwaterBuffalo · · Score: 1

    Only a true geek would need a search engine to determine what is considered "cool".

  114. timeliness by john_smith_45678 · · Score: 0

    Geez, this story was written Feb. 3 and was posted on /. Feb. 19??

  115. I Already Have This Software by serutan · · Score: 1

    I just tested it on feeds from the major news media. The highest bursti-ness readings are on the words "Bush", "Iraq" and "terrorism." The software package I'm using was developed by a team of millions of independent non-programmers, and is called "Duh!".

  116. Like so many before you, you have failed.

    I would like to remind you that second place is FIRST FAILURE. YOU HAVE LIKEWISE FAILED IT!

    --
    One god, one market, one truth, one consumer.
  117. This problem has been already studied in detail by egork · · Score: 1

    A prominent russian mathematician A.T. Fomenko from Moscow State University has already proposed and applied the method in question. His goal was to reveal hidden relations in the global history. The "citation index" of certain words would signal some important event like a battle or a outstanding personality. By connecting this words and the dates assigned to them in the ancient texts one could find new relations or use it as experimental evidence for historical theories.

    There are couple books of him even in English that are a great fun to read even for non-scientists. Check an abriged version of one of them.

    Here is the title of his fundamental work published:
    Empirico-Statistical Methods of the Analysis of the Narrative Numerical Material & the Applications of the Problem of Dating
    ISBN: 0792326059 - Hardcover - List Price: $266.50
    Publisher: Kluwer Academic Publishers - Published Date: 01/01/1994
    Author: A. T. Fomenko

  118. Duh? by torpor · · Score: 1

    If they buy Blogger, they can integrate the sorting/statistical analysis *into* the Blogger network, put it on a Google interface ... connect the nets, and oila:

    The answer to the question of 'when will it be realtime?' is moot.

    --
    ; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
  119. Errmm... dude ... by torpor · · Score: 1

    ... it's called the National Debt.

    If you don't keep consuming, and don't keep spending money (and thus moving material through the American universe), your government gets into trouble with the people it owes money to.

    --
    ; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
  120. Megatrends: Done that (20 years ago) by Exoman · · Score: 1
    (Just in case anyone's looking for prior art to head off this from patenting.)

    John Naisbitt did this back in 1982, scouring word frequency in various periodicals to come up with Megatrends.

    It's a sensible idea, and really can be forward looking, in a sense, as trends begin by deed, but spread by word.

  121. The first and last day this runs by gad_zuki! · · Score: 1

    Kottke.org something something.. COOL!!!!!!!
    theonion.com something something FUNNY!!!!!!
    Which Garbage Pail Jr Kid are U?! TRY IT!! I'M SMELLY MCGEE!!!!
    I'm gonna move out at 18!!! MOM SUCKS!!!!
    New Homestar Runner this week. SO COOL!!!!
    Dog bites man ;-P HAHA IRONY!!!
    IE update out!! AWESOME PRIVACY CONTROL DUDEZ
    My mom totally hates my hair!!! SUCH A BITCH!!!
    Wow you can block pop-unders!! KEWL ITS FREE!!
    Flash RULEZ!! This movie is wack ya'all!!!
    Buffy something something!!! !!!!!!
    Man bits dog back!! SUPER IRONY!!

    etc

  122. bin laden by floydman · · Score: 1

    so i guess there are times when saddam hussien and bin laden are cool...!!!!!
    wish i was cool too.....

    anonymous freeze

    --
    The lunatic is in my head
  123. Blogolalia by handy_vandal · · Score: 1

    "Word bursts" is too boring a term, moreover it's not very academic. I suggest "blogolalia" instead.

    --
    -kgj
  124. Let's see it in action by updog · · Score: 1

    The algorithms used to identify these sudden bursts are relatively simple, but very powerful, says Christos Papadimitriou, at the University of California at Berkeley. OK, show us! Why all the talk and no examples? If these simple algorithms exists, why doesn't the article give us a site that actually uses these algorithms, so we can see what's popular today for ourselves?

  125. where is an example? by updog · · Score: 1
    The algorithms used to identify these sudden bursts are relatively simple, but very powerful, says Christos Papadimitriou, at the University of California at Berkeley.

    OK, show us! Why all the talk and no examples?

    If these simple algorithms exists, why doesn't the article give us a site that actually uses these algorithms, so we can see what's popular today for ourselves?

  126. disturbing by Anonymous Coward · · Score: 0

    this show had a very disturbing quote.
    something to the effect of "teens today are like africa, they're just waiting to be colonized"

    wrong on so many levels...

  127. I don't know... by pluther · · Score: 1

    It (tickle-me elmo) might (tickle-me elmo) show (tickle-me elmo) certain (tickle-me elmo) old (tickle-me elmo) toys (tickle-me elmo) are (tickle-me elmo) making (tickle-me elmo) a (tickle-me elmo) major (tickle-me elmo) comeback (tickle-me elmo) when (tickle-me elmo) really (tickle-me elmo) someone (tickle-me elmo) is (tickle-me elmo) cheating.

    --
    If the masses can keep you down, you're not the Ubermensch.
  128. Hot Grits by jabber01 · · Score: 1

    The ultimate origin is lost in antiquity, but it has to do with an old troll fixation with Natalie Portman, whom the trolls wanted to see naked and petrified, so they could pour hot grits down her pants. Or possibly, they wanted to get her naked and petrified by pouring hot grits down her pants. At this point, I'm just going to say "hot grits" several more times, to create a local hot spot around the "hot grits" phrase, in case anyone ever does an analysis of phrase usage frequency in terms of "hot grits" in this thread - just to keep this comment on topic for both hot spot analysis and "hot grits".

    --

    The REAL jabber has the user id: 13196
    What you do today will cost you a day of your life

  129. Re:Jesus Saves!... by Anonymous Coward · · Score: 0

    ...which shows he's got to be on a tax fiddle

  130. Is that the Great White's newest hit? by TheMidget · · Score: 1

    Sorry, couldn't resist...