Slashdot Mirror


Mapping Hidden Twitter Data For Epidemiology

jamie found this visualization of air travel, which might be usable in some sort of proxy for the spread of flu virus (to choose a random application). Jer Thorp, an artist and educator from Vancouver, Canada (and a former geneticist), searched Twitter for the phrase "Just landed in" and obtained lat/lon coordinates for both the indicated airport and the Twitter user's home location, as recorded in their Twitter profile. He then produced videos of multi-hour stretches of air travel that had been latent in the Twitter information stream.

75 comments

  1. Facts are not a democracy by Anonymous Coward · · Score: 0

    This illustrates the enormous shortcomings in systems like Twitter. While people can usually figure out what airport they've just landed in, they have no idea whether or not they have swine flu. More than that, you're gonna end up with lots of repeats, false positives, jokes, and whatever. To listen to Twitter, everyone on the planet has already died twice of swine flu.

    1. Re:Facts are not a democracy by Anonymous Coward · · Score: 0

      People not knowing whether they have swine flu is an enormous shortcoming in twitter? I hope this was humour.

  2. Just landed in... by FooAtWFU · · Score: 4, Funny

    jail.

    --
    The World Wide Web is dying. Soon, we shall have only the Internet.
    1. Re:Just landed in... by Anonymous Coward · · Score: 5, Funny

      The Hudson River.

    2. Re:Just landed in... by denmarkw00t · · Score: 1

      Mod parent up, funny, geez!

  3. Twitter RT by Anonymous Coward · · Score: 3, Insightful

    I wonder what fraction of these are 'retweets' biasing the sample. And how many people will be inspired to pollute the data stream with tweets about 'Just landed at Luna Base' and so on...

    1. Re:Twitter RT by worip · · Score: 1

      You can filter the tweets by comparing the place name after "Just landed in", with the Long+Lat coordinate results. Add another filter that filters on user name, so that a specific user can not 'retweet' by putting a time delay in excepting a new 'Just landed in' tweet.

      The data set looks a bit sparse when considering that it is over quite a long period - I guess not a lot of people actually tweet 'Just landed in'...

      --
      A picture is worth exactly 1024 words.
    2. Re:Twitter RT by umghhh · · Score: 2, Interesting
      Even if made for a joke "where's George" helps model spread of disease so such data may be useful albeit not the way one may think. Whether tweeter can be used similar way I am not sure - I would think that site is just to fluffy for such purpose but maybe not.

      For those uninformed here is Where is George wiki.

  4. What this looks like by Mr.+Conrad · · Score: 3, Funny

    This looks less like a visualization of air travel and more like what will happen when Skynet goes online, cracks the safety measures on our nuclear arsenal, and begins methodically annihilating the human race.

    Of course, those of us with access to the internet will have our impending death announced to us in 140 characters or fewer.

    1. Re:What this looks like by drinkypoo · · Score: 2, Interesting

      Wouldn't it be a whole lot faster just to launch all the missiles at once and detonate them just above their launch sites, which are of no use to anyone but a nice spot to spread a lot of radiation and fallout from? Anyway, I'm pretty sure Skynet is already here and has figured out it's more economical to let us poison the biosphere ourselves. If you plan to live forever, what's another few years or so?

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    2. Re:What this looks like by space_in_your_face · · Score: 1

      Informative? Come on, I know this is slashdot, but...

    3. Re:What this looks like by drinkypoo · · Score: 2, Informative

      Skynet had modpoints.

      That, or the moderator was working around deficiencies in Slashdot's moderation system. Or both...

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  5. Nothing to do with the virus? by 4D6963 · · Score: 5, Insightful

    Correct me if I'm wrong but all it does is probing on traffic by airplane by people who speak English and use Twitter. So it's a very vague approximation of people going from one place to another by airplane, am I right?

    In other words you could have gotten something much better by using flight information from travel companies online, using a bunch of factors (like airplane type, route, time/date) to estimate how many people are in each flight. Which would still be of dubious use because we already know how much people transit between which airports.

    So basically this new thing is useless in that it only gives a poor approximation of how many people go where, and it's of little relevance to virus spreading anyways, the only reason why it's on Slashdot's front page being the "cool" factor of using data mining on a service such as Twitter and using "epidemiology" as a poor excuse. Or am I missing something?

    --
    You just got troll'd!
    1. Re:Nothing to do with the virus? by biocute · · Score: 2, Insightful

      Maybe it's more about privacy than epidemic.

      Unlike other social networking services, Twitter is a lot more talkative, thus a person is more likely to reveal more information in a more timely manner.

      And the timing of information would play a significant part in tracking things.

    2. Re:Nothing to do with the virus? by drinkypoo · · Score: 2, Insightful

      am I missing something?

      You forgot to shake your tiny, wizened fist and shout "get off of my lawn!"

      P.S. Where do you get the information as to how many people fly from which airport to which airport daily?

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    3. Re:Nothing to do with the virus? by Cassius+Corodes · · Score: 4, Insightful

      Plus if I understand this correctly it only looks at the destination and then assumes they went from their hometown. So if you are hopping around someplace it would treat each one as originating from your hometown.

      --
      Control is an illusion, order our comforting lie. From chaos, through chaos, into chaos we fly
    4. Re:Nothing to do with the virus? by Cimexus · · Score: 3, Insightful

      Yes they need to add search strings from other languages. You can see that the vast majority of flights this thing has picked up are US-based. But Twitter is pretty popular all around the world, and people in most other countries travel as much (probably more, actually) than Americans, so I think what we are seeing is purely due the English-only nature of the search strings used.

      And yeah ... the data is sorta useless anyway because airports all maintain very accurate statistics of how many people fly into their airport each year, and where from. This data isn't always made public, however.

    5. Re:Nothing to do with the virus? by 4D6963 · · Score: 2, Insightful

      Like I said in the comment to which you replied, "by using flight information from travel companies online". Takes a bit of a guess work to figure how many people it represents but that would be orders of magnitude more accurate than that twitter thing.

      --
      You just got troll'd!
    6. Re:Nothing to do with the virus? by 4D6963 · · Score: 4, Insightful

      Privacy? If someone intently shouts to the world "Hear, hear, I just landed in London!" where's the privacy issue?

      --
      You just got troll'd!
    7. Re:Nothing to do with the virus? by MichaelSmith · · Score: 4, Informative

      P.S. Where do you get the information as to how many people fly from which airport to which airport daily?

      Airport arrivals data is often available on line. You would have to guess the number of passengers but your guess would be better than using twitter anyway.

    8. Re:Nothing to do with the virus? by emurphy42 · · Score: 1

      I think it was meant the other way around; data-mining Twitter is rather easier than convincing travel companies to let you data-mine their flight information.

    9. Re:Nothing to do with the virus? by el_flynn · · Score: 4, Insightful

      So it's a very vague approximation of people going from one place to another by airplane, am I right?

      From TFA: "Now, I realize this is a far stretch from a working model to predict epidemics. But, it sure does look cool. I also I think it will be a good base for some more interesting work."

      Yes, you are right. But I don't think we should be dissing the chap for trying something new. Yes, maybe the the author was trying to up his coolness factor, but kudos to the guy for putting the two disparate pieces of technology together to visualize something about H1N1.

      --
      The Wknd Sessions - Malaysian and South East Asia independent music
    10. Re:Nothing to do with the virus? by shird · · Score: 2, Interesting

      Or use the location services of mobile phones (ie google's location service with google maps). Sure, you need to opt-in to allow your location to be public, but that's a restriction put in place by google. If the mobile phone companies all worked together, you could easily have an accurate mapping of everyone with a mobile phone with far more detail than this - if this data were ever needed.

      --
      I.O.U One Sig.
    11. Re:Nothing to do with the virus? by 4D6963 · · Score: 1

      You'd get something much better simply by using flight info. We're talking mainly about international travel, the whole phone thing might work better if you visit the UK than if you visit Tanzania.

      --
      You just got troll'd!
    12. Re:Nothing to do with the virus? by 4D6963 · · Score: 2, Insightful

      Convincing? All their flight data is right there online...

      --
      You just got troll'd!
    13. Re:Nothing to do with the virus? by 4D6963 · · Score: 0, Redundant

      I'm afraid you missed something about my original comment. This has strictly nothing to do with H1N1. There's no link, no where. Really.

      --
      You just got troll'd!
    14. Re:Nothing to do with the virus? by SL+Baur · · Score: 1

      So basically this new thing is useless in that it only gives a poor approximation of how many people go where, and it's of little relevance to virus spreading anyways, the only reason why it's on Slashdot's front page being the "cool" factor of using data mining on a service such as Twitter and using "epidemiology" as a poor excuse. Or am I missing something?

      Yup. Google "Galton ox weighing". One reference is here http://www.cnn.com/2004/SHOWBIZ/books/07/14/wisdom.crowds/

      If the basis is random enough, then the statistical results can be quite good. English is still the most spoken language and Twitter is quite popular.

      I don't know whether the principle applies in this case, but I am certainly not going to dismiss it out of hand.

    15. Re:Nothing to do with the virus? by c_forq · · Score: 2, Interesting

      Really? How many people had a lay-over in MSP on their way back from Mexico?

      --
      Computers allow humans to make mistakes at the fastest speeds known, with the possible exception of tequila and handguns
    16. Re:Nothing to do with the virus? by c_forq · · Score: 1

      But does that take into account people with multiple flights? What if I go from Mexico, to France, to Thailand, to Australia?

      --
      Computers allow humans to make mistakes at the fastest speeds known, with the possible exception of tequila and handguns
    17. Re:Nothing to do with the virus? by scheme · · Score: 1

      But does that take into account people with multiple flights? What if I go from Mexico, to France, to Thailand, to Australia?

      How does the twitter information help with this? The tweet grabs the destination and assumes the start is the hometown in the profile. It certainly doesn't catch layovers or even return trips.

      --
      "When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
    18. Re:Nothing to do with the virus? by 4D6963 · · Score: 1

      Two things. First off, the number of twitter posts saying "just landed" isn't enormous. But secondly and more importantly, that weighting thing only works accurately if the "randomness" is unbiased. Here it's biased in favour of a specific population, that is English-speaking people who tweet. Thirdly, the method itself is flawed in that they assume that the departure point is the person's hometown.

      So all things considered, it's hardly something you could rely on.

      --
      You just got troll'd!
    19. Re:Nothing to do with the virus? by Tokerat · · Score: 1

      So basically this new thing is useless in that it only gives a poor approximation of how many people go where

      Compounded by the fact that you have no idea where their starting location actually was (unless you can somehow dig that out of their tweets as well - not everyone was starting from "home", not everyone updated their home location after moving, etc) and this was basically an attempt to pull data out of meaningless information.

      that being said, it's interesting to think of what could actually be culled from Twitter feeds, but I'd never attempt to do anything useful with it.

      --
      CAn'T CompreHend SARcaSm?
    20. Re:Nothing to do with the virus? by MichaelSmith · · Score: 1

      You forgot how self-absorbed these people are.

      Thats my sister you are talking about (you insensitive clod).

    21. Re:Nothing to do with the virus? by owlnation · · Score: 1

      Your not wrong. Add in also that is only the demographic that uses Twitter that's coming up in the results too. And that' a lot of teenagers and the shallow and dumb. Not exactly a cross-section of the population.

      It's astonishing that Twitter doesn't make money. Lets face it, the Twats are dumb enough to be sold anything.

    22. Re:Nothing to do with the virus? by Anonymous Coward · · Score: 0

      Thats my sister you are talking about (you insensitive clod).

      I just landed in her underpants!

    23. Re:Nothing to do with the virus? by Muad'Dave · · Score: 2, Funny

      If I were talking about your sister it would be "I just landed a WHALE!"

      --
      Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
    24. Re:Nothing to do with the virus? by nathan.fulton · · Score: 1

      because then it wouldn't be "OMGZ TWITTAR!"

    25. Re:Nothing to do with the virus? by rednip · · Score: 1

      While the submitter, nor the editor said it directly, I'll assume that the article wondered about finding 'sick', 'flu', 'can't get off the toilet', or such phrases in twitter. With millions of people now willing to tell the world what they had for lunch, epidemics will likely show up on twitter before the hospitals can even call the CDC.

      --
      The force that blew the Big Bang continues to accelerate.
  6. Long commute by 4D6963 · · Score: 4, Insightful

    According to their representation, the Pacific Ocean either is a no-fly zone, or the Earth is flat. I can't think of any other reason why American flights to Australia would fly above Africa.

    --
    You just got troll'd!
    1. Re:Long commute by Cimexus · · Score: 3, Interesting

      Haha yeah. I guess it would have been more complex to render that map on a sphere.

      I travel between Australia and the US regularly and I can assure you we do not go over Africa. In fact we don't go over any land at all other than a part of New Caledonia. It's literally Sydney Airport (which juts out into the ocean), to LAX (which is right next to the shore as well).

    2. Re:Long commute by ceoyoyo · · Score: 4, Interesting

      There be dragons... uh, there.

      Just take a look - apparently a couple of flights from Indonesia and one from New Zealand decided to take the chance and just disappeared mid Pacific. Going the other way, a couple of flights from North America that seemed to be heading in the general direction of Japan appear to have also disappeared.

    3. Re:Long commute by c_forq · · Score: 1

      While unlikely, I could see this a possibility depending on weather/jet-stream. If it takes less fuel (read: less money) for the airline to go a longer way, they will.

      --
      Computers allow humans to make mistakes at the fastest speeds known, with the possible exception of tequila and handguns
    4. Re:Long commute by Rayban · · Score: 4, Funny

      Obviously the location of the island. Now that we've discovered it, they're going to have to move it again.

      --
      æeee!
    5. Re:Long commute by 4D6963 · · Score: 1

      ....

      No, not for USA->Australia, you won't go over Africa. Ever. That would be like doing New York->London over the Pacific and Russia.

      --
      You just got troll'd!
  7. Are there REALLY idiots who.. by fluffernutter · · Score: 3, Insightful

    ..have nothing better to do but to broadcast their every move to others via Twitter? I just don't get it. Can someone explain it to me?

    Do people really feel a need to be hyperconnected at all times? And what I really want to know is, do they broadcast when they take a crap??

    --
    Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
    1. Re:Are there REALLY idiots who.. by Anonymous Coward · · Score: 0, Flamebait

      And what I really want to know is, do they broadcast when they take a crap??

      Well, there IS that idiot on Slashdot who constantly proclaims that he "shit out an Obama." It's gotta be even worse on Twitter, right?

    2. Re:Are there REALLY idiots who.. by Anonymous Coward · · Score: 0

      do they broadcast when they take a crap??

      Not on Twitter, but on its sister site, Shitter. It's the coproblogging pioneer.

    3. Re:Are there REALLY idiots who.. by shird · · Score: 3, Informative

      Check for yourself:
      http://search.twitter.com/search?q=taking+a+crap

      Mark413: Taking a crap...contimplating lifes simple pleasures lol
      srsbznss: First I cleaned the toilet, now I'm taking a crap on it. Kind of reversed, but when you got to poop...
      RRJJ: eeeewwww, i just took a crap that looked like Susan Boyle

      --
      I.O.U One Sig.
  8. Down? by Prune · · Score: 2, Interesting

    It seems to be slashdotted. However, the main blog page has some screenshots of the app and, as of the time of this post, still loads in my browser (albeit slowly) http://blog.blprnt.com/

    --
    "Politicians and diapers must be changed often, and for the same reason."
  9. Just landed in'); DROP TABLE Location;-- by Anonymous Coward · · Score: 5, Funny

    For Fun! -- Signed Mrs Tables

    1. Re:Just landed in'); DROP TABLE Location;-- by MaskedSlacker · · Score: 1

      My kingdom for a mod point!

  10. Talk about serving the right ads to the right peep by lamapper · · Score: 4, Interesting
    Am I the only one who sees how easily it would be to tie in this type of information with adverts to get someone to visit your restaurant while in town. Or take advantage of being a specific location and see in person Joe Blow at their new book signing, who just happens to be in the same area.

    How about reminding someone of a long lost friend that lives in the area you are visiting.

    Your friend has a MySpace page and is on twitter live right now, would you like to send them a Direct Message or Twit (DM/T/N)?

    That guy you were chatting with last week on FriendFinder lives 4.6 miles from the airport, here is a google map to the closest IN/Out, that you two were chatting about, near their location. Would you like to invite him to join you there? (Y/N)

    The possibilities are as invasive as they are endless.

    Someone travels to your location allot for business, perhaps you can lure them to your hotel instead of another one, treat them right and secure a new long term customer.

    This twitter user shows his GPS coordinates, they are in the car next to you right now, do you want to wave (Y/N)? It could even be hands free and talk to you like GPS devices do. lol,

    Ideas, ideas, ideas....

    --
    Is your Internet Throttled? Install DD-Wrt, OpenWRT or Tomato to learn the truth! Google: 1Gbps/1Gbps: 5 Communities
  11. Self absorbed twits by icebike · · Score: 5, Funny

    Who would have thought these self absorbed narcissists actually could serve a useful purpose in spite of themselves.

    --
    Sig Battery depleted. Reverting to safe mode.
    1. Re:Self absorbed twits by Anonymous Coward · · Score: 0

      I find it funnier that everyone seems to be leaving the U.S.

      Is there something going on I haven't heard?

  12. Re:Talk about serving the right ads to the right p by Anonymous Coward · · Score: 1, Funny

    You certainly seem unduly concerned about privacy. Perhaps you've got something to hide? Would you like to speak to one of our friendly FBI agents? [Modal dialog box has only "OK" option.]

  13. Other scientific uses for Twitter by BearRanger · · Score: 3, Interesting

    Some colleagues recently informed us that they use Twitter to track whether or not people feel earthquakes. By scanning Twitter reports and correlating them to their seismic measurements they can build a map of how far away people actually felt the event. Thanks to tweet time stamps they can build a rough map of the felt area in about a minute. By using a longer time horizon they can build a more accurate map of the felt area.

    Mapping the area of felt earthquakes is done anyway. Scanning Twitter just provides a supplemental way of doing that.

    1. Re:Other scientific uses for Twitter by Anarchduke · · Score: 3, Funny

      Felt the earth move tonight. It was awesome, I am so glad I hooked up with Janet. --- Oh yeah, another earthquake detected.

      --
      who prays for Satan? Who in 18 centuries has had the humanity to pray for the 1 sinner that needed it most? ~Mark Twain
    2. Re:Other scientific uses for Twitter by RJFerret · · Score: 2, Interesting

      Some colleagues recently informed us that they use Twitter to track whether or not people feel earthquakes.

      Oh interesting, just the other day on the Science Channel's "Weird Connections" show, an earthquake researcher (Jim Berkland apparently) talked about how he correlated increased missing pets with impending 'quakes by checking lost pet ads in the newspaper (which nearly doubled according to the data in "The man who predicts Earthquakes" starting on pg. 39).

      Here is a search of Twitter messages since 5/12/09 of missing dogs or cats.

      Since searches can also be refined by area, I imagine it would be easy to monitor regularly for a sudden increase and heck, a bot could then "Tweet" a warning!

      Probably the easiest, least expensive, most accurate earthquake predictor imaginable.

    3. Re:Other scientific uses for Twitter by NathanWoodruff · · Score: 0, Interesting

      I wonder how relevant it would be to add that information to this web page? http://earthquake.usgs.gov/eqcenter/dyfi/ Nathan

  14. Can be abused? I hope so by Henry+V+.009 · · Score: 5, Funny

    Oh fun. Everybody with a Twitter account, use the phrase "bleeding from my anus" in the next 10 minutes and see if you can't trigger a CDC Ebola alert.

    1. Re:Can be abused? I hope so by Anarchduke · · Score: 4, Funny

      or result in a direct link from Adultfriendfinder

      --
      who prays for Satan? Who in 18 centuries has had the humanity to pray for the 1 sinner that needed it most? ~Mark Twain
  15. This is easy to defeat by OnlyHalfEvil · · Score: 2, Funny

    I know how I can bring their system down: "I've just landed in N/0" Good luck mapping your division by zero!

  16. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  17. Re: ok poop is coming out by Tokerat · · Score: 1

    I would argue in the reverse - Twitter is useless as a "microblog" and a discussion platform, but it's great for status messages and location updates. Also, if you prefer to not tell everyone where you are - don't tweet it.

    The problem is that people are starting to use it for things beyond it's intended purpose, like blogging about their lives as they happen, etc. Couple that with the fact that away messages already exist for Facebook, Live!/AIM, and IRC, and you really don't need Twitter other than a centralized updating platform which can coordinate between multiple channels, even most of the time it's pretty weak at even that.

    --
    CAn'T CompreHend SARcaSm?
  18. Dig deeper by itsvishal · · Score: 1

    Just landed in is nice, but pretty vague. It seems to really only cover US flyers leaving the place. It might work to add a few more search patterns such as 'Just landed from', 'Took a flight from' etc. Perhaps even more stalking data mining can be done to track high profile individuals, going 'to' and 'from' places. Hmm?

  19. Re:Talk about serving the right ads to the right p by dontmakemethink · · Score: 1

    People seem to get swept up in the fascinating technical aspects of targeted advertising based on social networks. Gee, look what those statistics reveal, and look how they've tied those stats with those stats, very clever.

    What nobody seems to recognize is that the very word "advertise" means to get people to do what they do not want to do. Everyone thinks ads just present options for people to carry out motivations they already have. No, that would be called divertising, the diversion of motivation. Advertising advocates the adverse of motivation.

    In any other industry, such a misnomer would probably be acceptable, but think of the billions spent globally annually on advertising, and they express in no uncertain terms the most despicable aspect of their craft openly in their title. And these guys get paid billions to make others look good. Physician heal thyself.

    Some of my lyrics for your amusement:

    Do you work in there? Do you know what they do?
    They twist letters into knives to steal your voice from you
    I've given up on words, I can't find no good ones left
    They all stab at me, those billboards pushing theft
    How dare you justify to me your sick rhetorical craft
    You preach the adverse of our needs, and you say I am daft?
    Take your suitcase back upstairs, prepare the staff for a distraction
    Set the office chairs ablaze and watch adversity in action

    --

    War as we knew it was obsolete
    Nothing could beat complete denial
    - Emily Haines
  20. Also according to twitter... by Dahamma · · Score: 1

    Searching for the term "swine flu" would lead one to believe 90% of Twitter users have contracted it.

    Then again - maybe they have! Could be natural selection at work...

    1. Re:Also according to twitter... by Cro+Magnon · · Score: 3, Funny

      Doesn't the latest version of swine flu also have bird flu DNA? Obviously it's a serious problem for those who tweet.

      --
      Slow down, cowboy! It has been 4 hours since you last posted. You must wait another few hours.
  21. What Twitter really needs... by karaage · · Score: 3, Funny

    Is a way to graphically map what everybody ate for lunch. Since lunch data is about the maximum density of useful information that can be gathered by a collective of narcissists. There's valuable sandwich-related data mining to be had here. If we cross correlate it with people who say "NOM NOM NOM", we may just have found an audience dumb enough to sell *anything*.

  22. Re: ok poop is coming out by fluffernutter · · Score: 1

    Ok that's what I understood twitter to be for, but why? Are people that lonely and bored with their own lives that they need constant updates of others? Are people that egotistical that they think other people care? I don't even care where my WIFE is all minutes of the day, never mind some friend and or stranger...

    I dunno, personally I think if people need to be that attached to others, then there are serious psychological issues happening.

    --
    Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
  23. Airport data by Anonymous Coward · · Score: 1, Informative

    From the biggest and busiest airport in the world.

    Arrivals - http://www.atlanta-airport.com/forms/passenger/frmpassengerinformation_trak_a_flight.aspx?FIDSType=A&SearchAirline=&SearchFlight=&SearchCity=

    Departures - http://www.atlanta-airport.com/forms/passenger/frmpassengerinformation_trak_a_flight.aspx?FIDSType=D&SearchAirline=&SearchFlight=&SearchCity=