Slashdot Mirror
Mapping Google News
CousinLarry writes "A neat project called Buzztracker.org has been mining Google News for over a year and keeping track of relationships between geographic locations mentioned in articles.
The results are some really cool maps that actually seem to reflect the "buzz" of the day - check out the Vatican clusters from earlier this month, or the global New Year's chatter. You can also dig down into the articles from which the maps were generated."
Where is Slashdot on the map?
SEMANTIC WEB!
Does that map remind anyone of the old game called Empire, or is it just me?
Makes me want to load it up again.. any modern implementations of it around?
This is by far one of the most interesting uses of data-mining I've seen in while. Neat to see what are the hotspots, as far as news goes, in the world.
The guys at Buzztracker desrve a cookie (edible variety).
You can't defeat physics.
..no, literally. its made up of old news..
Starsucks
Communist conspiracy?
In America, you spam computers In Soviet Russia, computers spam you!
Well when you think about it aren't those the exact places you'd expect to be hotspots?
What a cool site, and it works very quickly and is not overflowing with advertising crap?
Acting stupid isn't much fun when there's someone around who knows better
I should start a website, beertracker.org, to keep track of my daily buzz.
http://perljam.net/notes/interesting-google-satell ite-maps/
-ted
Impressive. Can't wait to see it when they add baby rollers, heavy diggers, and funky bombs.
It is a solemn thought: dead, the noblest man's meat is inferior to pork.
I've noticed an upsurge in "Living Willing" spam since the Terry Schiavo story and even a few Pope-related offers.
Two wrongs don't make a right, but three lefts do.
It looks like the code needs a bit more tuning. http://www.buzztracker.org/index.html lists Nelson, NZ, as one of the hot spots. Clicking on that lists a bunch of articles about apartheid. I think the site code misinterpreted a reference to Nelson Mandela in one of the articles.
I remember about a year ago or so, there was a guy who was mining google news to produce an RSS feed. IIRC, google politely demanded that individual stop offering this to people. I can't find the article to cite this, maybe someone can help? At any rate, I wonder how google will feel about this.
1. Map out the world in x and y coordinates.
2. Feed google buzz data into huge neural network.
3. Predict location and magnitude of future events.
4. ???
5. Profit!
Apparently they didn't Google their own name, or else they would've noticed the name was already in use for a fairly popular music composition program.
Bears don't normally eat things that talk and move backwards.
Here's a website with both txt and pdf of the order to pull my app that parsed google news:
http://homepage.mac.com/fahrenba/gn/gn.html/
James Tiberius Kirk: "Spock, the women on your planet are logical. No other planet in the galaxy can make that claim."
New Google mappings
Goo mapping news
Mapping new Googles
New mapping goggles
Have you read my blog lately?
Now, take the data and put up some nice animations, archive the first 100 articles or so and put it into some nice database to mine for interesting stuff. Should not be too hard to script together the data gathering, you can already start fetching stuff while developing the functionality and frontend.
;)
Someone wanna join? This cries 'distributed database'...
Who is General Failure and why is he reading my hard disk?
It's just a screenshot from the NORAD command center!
It would be really cool to see an animation of the map over time, to see how world attention 'sloshes' around. Even better if it was combined with a ticker showing which significant world event corresponds to each burst of activity.
- find where there are lots of new jobs being generated
- view up-and-coming areas by their positive "buzz" (new creative hot spots, architecture, etc...)
- find areas of town with great new restaurants
I think this is where it starts to get exciting (and more useful). Mapping Google news? Meh. Mapping the northwest, and giving that information to Citysearch? You betcha.concrete5: a cms made for marketing, but strong enough for geeks.
Too bad. They have it already.
Some of the smaller map points are a bit broken.
There's a bunch of articles linked from a clickable hotspot in Nelson, a small-ish city in the South Island of New Zealand. They're all about people with a surname of "Nelson", as far as I can tell, nothing to do with the geographical aspect.
You've won the story.
The big circle in the US is called "Washington", which is rated at 03%. It obscures "New York" in the GUI. Boston is available, and the only other US buzz is Grand Rapids, apparently on the strength of a local paper's report 2 days ago of a resident killed in Cairo. I find all that hard to believe, or at least to make into any sense. The GUI is unusable, and the mapping of data to "reality" defies sensibility. I think the buzz has gone to their heads, and they should put the pipe down quick.
--
make install -not war
How are they parsing google news content? Google news does not yet offer an API, correct? What are they doing, screen scraping? You can only query google programmatically about 1000 times a day, I think.
I wish I had more details...
And this is a REALLY stupid aspect to tackle--connections between cities.
THe real cheese would seem to be in word counts, and connections between words--like "economy" and "recession", etc.
eat shiat and bark at the moon
Why haven't they done this with porn and geographically linking ip addresses?
If, they represented this in hierarchical format, the middle east would dominate by picking up points from children Gaza, West Bank and Palestine (not to mention Iraq). Baghdad is probably a good example here. How much actually happens in areas outside of Baghdad proper but gets labled baghdad anyhow.
Who are you? The new #2 Who is #1? You are #617565. I am not a number, I am a free man! Muhahaha.
about the quality of this.
... there seem to be an awful lot of stories about Nelson Mandela but not so many about him being in New Zealnd.
Check out Nelson, New Zealand
Seriously, what's the purpose of this?
Why do we need this?
A map that showed where the stories getting the least attention that contained certain keywords - famine, Schiavo, wobbegong, whatever - came from would strike me as more interesting.
We already know where the stories indicated by this map are coming from, because they're taking up ridiculous amounts of space on the front pages of newspapers everywhere.
Also it is distorted by where it's consulted from, if I enter googlenews encounter more news of latin america that those that appears there.
>Linux is not user-friendly.
It _is_ user-friendly. It is not ignorant-friendly and idiot-friendly.
I'd like to see keywords mapped, especially the word "breakthrough," which I look up on Google News when I'm bored.
this site shows what sources google has linked to from the past few weeks
- Teja
I started to play around with morphing each of the days images into the next. I'll spend more time away from work trying to get that to work. The effect for the month of April was interesting. Now to watch it for the full year, that would be very cool.
Ted
Fantasy remains a human right; we make in our measure and in our derivative mode... -- JRR Tolkien
I've had the thought that it might be cool to implement an anti-news site that would do something like show you links to New York Times stories that have never been referenced by the top page of Google News.
You were using an HTML scraper/screen scraper to parse google news? THe link you posted is dead, BTW.
Did you want you to stop parsing google news or just stop offering it to people via your website?
eat shiat and bark at the moon
One glance at the map shows dramatically how irrelevant Russia is becoming to the rest of the world. How often would such a map have bypassed Moscow entirely during the Soviet era?
Say what you want, but it's interesting to note that the current buzzspots are aligned exactly along the main East-West axis in Eurasia (from China to Europe) as indicated by Jared Diamond in Guns, Germs and Steel.
Victims of 9/11: <3000. Traffic in the US: >30,000/y
Anyway: I stumbled across a weird Google behavior the other day. If you do a regular Google for "read news" you get some weird results at the top of the results page:Try it: http://www.google.com/search?q=read+news
Anyone have any idea what that is? New feature still in development? Old feature never finished? Documented feature I'm being stupid about? Ordinary bug?
Inquiring minds want to know. Well, not really, but Slashdot readers might. Well, me, anyway.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
How long until Google invites the creators to join the team for coming up with such a great idea? Or failing that, aquire the rights to the concept and implement it.
Google have a habit of doing great things with software they get hold of, can't wait to see what they do with this.
How many people can read hex if only you and dead people can read hex?
One draws maps with red circles on them.
We already know where the stories indicated by this map are coming from, because they're taking up ridiculous amounts of space on the front pages of newspapers everywhere.
Exactly. If it hadn't been for the Tsunami, would we have seen as many stories from adjacent countries, for example?
Just because it's not reported, doesn't make it not news. It's just that our filters screen out things that aren't the latest thing.
-- Tigger warning: This post may contain tiggers! --
www.paulrademacher.com/housing
A cool combination of Craigslist housing listing and Google maps. Seems to be very well done.
find / -name "*.sig" | xargs rm
It would be interesting to watch an animation of where the Buzz is over a period of time.
Sig, we don't need no stinking Sig!
Take the / off the end of the link, like this and it should come up
Free Online Woodworking Resources Directory
http://www.buzztracker.org/2004/12/26/index.html
...
i was under the impression that there was nothing else in the news around that time
... must be right about where the servers for buzztracker.org are located.
you did not answer the question--did google want you to stop querying and parsing, or stop showing on your site the results of your queries and parsing?
eat shiat and bark at the moon
What we have here is one computer algorithm aggregating another computer algorithm's assessment of "newsworthy," with no provision for hindsight or fluff-vs-historical weighting. It's a neat idea, and the graphics are pretty slick, but I don't see any real value here.
all it really tells you is where all the reporters are. I don't see how this would be very useful at all: by the time all the reporters are in sri lanka for instance, the tsunami has long past.
http://www.marumushi.com/apps/newsmap/newsmap.cfm has an interactive almost-realtime flash map of google news.
News are shown as rectangles, color coded by topic, size-coded by the importance (number of related news), etc. And you can back track topics by time, you can see a topic grow as news spread and shrink as people stop writing about it. Best viewed on huge screens.
That is all.
I'm surprised no one has mentioned the "News Map":m ap.cfm
http://www.marumushi.com/apps/newsmap/news
It's very cool. Not a geographical map, but a spatial one, with quantity of stories being graphically displayed with size.
It's actually both:
From the text (how did I get marked redundant in my first post, even if I did screw up the url somehow):
In the hope that these events have resulted from your inadvertence rather than your deliberate actions, we propose the following:
1. We demand that you cease and desist using our search service in a manner that is not authorized by our Terms of Service. This includes, but is not limited to, (1) no longer sending automated queries to www.google.com, or other affiliated sites, and (2) no longer using search results from
www.google.com or other affiliated sites, except in accordance with our terms of service and this letter. This applies to the GoogleNews menubar interface to Google News as well as any other products or sites that you operate or control.
2. We demand that you cease and desist using the mark GoogleNews or any other mark or name that incorporates our famous GOOGLE mark or any similar marks.
3. If you remain interested in providing our award-winning search services to your users, we suggest you visit the variety of programs we offer at http://www.google.com/services/.
James Tiberius Kirk: "Spock, the women on your planet are logical. No other planet in the galaxy can make that claim."
Why, that would be Sir Tim Berners-Lee, my dear boy. Please do try and get it right, next time?
This map isn't accurate. You're not reporting on the news. You're reporting on what made headlines. There's a big, big difference.
More people are murdered in Detroit than in than in Baghdad or the surrounding area.
More Americans are kidnapped in Mexico in 3 days than in Iraq in a months' time.
Isn't Mexico supposed to be a friendly country?
Why does the press ONLY focus on Iraq?
Clinton sent us into Bosnia. In fact, we're still there, and the only improvement was the arrest of Milosevic. Since then, they've had as many troubles as they had before. Why doesn't the press report this?
The truth is, the press is HEAVILY biased. They all take their lead from the NY Times, and the NY Times is as biased a newspaper as biased can be.
for ($i=0; $i< $num; $i++) {
//$desc = eregi_replace(" - .* ago</font><br>", "<br>", $desc);
//$desc = htmlspecialchars($desc);
.= " <item>\n"; .= " <title>". $title ."</title>\n"; .= " <link>". htmlspecialchars($item_url) ."</link>\n"; .= " <description>". $desc ."</description>\n"; .= " </item>\n";
.= " </channel>\n"; .= "</rss>\n";
//print "<pre>";
//print "</pre>";
$item_url = $items[1][$i];
$title = $items[2][$i];
$title = strip_tags($title);
$desc = $items[5][$i];
$desc = strip_tags($desc, $allowable_tags);
$output
$output
$output
$output
$output
}
$output
$output
print $output;
// More debug stuff
// print htmlentities($output);
?>
I understand your point, however I think it is partially based on a false premise: In reguard to Nov 3rd. The site tracks cities, not states.
After checking Dec 26, 27, 28, and 29th they do have Indonesia, but it doesn't show up until the 28th (and then under Jakarta only). I would guess this is due to them not having Sumatra or Banda Aceh in their keyword search system.
I also notice that most cities in the US other then Washington and New York seem to almost never show up - could it be that their "selection of articles" is a bit limited (refering to the above's 2nd paragraph)?
you dick, I'm sick of people bitching about google news. Face it, Google is one of if not the largest inovators right now in the tech world, you can't exactly ignore them.
In reference to my own comment about their keyword search system I find it amazing that they lack Banda Aceh and yet have Srinagar from just a few days ago. I would have thought they would have had neither or both. I wonder what their keyword criteria is?
h tm l
http://www.buzztracker.org/2005/04/07/Srinagar.
Would it be neat if MSN ran some queries on their Messenger servers and created a map with points on the map being accounts and lines between points being contacts for accounts.
Guess the map would be huge but still interesting.
Some of you may find this interesting
last few months have been the terri schiavo case in all the headline news and even more so blogs.
hmm.. a blog map, now that would be interesting!
Nice try but the site really doesn't show anything which does any good.
"you dick, I'm sick of people bitching about google news. Face it, Google is one of if not the largest inovators right now in the tech world, you can't exactly ignore them."
Until they turn into Apple, and start suing people.
Well, the first 12 days anyhow.
Quick 'n' dirty animated GIF:)
http://www.cybertects.co.uk/scirocco/fun/news.gif
"Palestine" was a name given by Romes to the province comprising several territories conquered by them - and Gaza was 100% sure among them.
It is quite confusing to use the name of Palestine nowadays, when the map of the region changes so dramaticlly. For example, before 1922, British Palestine included the territories not only of the modern Israel but also that of the Kingdom of Jordan! In 1949 Gaza and Judea/Samaria (the latter often referred to as the West Bank) were occupied in 1949 by Egypt and Jordan respectively.
As to the real source of all of the problems in the Middle East, hardly it is the situation with the Palestinian Arabs (which you probably mean here), but instead it a complex combination of problems, including lack of industrial development in the region, complete lack of democracy, extremely levels of education, low level of life, religious fanatism, and corrupted regimes of the Arab nations of the region.
Google News has a global news scope. The rest of the world was not reporting on electoral intricacies in Ohio. The tsumani reports were reported most heavily from the 27th on.
Notice the weeks before the November U.S. presidential election. I think it's obvious what the world press wanted people to focus on. The heaviest coverage of Iraq took place in the weeks before and during the presidential election. Of course Iraq is the biggest story in the world for the past 2 years, but the fequency was much much higher during our presidential elections.
I disagree with one poster, who claimed that there are more murders in Detroit than Baghdad.
On the other hand, this *does* only map headlines. Two weeks ago, a completely idiotic media frenzy evoked by the US adminstration and the Republicans would have made Pinellas Pk, FL hotter than Iraq, Washington, D.C., or the Vatican. (Terry Schaivo).
You'd need to correlate this in time (has this been in the news in the last (curve) year (or whatever), and weight it with population (are there 15 people in 200 km, or 1.5m?)...and the interesting news would be in areas with little-to-no headlines right now, that have been headlines in the last six months, and have a good-sized population. That's where something is being ignored, in favor of Michael Jackson/the Pope/etc.
mark
Then you can have an animation of where the buzz is by using your favourite slide-show-creator.The junkfilter says that I should have fewer junk characters so I'm guessing I need to fill this out a bit so that the junk-filter will allow me to post this.
Tracking the *datelines* of the articles is a lousy way to track what the article is about, and it seems like that might be what they did. There weren't many reporters on the ground in Aceh province...
Who are you? The new #2 Who is #1? You are #617565. I am not a number, I am a free man! Muhahaha.
Oh, and heaven knows the rest of the globe isn't affected by who becomes President of the United States. That's why there was no international reaction to Bush's re-election.