Optimizing News Sites For Google News

malibucreek writes "More trouble for Google News? Yesterday, it was Google News censoring stories for China. Today, the Online Journalism Review details a potential conservative bias in the site's algorithm for news search results. The story also includes some details about how Google ranks stories on its news page. Turns out that on Google News, backlinks do *not* improve search positioning."

Comments

It's google's job to give balanced news

[BoldAC]

keywords and phrases that match users' precise searches and to write in informal, accessible language.

The article also suggests that using the name is full form, repeatedly, and using keywords in your title makes it receive a higher rank of google news.

Yahoo news is filtered by people; google news is completely automated.

From porn to religion... from the left to the right... many groups have figured out how to manipulate search results. It's life or death in the web world to optimize, It's google's responsibility if they are going to deliver news that they deliver both sides of a story.

Re:It's google's job to give balanced news

[Anonymous Coward]

This manipulation would never happen in the mainstream media.

Regards,
Dan Rather

Re:It's google's job to give balanced news

[Nos.]

However, the question becomes, is Google actually serving news? I honestly don't know. They are basically doing screen scraping (or RSS feeds) to display topics from other sites. Does this consitute serving news? Tough to say. Obviously the content is current events, however, Google doesn't write any of the content. Where does their responsibility lie?

Re:It's google's job to give balanced news

[Red Alastor]

I don't know, where does Slashdot responsibility lie ? They are doing exactly the same thing.

Re:It's google's job to give balanced news

[Jahf]

Google isn't reporting or delivering news. It is indexing those sites that do.

I don't see Google as the place to go when I want to find out what is happening today. I find it the place to go when I read a blurb on one news site and want to get more details or an alternate view from another site.

It would be like using a stock exchange ticker to decide what company is making news ... the bigger the company or the more controversial the news, the bigger they change in their symbol. That doesn't mean it is relevant to me or that there is not more important news out there.

Re:It's google's job to give balanced news

[hunterx11]

Except that Google doesn't post dupes a week later :)

Re:It's google's job to give balanced news

[Senjutsu]

Google isn't reporting or delivering news. It is indexing those sites that do.

But if it wants to remain relevant, it needs to make sure it index those sites in such a way that a balanced presentation of respectable news sites are presented for a query. If the top stories continually run along the lines of "John Kerry is a Gay Commie Space Alien" just because some 2nd tier nutso conservative blog figured out how to best exploit the indexing algorithm, Google News will quickly become useless.

Re:It's google's job to give balanced news

[droleary]

Obviously the content is current events, however, Google doesn't write any of the content. Where does their responsibility lie?

It's really simple; hell, it's even in the subject you're responding to. As a news aggregate, just as with a search engine, bias is a bad thing for Google. I run an aggregate of my own (plug, plug :-), and the very idea that I should favor one site over another (aside from the stated goal of who gives a more timely announcement) is completely bankrupt of any ethical responsibility my site has to it's users.

The old standard of "appearance of impropriety" holds at least as well for Google, too. Same is true for Slashdot article selection. If anyone is getting kickbacks or has some other unstated criteria for selection, that is irresponsible and should not be tolerated. If it's just a bug in their code, a fix will keep their reputation intact. If it's intended at any level, it just gets added to the scorecard that people have started due to questionable action as of late on Google's part.

Re:It's google's job to give balanced news

[Tanktalus]

It's a tough call to say what's "balanced". A rather crude method is to say "50-50". But that doesn't take into account the "fringe" parties, independants, etc. Should all candidates be given equal airtime? Personally, I don't think that would be ideal - I really could care less about hearing about most of the other candidates.

Suppose, then, we come up with some sort of hand-waving idea of balanced being relative to the vote that each candidate will receive. Ignoring for a minute the obvious time-continuity issues, this would definitely be keeping the fringe to the fringe, but with the obvious downside of forcing a two-party system. No one else will get enough airtime to warrant voting, keeping them perpetually on the outside of the electoral process.

Maybe what we really want is "unbiased"? Report all the news, all sides, and let the populace decide. Sounds reasonable - even though some^Wmost voters will ignore the information, it's their choice to be uninformed, rather than the news outets' choice. There would be two ways about this: first, you can just take all the press releases and release that as news (the easy way), or you can research and look for all sides (the hard way). Which one do you think most people would do? Yeah. And if you do research, inevitably, you'll find some sides utterly unbelievable, and fail to report them in an unbiased manner, if you report them at all.

Short version: easier said than done, I think.

Re:It's google's job to give balanced news

[operagost]

But with worse spelling and grammar!

Re:It's google's job to give balanced news

[budgenator]

Google just searches and indexes the web, it's news index has a few additional filters for lameness, so it's obvious to me if users are searching for terms, that the reputable newss providers aren't using they will come up short on the ranking; and are probably a little bit out-of-touch with their readers. Publishing on the web is different than publishing in print and the media is going to have to learn.

All of us geeks have just learned how to search on google news to get a ballanced index, search for "kerry" + "john kerry" that's all

Here come the naysayers!

[Eeknay]

And so the circle is complete. People will now start to attack and slander a once good service, because, hey, it's had its good run. I for one welcome our new evilmegaglobecorp, Google.

It is not Googles responsibility

[cbelle13013]

I'm mildly confused how something automated can have a "conservative leaning" when people aren't doing the crawling.

No, its not going to crawl through a Ih8tebu5h's livejournal entry for 'news' or other blogger oriented 'news'.

Wasn't there a slashdot article a while ago about Google having a seperate section for bloggers so they didn't skew news? Not that all bloggers are liberal, but most of the internet savvy folks I've met are.

Re:It is not Googles responsibility

[gowen]

Not that all bloggers are liberal

Hardly. To its credit, blogging seem to attract self important sociopaths of every political hue.

Re:It is not Googles responsibility

[Zebbers]

Some human coded it somewhere down the line.

Re:It is not Googles responsibility

[LMCBoy]

No, its not going to crawl through a Ih8tebu5h's livejournal entry for 'news' or other blogger oriented 'news'.

No, the article shows that Google News *does* use popular blogs in its results; in fact that's the whole point of the story: that searching for "John Kerry" on Google News presents you with an inordinate number of anti-Kerry rants on conservative blogs, rather than the "mainstream" news results that you get when searching for "George Bush".

The article doesn't try to infer some kind of conspiracy from this; rather, it's probably due to the fact that bloggers typically repeat the full name throughout their articles ("John Kerry is unfit for command! John Kerry is flippity-floppity, and John Kerry speaks French!"), whereas actual news articles tend to revert to "Mr. Kerry" or simply "Kerry". It is mildly interesting that GN indexes these political blogs, though.

Re:It is not Googles responsibility

[LostCluster]

The basic problem forming in the media right now is that there's two distinct flavors:

News reports try to be fair... but the people who do such reporting tend to altruistic people who have a hard-to-hide bias towards the left, always wanting to file a feel-bad-for-this-person report that paints the little guy as a victim and the big company as the bad guy.

Then there's news analysis... that usually lands on the right because the best bigmouths tend to be right-wingers. Even if you disagree with every word they say, they're still more fun to listen two than a left-winger. Fox News Channel frequently has one-from-the-left, one-from-the-right debates on their air, and the right-winger usually is able to talk in soundbytes and talk over the opponent to the point that they appear to "win" the debate more often.

Here's what throws Google for the loop... There's only one AP, and there's only one Reuters. Stories that come out of those two agencies appear in hundreds of web pages, yet there are hundreds of right-wing opinon writers who all express similar ideas in completely different words. Therefore, the right-wing opinion pages sometimes can drown out the left-wing reporting by simply having more entries in the list.

So....

[cr0y]

Not trying to troll here, I don't understand why people are trying to call shinanigans on Google, if they have a bias then that is their right to. If you do not like the services they are providing then don't use it. It's not like they are slandering anyone or posting false headlines.

Re:So....

[stratjakt]

It's a percieved conservative bias, not a liberal one. That's the "problem".

Bias is okee-fine, so long as your bias and my bias are the same.

Re:So....

[revscat]

Not trying to troll here, I don't understand why people are trying to call shinanigans on Google, if they have a bias then that is their right to.

Sure, but if they paint themselves as being equananimous in their presentation then they should be held up to that standard, and criticized when they don't meet up to it. If they want to be biased one way or another then so be it, but they should be upfront about it. It's like Fox; it's not so much the fact that they are conservative I disagree with, it is that they are dishonest in saying they are fair. I actually subscribe to a couple of conservative magazines because of their quality, but they do not deny or try to hide their slant.

To put it another way: Lying is wrong.

Re:So....

[Zapman]

If you actually RTFA, you'll see the real reason burried a little more than half way down:

"I think what you're seeing is an odd little linguistic artifact," said Zuckerman, former vice president of Tripod.com and now a fellow at Harvard's Berkman Center for Internet and Society who studies search engines. The chief culprit, he theorized, is that mainstream news publications refer to the senator on second reference as Kerry, while alternative news sites often use the phrase "John Kerry" multiple times, for effect or derision. To Google News' eye, that's a more exact search result.

Basically, google is doing exactly what we told it too: looking for the most links with 'john kerry' in it.

"Computers are out to destroy us. This can be proven by the fact that they do exactly what we tell them."

Re:So....

[TheFlyingGoat]

Exactly. How many people are complaining that Slashdot slants to the left? I'm a card-carrying Republican and once in a while it annoys me, but I'm smart enough to ignore the bias and just look at the facts. Same goes if I watch Fox News. I would assume most other Slashdot readers are the same.

Our local newspaper (the Milwaukee Journal) is awful when it comes to being liberally slanted, and while the conservatively slanted public radio shows often try pointing out the bias, it's just ignored by the newspaper and the public. There is NO unbiased news. That's just something we have to live with.

This just in...

[Anonymous Coward]

i) world saturated with unreadable political blogs, many right wing.
ii) man who is actually President gets more genuine international news coverage (speeches, commentary, policy, state visits and campaigning) than man who isn't (basically just campaigning).

Thus aforementioned blogs tend to show up prominently in News digests about non-President, because there isn't much to say about him.

/ ~Rocket Science

Article text has excellent theory.

[mopslik]

"I think what you're seeing is an odd little linguistic artifact," said Zuckerman, former vice president of Tripod.com and now a fellow at Harvard's Berkman Center for Internet and Society who studies search engines. The chief culprit, he theorized, is that mainstream news publications refer to the senator on second reference as Kerry, while alternative news sites often use the phrase "John Kerry" multiple times, for effect or derision. To Google News' eye, that's a more exact search result.

Seems reasonable enough to me. Most of the major news I catch does indeed refer to Kerry without his first name. Likewise for Bush.

Hardly an intentional bias.

Re:Article text has excellent theory.

[slungsolow]

The majority if news organizations follow standards similar to the AP Style Guidelines. When dealing with proper names you usually would do the following:

On first reference, use a person's full name, including the middle initial, and title if important to the story. On second reference, use only the last name with no title. In the following example, for instance, we assume that on first reference the person was called Dr. Donald Drumm. The following are possible second-reference uses: The doctor agreed. Drumm agreed.

Crosshairs

[moankey]

So now they are IPO'd it seems they are under a different microscope.

Pre-IPO couple of college kids that worked hard and are smart and made the world better.

Post-IPO, this company is the new MS, look at all the sinister, conspiring things they do, always knew they were no good.

Whats next Google supports terrorism? I guess whatever sells papers or click throughs.

Re:The bias is in american culture

[geoffspear]

Only if "probably" means "definitely not".

Time and Newsweek both have significantly higher weekly circulation. US News doesn't even seem to try to hide its bias; it seems like the very first thing in every issue is an editorial expressing views slightly to the right of Karl Rove.

News Flash: There is no unbiased news

[chia_monkey]

Sorry to burst anyone's bubble, but there is no unbiased news anymore. The media...print, radio, online...is mostly controlled by a few of the major conglomerates. Not only that, but they all have their slants on what is reported and how it is reported. Here's an interested quote from WSJ Opinion Journal

"The chairman of the entertainment giant Viacom said the reason was simple: Republican values are what U.S. companies need."

It's nice to know the media is deciding what to let through and what to report "in our best interest".

Re:News Flash: There is no unbiased news

[the arbiter]

Republican values: They are the party of business.

1. What you do with your body is our business.
2. Telling other countries what to do is our business.
3. What you think is our business.
4. Giving your tax dollars to the wealthy is our business.
5. What you say is our business.
6. Your religious beliefs are our business.
7. Ensuring big business pays its fair share is none of our business.

Clear that up for you?

Our polarized society is the problem

[humuhumunukunukuapu']

This is just a reflection of how polarized our society has become; it was accelerated post 1994, and 9/11 -> Iraq has sent it around the moon and back again.

The article really just re-enforces my thought that it doesn't really matter what news source you read at any point in time, as long as you are reading many different sources on every side of an issue [to the extent possible]. Then you can settle on the truth being somewhere in the middle.

but this is just bullsh!t no matter which side you are on:
http://www.cbsnews.com/stories/2004/09/24/politics /main645393.shtml

My iPod has a liberal bias

[Anonymous Coward]

EVERY time I select Toby Keith it plays the Dixie Chicks.

I realize it's anecdotal, but

[switcha]

I just did a search for each, and Kerry's was pretty much down the line (pos/neg), while "George Bush" yielded four hits out of ten in the first list just from dailyKos.com, a, by any standards, rampant Bush-bashing blog. Actually, I briefly scanned the articles and only 2 were neutral/positive for Bush.

Apparently, it falls the other way as well, but the very fact that a blog on either extreme of the spectrum is showing up that much is a little disconcerting.

Punditry of all stripes is great and I read a ton of them from both camps regularly, but I come to Google News for news, not the OpEd page.

I have a simpler explanation

[Otter]

This isn't about optimizing, it isn't about bias at Google, it isn't explained by ritually invoking the evil spectre of Fox News...

The "second tier" conservative sites write positive things about George Bush and negative things about John Kerry. The analogous liberal/left sites (who don't seem to rate sneering comments about their importance) write negative things about George Bush but have zero positive enthusiasm for Kerry. Therefore, "George Bush" gets both pro and con results; "John Kerry" only gets con. No conspiracy required, just an uninspiring candidate.

You can see the same thing, by the way, on bumpers. Here in John Kerry's home state, there are a zillion anti-Bush bumper stickers and about as many pro-Bush stickers as pro-Kerry stickers. Are cars optimizing their bumpers for my eyes?

Re:Google News Republican Bias?

[geoffspear]

No. All media has a liberal bias. I saw it on Fox (which also has a liberal bias, being part of the evil monolithic media itself.)

Re:In other news...

[TidyKiller]

At the announcement that Google would become a private company (please excuse poor use of terminology), I've heard a lot of SE Gurus talk about the negatives that would follow. According to said Gurus, Google is incredibly influential because it is turned to the most for searches. The way sites are indexed can give them a big lead against other sites.

People are damned cynical. I think that Google will be recieving a lot of flak in the future for doing what it should do as a company: make a profit. If leaning towards the right makes them a buck, then I find it hard to believe they'd do otherwise. It may not be right, but it is their right.

Google-like Systems Need to Understand Expertise

[ctwxman]

One of the biggest shortcomings of the Google News method is not taking into account the source's expertise, implied or otherwise. For instance, domestic US stories are often headlined using Xinhua or The Scotsman as the lead source. It would seem that you will get more detail and understanding from a source closer to the story, or specializing in the story's subject. A Connecticut newspaper or TV station is going to give me more detail and perspective on a story taking place here than someone far away. This weekend, this headline was featured on Google News (I wrote about this in my blog, so I have it at hand): The Sopranos buries the competition. That's a valid story in entertainment news, but the source was, "The Scotsman - Scotland's National Newspaper Online." The next listing was for the Sydney Morning Herald (Australia) followed by ABC News and Planet Out. Truth is, as interesting a tool as Google News is, we still need editors and reporters to weigh facts and sources and see inherent weakness or bias in what is often passed off as complete and balanced facts.

Sounds fishy

[Asprin]

What are the odds that the political landscape Google is surveying actually is more conservative than OJR thinks? If they detected a difference between the sites which use human editors and the Google aggregators which do not, what are they really measuring here - the biases of the Google algorithms or the biases of the other human editors? Correct me if I'm wrong, but Google only knows what it finds.

Just a hunch, but I bet these guys are still trying to figger out why Fox News is so dang-ole popular.

umm..

[helix400]

Conservative bias in Google news? It's just an aggregate..it picks up news from all sides of the spectrum. Because of that, it also displays left leaning sites like Salon, and extreme left-leaning blogs such as dailykos.com.

But then, I suspect the reason this article was approved is because it appeals to michael's left leaning bias, which he unapologetically admits he has. As he said: "I'm trying to dispel all notion that I'm unbiased, or that I'll be presenting everything in an entirely unbiased fashion. If my biases totally offend you, you might want to go right now to your user preferences and check the box to block stories posted by me."

censorship through google ignorance/laziness

[WormholeFiend]

If you go to google.ca, click news, there is a Canada News section in English...

But if on the google.ca page you click on Google.ca offered in: Français, then on Actualités (News), you're forwarded to the google.fr (France) news page.

France != French Canada

Potential is key

[follower_of_christ]

Today, the Online Journalism Review details a potential conservative bias in the site's algorithm

Conservatives probably see articles like the following and start sniffing around for conspiracy. Whether a conspiracy exists or not. I'm starting to see a common thread amongst conservatives of boycotting orginizations that even hint liberal ideals. As a conservative myself I see a large movement away from the major media by most of my conservative friends around the nation and world due to "media bias" and its presentation of liberal ideals. (I'm probably redudant here.)
The advent of the internet, blogs, and talk radio allow this to happen. It saddens me because I feel that there hasn't been substantive debate in over a decade because both "new" and "old" media has bias and both camps are clinging on to the media that shares their views and shuns out the opposition.

I'm longing to have a healthy debate about issues rather than a shouting match where both people leave mad feeling more "right" than when they began.

Article
Article

Re:The bias is in american culture

[Concerned Onlooker]

Going back for a moment to the 2000 Presidential election and a study by the Pew Charitable Trusts Project for Excellence in Journalism:

Tone of Coverage for Gore & Bush *
Gore Bush
Positive 13% 24%
Neutral 31% 27%
Negative 56% 49%
Total 100% 100%

* source

Not to mention--but I couldn't find the source--that I think over 50% of newspapers endorsed Bush in 2000.

If you want neutrality...

[FooAtWFU]

go to Wikipedia, where neutrality isn't just a Nice Thing, it's the #1 policy . There's a Current Events spot on the front page.

Oh? What's that? It's not as comprehensive? Well, it's a wiki, not a search engine! Seems you just can't have it both ways...

Note that there is talk of a WikiNews run by the MediaWiki foundation, but at present it is mostly idle speculation, and no real plans to make such a site.

Re:Google-like Systems Need to Understand Expertis

[Once&FutureRocketman]

I don't disagree that it would be useful if their algorithm had some effective way of separating the cluefull from the clueless. However, your statement that

It would seem that you will get more detail and understanding from a source closer to the story, or specializing in the story's subject.

is not to be taken for granted. In particular, it is often the case that foreign reporting of domesitic issues is more balanced and useful than what we get from American news sources.
Particularly under this latest administration.

Conservative Bias?

[nwbvt]

Lets, the first 25 of todays results for news searches on George W Bush and John Kerry:

Bush: 17 negative headlines
Kerry: 6 negative headlines
(For the record, I am not reading each and every article, just counting it if the headline appears to be negative. Also, I am also counting headlines that bash both candidates as negative).

Sorry folks, I don't see the 'conservative bias'. Granted one would probably expect a few more negative results with regard to the current president regardless of which party is in office, today Bush had nearly three times as many.

No, I'm not arguing that Google news always has a liberal bias (it uses algorithms, not editors, to decide what to post), just that finding a few conservative-leaning headlines after a few experiments (they only loosely document two, though they claim there were others) is not evidence of a conservative bias.

Of course backlinks aren't used for news

[Flexagon]

Turns out that on Google News, backlinks do *not* improve search positioning.

Seems quite reasonable. After all, being news, how is it going to have many backlinks? And how are they all going to be found while the news is still new? By the time the news is old enough to appear in Google's regular results, backlinks become useful. Am I missing something?

Re:Definitely a troll

[Concerned Onlooker]

The group International Answer does exist but their agenda does not seem to be to support Saddam Hussein, as the parent would suggest. They may not even be radical leftists.

A thing may indeed be impossible to achieve, but that does not mean one should not attempt it anyway. I don't think we'd be well served to go back to the yellow journalism days. Thompson's Gonzo journalistic style--which is really just a first person narrative or even documentary--has a place but there are those of use who want a more complete perspective.

This does not mean getting exact opposite pieces of information from both sides. It means getting both sides to comment on a topic.

Aiming for a high standard but not reaching it is better in my mind than aiming for a low standard and hitting your mark.

Re:Hatesites?

[MartinB]

Not to mention the telegraph is openly and proudly conservative. Just ask its owner Conrad Black.

1. That'll be former owner, Conrad Black, currently accused of embezzeling £223 million from the Telegraph Group.
2. Conservative in the UK has a very different meaning to that on the left hand side of the Atlantic. Both US political parties are to the right of mainstream UK politics.
3. Historically, the Telegraph has always been the newspaper reflecting the views of the generally well educated, landowning segment of the rich.
4. In the years since the Blair government came to power, it has moved from its former stated editorial position of 'broadly supportive of the [previous] Conservative [UK sense] government' to a mixture of generic libertarianism, support for their readership's areas of interest - foxhunting and maintaining heriditary peers in the House of Lords primarily - and increasingly rabid attacks on the Blair administration. In this last point, they appear to be taking baby lessons from the US Conservative approach.

The daily mail is the brit liberal paper, btw.

I can tell you don't read it, or rather, don't read it in context of the rest of the national press here. You're woefully misinformed: the Mail is somewhat to the Right of centre.

Here's an approximate right-to-left split of the national press (Sunday papers omitted for simplicity):

• Right of Centre (in very approximate order, although papers in different markets are often hard to directly compare):
  • Daily Express
  • Daily Telegraph
  • Sun
  • Daily Mail
  • Times
• Left of Centre (again, in approx leftward ascending order):
  • Independent (although on some issues, much further left than the Guardian)
  • Daily Mirror
  • Guardian

Alternatively, there's the traditional definition set of the national press quoted for the benefit of novices such as yourself, which is still sufficiently accurate to be a useful rule of thumb:

The Times: Read by the people who run the country. Daily Mirror: Read by the people who think they run the country. Guardian: Read by the people who think they ought to run the country. Morning Star: Read by the people who think the country ought to be run by another country. Daily Mail: Read by the wives of the people who own the country. Financial Times: Read by the people who own the country. Daily Express: Read by the people who think that the country ought to be run as it used to be. Daily Telegraph: Read by the people who think it still is. The Sun: Their readers don't care who runs the country as long as she has big tits.