Slashdot Mirror


Google News to Host Wire Service Stories

knhasan writes to tell us that Google has just announced a new program in which they will host wire news stories directly on their site. This is widely believed to be the first concrete fallout from recent troubles with Agence France Presse (who sued Google for alleged copyright infringement) among other wire services. "The new feature unveiled Friday is called 'duplicate detection,' which lets Google News identify the original source of a story that may appear in tens or hundreds of news outlet Web sites. If the source story is from one of the four news service agencies that Google has licensing agreements with, Google will display the story on a page that it hosts."

63 comments

  1. Woohoo by Ironspork · · Score: 3, Funny

    So we can finally stop getting the same story posted over and over? (hint, digg and /.) If their stories are decent, I'm moving over there....

  2. Where's my nose? I had it a second ago... by DumbSwede · · Score: 0

    First news story on Google wire service: Agence France Presse cuts off nose. To spite face suspected.

  3. Finally... by bcdm · · Score: 5, Funny

    A website that cares about duplicate detection! It's as if they understand that readers don't want to read the same story again and again and again! I wish I could think of another website that would benefit from this technology!

    --
    I can has sig?
    1. Re:Finally... by ettlz · · Score: 5, Funny

      A website that cares about duplicate detection! It's as if they understand that readers don't want to read the same story again and again and again! I wish I could think of another website that would benefit from this technology!

    2. Re:Finally... by Anonymous Coward · · Score: 5, Funny

      From:SPAA (Slashdot Posters Association of America)
      To: ettiz (639203)

      Sir,

      We note with some regret (but mostly with glee) that you have duplicated, copied and otherwise stolen one of our (potentil) members Intellectual Property. It is estimated that this heinous crime has cost our (potential) member in excess of $10,000 per view. You have stolen this persons livelihood and he will now be required to live on the street and will, most likely, die of starvation and sorrow before the next equinox. Given these indisputable facts, and your clear malicious intent, we hereby notify you that unless you pay $1,000,0000 immediately to the SPAA (we will then hand 0.001% of the settlement directly to our member) we will be obliged to take you (or someone like you, or possibly someone dead) to court.

      We expect and demand your immediate co-operation. (or that of an employer, your children, your ISP or your cat)

      Signed

      Tod Hsals
      (Vice deputy junior legal officer)
      Slashdot Posters Association of America

      (For clarity, this is not a legal notice, you need take no action, and its totally off topic, if slightly pertinent to your last post.)

    3. Re:Finally... by rednip · · Score: 1

      I wish I could think of another website that would benefit from this technology! What technology? figuring out if many different stories all have the exact same text, by the same author, and attributed to a single news service? Granted, Slashdot has had a history with dupes, but word for word dupes are a different story.
      --
      The force that blew the Big Bang continues to accelerate.
    4. Re:Finally... by Anonymous Coward · · Score: 0

      A website that cares about duplicate detection! It's as if they understand that readers don't want to read the same story again and again and again! I wish I could think of another website that would benefit from this technology!

  4. It's a good thing by Evets · · Score: 3, Interesting

    From a consumer standpoint, I really like this move.

    It seems to be completely random which site a given story will point to and there are times when I click through to a news item and I'm immediately skeptical of the source site. If a news vendor isn't doing any sort of value-add, I don't see why I should get sent to bob's scraped wire site versus a trusted major news source.

    1. Re:It's a good thing by LiquidCoooled · · Score: 3, Insightful

      I go to google news specifically for the variation in sources.
      I like to see the varying articles and standpoints available for the same story (usually in the editorials around the actual feed).
      If google starts feeding me a single specific variation with one POV its worse in my book.

      --
      liqbase :: faster than paper
    2. Re:It's a good thing by MBraynard · · Score: 5, Informative
      Try reading. It's easier than being stupid. Let me explain.

      Hundreds of websites run the same identical AP/AFP/Reuters/UPI/Etc. wire services news story. Until now, G news would show ALL of those stories.

      Now, when it detects a story that is from a wire service and just reposted, it will show it on the Gnews site, as Gnews has a subscription for all of them (except AFP apparently.)

      However, if the Corner Podunk Press writes an original story on the shuttle launch, it too will be linked.

      The loser here are news sites that do run wire service stories as they will no longer be getting free traffic from Gnews. This might encourage them to be more original.

    3. Re:It's a good thing by McDutchie · · Score: 1

      Try reading. It's easier than being stupid.

      Oh dear. If only that were true, the world would be a much better place.

    4. Re:It's a good thing by Anonymous Coward · · Score: 0

      Try reading. It's easier than being stupid. Let me explain.

      From the grandparent: "usually in the editorials around the actual feed" (emphasis added).

      Let me explain.

      While the large agencies put out articles and individual newspapers republish those usually the individual newspapers add their own twists. Sometimes this is just the headline. Other times the content of the article is edited. The interesting thing here is not just the news, but the meta-news. Individual newspapers add their own editorial biases to news stories. By sampling a large number of newspapers, even on the same topic, even when the original source is identical, gives you more insight into these biases. Of course because newspapers generally represent the populous they serve it also gives you some insight into the world at large.

      I don't know about the grandparent but this is how I've used Google News. That's not to say I exclusively use it like this; typically one article is good enough. But when there's a story I'm particularly interested in it is a useful technique to gain some insight.

      But given how you seem to feel the need to insult people from a position of ignorance I doubt you'd be interested in such things.

    5. Re:It's a good thing by MBraynard · · Score: 1
      Google News is my home page and has been since 2003.

      Now let me explain to you.

      Editorials on their own pages do not get overlooked by this program.

      Re-writing a headline OUGHT to be overlooked.

      And by far, the vast and fast majority of the stories that gnews had been linking were 100% identical. 50000 newspapers carry AP, 50000 newspapers have website, there would be 50000 linked stories on the topic on GNews, making it impossible to find the actual original content.

    6. Re:It's a good thing by martin-boundary · · Score: 2, Interesting
      I'd mod you up if I had points, that's spot on. The "upstream sources" are syndicated wire news feeds and columns, and the downstream media (that includes "trusted major sources" like the NY Times) pick and choose the bits of a given story that they want to show, and rewrite it for effect and desired story size.

      It's actually very easy for Google engineers to identify the sources, because they _have_ all the possible source texts at their disposal: anybody who subscribes to AFP/Reuters/AP/etc obtains the raw sources. Then it's just a matter of writing a program which computes the percentage of sentence overlap between a downstream story and each of the possible raw stories - with a precomputed index it's very simple: anybody can do this with open source indexing tools and open source text analysis tools on freshmeat.

      The case of images is both easier and more difficult: in most cases, the images on a news story are syndicated and used "as is", so a simple pixel comparison gives a match to the syndicated source, but sometimes the image is cropped or modified, and then it's difficult to identify, but not necessarily impossible.

      Unfortunately, it's not possible to infer the bias of the nation wide (or world wide) reading population with your method, as the article collection system used by Google introduces a publication bias itself, which swamps the natural proportions. All you get is a non uniform sampling of news sources, which at best tells you the possible extremes of viewpoint assuming they reflect their reader base, but that's still interesting.

    7. Re:It's a good thing by Anonymous Coward · · Score: 0

      ok, maybe I should have bolded the word "around". It's not just headlines or pure editorials - it's editing the original article. It happens, it's interesting to see and observe.

    8. Re:It's a good thing by Anonymous Coward · · Score: 0

      In my perspective it's not only a cool idea, it's also a slap in the face to those who would challenge google. If you can't Join Em, Kick the shit out of em' should be googles NEW motto. Perhaps, "if they sue you, undermind their entire model and wipe them off the face of the earth." would be nice.

    9. Re:It's a good thing by Epistax · · Score: 1

      However, if the Corner Podunk Press writes an original story on the shuttle launch, it too will be linked.

      However, if the Weekly Podunk Herald writes an original story on the shuttle launch, it will not be linked.

      We hate them.

    10. Re:It's a good thing by watchingeyes · · Score: 1

      Are you retarded? Did you even read the summary?

      --
      http://watching-eyes.blogspot.com/
    11. Re:It's a good thing by watchingeyes · · Score: 1

      disregard my post, didn't even post it on the right story :-P

      --
      http://watching-eyes.blogspot.com/
    12. Re:It's a good thing by coaxial · · Score: 1

      The loser here are news sites that do run wire service stories as they will no longer be getting free traffic from Gnews. This might encourage them to be more original.


      Yeah. I hear the The North Platte Bulletin has an opening for a Beijing bureau chief.

      Wire services exist because it is impossible for every news outlet to cover every story. Hell, given the buisness situation in the newspaper industry, even the major papers are drawing down their national and international coverage and relying more on wire services. I'm specifically thinking the LA Times' recent hyperlocal direction.

      The idea that somehow that newspapers sites were getting reliable significant "free traffic" from google news is absurd. gnews's site selection algorithm is essentially random. We've all seen examples where gnews would highlight some white supremacist news site. (Yes, Google has since blacklisted that site from gnews.) It's no secret who the top sites featured on gnews are, its the old standbys. In all actuality, having gnews highllight some small newspaper could actually cause more problems for the paper, when their site is inevitably slashdotted, or gnewsed as the case may be.

    13. Re:It's a good thing by Aidan+Steele · · Score: 1

      Now, when it detects a story that is from a wire service and just reposted, it will show it on the Gnews site, as Gnews has a subscription for all of them (except AFP apparently.)

      You sure about that? Google displays wire stories from AFP, on the Google News FR website at the very least. Also, this has been happening for at least a month. I figured this was a non-event for ./ goers, but apparently we're just extraordinarily slow on the uptake.

    14. Re:It's a good thing by jnnnnn · · Score: 1

      And if those "reposter" news outlets do become more original (or die off), the wire service loses big time - as it loses lots of subscribers.

      Perhaps Google is doing this as a way to hit back at AFP for all the lawsuits?

    15. Re:It's a good thing by Anonymous Coward · · Score: 0

      So just click "Sort by date with duplicates included", and viola! Articles sorted by date with duplicates included!

    16. Re:It's a good thing by MBraynard · · Score: 1

      If they change it or add to it, then Gnews won't filter it out because it won't show as identical.

  5. Let me guess who'd be called "duplicate" by sethstorm · · Score: 0

    "The new feature unveiled Friday is called 'duplicate detection,' which lets Google News identify the original source of a story that may appear in tens or hundreds of news outlet Web sites. If the source is AFP, would they just be considered a duplicate out of spite?
    --
    Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
  6. No Reuters? by RockMFR · · Score: 0

    I am sad :(

  7. All your newsbase are belong to us by symbolset · · Score: 0

    Checkmate. Google owns the news. Game over for your local paper. I predicted this many years ago.

    It's about time.

    --
    Help stamp out iliturcy.
    1. Re:All your newsbase are belong to us by Anonymous Coward · · Score: 2, Insightful

      I've never read my local paper for anything other than local news, which Google doesn't report.

    2. Re:All your newsbase are belong to us by larry+bagina · · Score: 1

      no, this kills news websites (breitbart, apnews.myway.com, etc) that run nothing but wire stories. Currently, story selection etc is computer generated and supposedly fair. I suspect they'll set up something to do user news aggregation/selection/mashup (like drudge report) sooner or later.

      --
      Do you even lift?

      These aren't the 'roids you're looking for.

    3. Re:All your newsbase are belong to us by westlake · · Score: 4, Interesting
      Checkmate. Google owns the news. Game over for your local paper. I predicted this many years ago.

      The game begins for your local paper.

      The Niagara Falls Reporter is a free tabloid that efficiently - and hilariously - extinguished the career of the most corrupt and incompetent mayor this border town has known in living memory.

      It succeeds by relying on a minimal staff, reporting and opinion with strong local roots - in John Hanchette, for example, it has a founding editor of USA Today,a former editor of the Niagara Gazette and a man with a Pulitzer to his credit and a national reputation as a journalist and teacher.

  8. I'm hoping for better reporting. by khasim · · Score: 2, Interesting

    So, there won't be duplicates.

    Which means that in order to attract people to YOUR news site, you'll have to ADD something. Either background research, interviews, commentary, etc.

    Sure, the commentary might not be "better". It will probably still be biased. But the facts should appear more consistently now.

    1. Re:I'm hoping for better reporting. by LiquidCoooled · · Score: 1

      Does this mean Roland will get a wagerise?

      *duck*

      --
      liqbase :: faster than paper
  9. So... by trondotcom · · Score: 0

    So the original news by default is theirs? Hopefully there won't be the same for other services.

  10. Agence France Presse should know that by unity100 · · Score: 1, Insightful

    i didnt even know they had ever existed before google. And they should also know that i will probably forget them in a short while since they are not on google anymore. they lost enormous exposure to pursue some petty cash compared to what the exposure would have brought.

    they can shove their copyright up their "butté" now - for its probably useful for only doing that compared to what it might have been.

    1. Re:Agence France Presse should know that by Anonymous Coward · · Score: 1, Funny

      Newsflash: Poster on Slashdot ignorant about one of world's largest news agency. AFP reported to give a shit.

      Film at eleven.

    2. Re:Agence France Presse should know that by larry+bagina · · Score: 1

      Did you read the article? Google signed a deal with AFP to license/host AFP stories. They are on google now, not just excerpts.

      --
      Do you even lift?

      These aren't the 'roids you're looking for.

    3. Re:Agence France Presse should know that by Anonymous Coward · · Score: 0

      they can shove their copyright up their "butté" now


      butté (Noun, feminine): mound, hillock?
    4. Re:Agence France Presse should know that by rtb61 · · Score: 1

      Well this will certainly please google's Chinese government masters no end, not only will google be able to censor searches to troublesome freedom and democracy news articles, they will now be able to directly censor the news items themselves, either via deft electronic automated editing based upon key words and contextual word sentence structures (go ahead google patent it) or just deleting the articles all together.

      --
      Chaos - everything, everywhere, everywhen
    5. Re:Agence France Presse should know that by unity100 · · Score: 1

      Newsflash : if a poster on slashdot and a long time resident of internet doesnt know about afp, "the world's largest news agency', then it means that 'world's largest news agency' is not that much 'world's' agency.

    6. Re:Agence France Presse should know that by unity100 · · Score: 1

      eh, rather a mocking 'frenchalization' of plain old butt.

  11. In related news... by Spy+der+Mann · · Score: 2, Funny

    the UPSTO has just received a patent application from some Dr. McLeod for a "method and apparatus to discriminate identical story submissions in news sites, in which there can be only one".

    1. Re:In related news... by slarrg · · Score: 1

      the UPSTO has just received a patent application It's a good thing they didn't file at the USPTO. There may still be time for Google to get a real patent.
  12. dup detection -- fantastic for the smaller papers by sdedeo · · Score: 2, Insightful

    I don't know why I'm posting so much today, but anyway.

    This is very useful. As far as I can tell, it only means that you won't get a billion copies of the same AP, Reuters, etc. press release that many papers, because they have cut their staff, print in lieu of actually doing their own research. This is fantastic news, and will hopefully be another reward for smaller newspapers who do actually do something instead of print ads for car dealerships. If the Sasquatch Press has a Middle Eastern correspondent, their journalist's work will not be lost in the spam flood of AP articles. Of course, the Sasquatch Press won't have such a correspondent, but they may indeed have one for Sasquatch City -- who probably knows a hell of a lot more than the AP reporter airlifted in when the Sasquatch Robots gain consciousness.

    --
    Protect your liberties. Donate to the ACLU
  13. everybody should have seen this coming by larry+bagina · · Score: 1

    Why should google redirect you to joe random's copy of the same AP story when they can display it themselves?

    In a similar fashion, froogle/google products (and google in general) has a similar situation with dozens of e-commerce (often shitty osCommerce based) stores selling the same stuff from the same supplier, drop shippers, etc. Since google is an information company, they probably don't need to worry ... yet.

    If this helps kills off websites that add no value, then good riddance.

    --
    Do you even lift?

    These aren't the 'roids you're looking for.

    1. Re:everybody should have seen this coming by Wesley+Felter · · Score: 1

      Why should google redirect you to joe random's copy of the same AP story when they can display it themselves?

      It costs Google nothing to link to another site, while it costs them to syndicate wire stories. OTOH, if Google displays the story they can keep the ad revenue.

    2. Re:everybody should have seen this coming by Kadin2048 · · Score: 1

      It costs Google nothing to link to another site, while it costs them to syndicate wire stories. OTOH, if Google displays the story they can keep the ad revenue. Since there are lots of sites around that do nothing but republish wire service reports in order to bring in ad revenue, it stands to reason that the ad revenue is more than enough to pay for the wire service.

      Therefore, what Google is setting up is a paying proposition. They're end-running all the eyeball-grabbers.
      --
      "Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
  14. Where's my perspective? I had it a second ago... by Anonymous Coward · · Score: 0

    Actually why didn't Google do this in the first place? Agence France Presse was within it's rights, but Google going with what it's presenting now would have kept the problem from arising to begin with. If not Agence France Presse then it would have been someone else sooner or later.

  15. made the suggestion a year ago by Anonymous Coward · · Score: 0

    I made a similar suggestion to Google a year ago. I was looking for a different source on a story and Google kept showing the same headline story for several pages. They were all different news hosts, but it was the same article. It was very annoying. I'm likely not the only person though.

    captcha: imbecile, how very appropriate

  16. Until now we didn't have a Middle East reporter by Infonaut · · Score: 1

    If the Sasquatch Press has a Middle Eastern correspondent, their journalist's work will not be lost in the spam flood of AP articles.

    You're absolutely right. I'm Stormy Flowers, Editor in Chief of the (Toenail Sands, OR) Sasquatch Press. Nobody knows the Sasquatch Robot story like we do, but there's no reason we can't branch out in to Middle East coverage. I'm sending the ever-reliable Bruce Foggins out to Iraq on the next available C-17.

    Thanks for thinking of the Sasquatch Press!

    --
    Read the EFF's Fair Use FAQ
  17. Nope. by raehl · · Score: 2, Funny

    Try reading. It's easier than being stupid. Let me explain.

    Not true. Being stupid is easier. No explanation required.

  18. Re:PLEASE REPLY TO THIS COMMENT WITH YOUR BEST TRO by Anonymous Coward · · Score: 0

    http://lbn.threat.tv/apple_ipod_nano_exclusive_pre view.swf

    Exclusive video of the new iPod Nano, looks very real - I can't wait!!!

  19. Whoever modded this down is an idiot by symbolset · · Score: 2, Interesting

    The newspaper I was working for when I predicted this is still available at its vestigal domain name here where I helped set it up.

    At the end of a meeting to review a very expensive (>100K$) demographic survey in 1992, I spoke my mind. I told him a number of things, including that the toxic ink on dead tree business model wouldn't last forever, that communities were more important that forums, that the Internet wouldn't be male dominated forever and that user generated content was more important than expert generated content. He thought I was a flake. It cost me my job to tell him what I really thought, and I was right. It cost him >100K to hear what the demographer thought he wanted to hear.

    I don't regret it at all. He was an idiot too and he deserved to miss out on the .boom billions he could have had.

    --
    Help stamp out iliturcy.
    1. Re:Whoever modded this down is an idiot by caluml · · Score: 1

      that communities were more important that forums
      Define community for me?

  20. Re:Where's my perspective? I had it a second ago.. by watchingeyes · · Score: 1

    Really? Where are the court rulings that what Google was doing was illegal? There's only one I'm aware of, in Belgium, that is under appeal.

    --
    http://watching-eyes.blogspot.com/
  21. Great news for legit sources by hugoguzman · · Score: 1

    This is great news for independent sources that actually publish original content. They won't have as much copycat content to go up against.

  22. Good. by TechwoIf · · Score: 1

    I use goggle news. My main beef with it is for a major news stories, I see there is about 1,000 plus articles. However, when I pull up the full list of them and start clicking some of them to read more of the story, they are all the EXACT same thing. I would love it if they detect the dupes and just one link to the major wire services and the others are NOT reprinted wire stuff. I like to get more details and there is usually local paper that does it own story with more details, but it gets buried in 1,000 links of the same story from the wires.

    1. Re:Good. by cyberstever · · Score: 1

      Does anyone here realize that most of the AP stories are written by those very same small papers that google will now no longer link to? You don't think the AP actually has 1000's of reporters do you? Seems like a bad deal for the originating sources. Why is this better than picking the source closest to the actual story and posting that story? Dupicate detection is a red herring and something they could already do - how else did they know there were N related stories to begin with?

  23. Sounds Like The Wire Services Got Hosed by logicnazi · · Score: 1

    I'm sad to see google giving in a situation that is obviously a matter of fair use.

    However, unless the wire services are getting a fairly substantial *percentage* of the ad revenue they are the ones who are ending up as the losers. Sure had google merely licensed the content it would have been a victory for the wire services but the whole original detection bit is a stroke of genius. It may look like a nice gesture but the net effect will be to greatly reduce the value newspapers get from hosting wire service stories on their sites and thus reduce the number of clients for the wire services.

    --

    If you liked this thought maybe you would find my blog nice too:

  24. In context by symbolset · · Score: 1

    The community is the people. The forum is the place where they meet. More to my point a forum is usually a place where the few educate or entertain the many, while in a community everyone participates, even if it's just to set the limit of what they'll tolerate before they leave.

    --
    Help stamp out iliturcy.