Murdoch-Microsoft Deal In the Works
Hugh Pickens writes "The Financial Times reports that Microsoft is in discussions to pay Rupert Murdoch's News Corp, owner of newspapers ranging from the Wall Street Journal of the US to The Sun of the UK, to 'de-index' its news websites from Google, setting the scene for a search engine battle that could offer a ray of light to the newspaper industry. Microsoft is desperate to catch Google in search, and, after five years and hundreds of millions of dollars of losses, Bing, launched in June, marks its most ambitious attempt yet. Microsoft's interest is being interpreted as a direct assault on Google because it puts pressure on the search engine to start paying for content. 'This is all about Microsoft hurting Google's margins,' said the web publisher who is familiar with the plan. 'It's easy to believe that [Microsoft] may spew senseless riches into publishers' pockets, radically distorting the news market, just to spite Google,' writes Rob Beschizza at BoingBoing. 'Murdoch could be wringing cash out of a market he knows is doomed to implosion or assimilation. And he doesn't even have to be an evil genius, either; he just has to be smarter than Steve Ballmer.'"
The children are right to mock you AC. Google honors robots.txt, if a news outlet doesn't want their site indexed, all they need to do is put a deny rule in it.
How we know is more important than what we know.
He does put a robots.txt file in his sites. See for example
http://www.timesonline.co.uk/robots.txt
http://www.thesun.co.uk/robots.txt
He's put loads of crawlers on it. Googlebot isn't one of them, because he presumably is happy for it to visit.
he already uses a robots.txt file,
User-agent: * /printer_friendly_story /projects/livestream
Disallow:
Disallow:
#
Sitemap: http://www.foxnews.com/google_news_news.xml
Sitemap: http://www.foxnews.com/google_news_entertainment.xml
Sitemap: http://www.foxnews.com/google_news_opinion.xml
Sitemap: http://www.foxnews.com/google_news_politics.xml
Sitemap: http://www.foxnews.com/google_search_news.xml
Sitemap: http://www.foxnews.com/google_search_entertainment.xml
Sitemap: http://www.foxnews.com/google_search_opinion.xml
Sitemap: http://www.foxnews.com/google_search_politics.xml
Sitemap: http://www.foxnews.com/google_search_sections.xml