Google and Yahoo! Working Together On Better Web Indexing

Karzz1 writes "In an exclusive video interview with WebProNews, Yahoo and Google announced a collaborative site called sitemaps.org. Yahoo!'s Tim Mayer states in the video, 'This is something we are announcing tonight at around 9 PM tonight (Las Vegas) Google and Yahoo have gotten together to provide webmasters and publishers a unified way to send their content... let our search engines know about new and existing content.'"

Comments

What About Microsoft?

[eldavojohn]

Well, I went to the sitemaps.org site & looked around for the people owning/running/maintaining the page. In the TOS, I found it to start with:

Terms of service

This is a contract between you and each of the sponsors of Sitemaps.org: Google, Inc., Yahoo, Inc., and Microsoft Corporation (referred to collectively in this agreement as the "Sponsors," "we," or "us"). By using the Sitemaps.org website (the "Website") you agree to be bound by the following terms and conditions (the "Terms of Service").

Scope of Terms of Services; License

These Terms of Service govern your use of the Website. The Sponsors' copyrights in the sitemaps protocol specification, as published on the Website (the "Specification"), are licensed to you under the Creative Commons Attribution-ShareAlike License (version 2.5). Other than the Sponsors' copyrights in this Specification, no intellectual property rights of any kind are granted or may arise under these Terms of Service, whether express, implied or otherwise.

So as you can see, Microsoft is also involved in a project under the Creative Commons Attribution-ShareAlike License (version 2.5). Which is in and of itself something newsworthy in my opinion--since they're so often played as the bad guy on Slashdot. Frankly, the article states:

"The quality of your index is predicated by the quality of your sources and Windows Live Search is happy to be working with Google and Yahoo! on Sitemaps to not only help webmasters, but also help consumers by delivering more relevant search results so they can find what they're looking for faster," said Ken Moss, General Manager of Windows Live Search at Microsoft.

So why is Microsoft omitted from the summary & title of this news? Surely their Windows Live Search is contributing just as much as Yahoo!'s search or Google's search engine.

I'm confused--when Microsoft does something good, do we just ignore it? You know, I'm all for criticizing their evil plans for world domination in the software market but shouldn't news be subjective not objective even if it is only for nerds?

Side note, I'll bet this post hits rock bottom like any other post that says something positive about Microsoft.

Re:What About Microsoft?

[HoldenManiaC]

Its not the fact that we ignore the good things that Microsoft do...its the fact that that Microsoft rarely do a sumthing good that we get distracted by the bad things its doin....take XP for example....everytime Microsoft do something good, i still can't help but want to throw my computer out the window because XP is always crapping out on me. plus the fact that the good things that Microsoft *rarely* do are always so small and insignificant that no one cares.

Re:What About Microsoft?

[Kozz]

I'm confused--when Microsoft does something good, do we just ignore it?

You must be new here.

Re:What About Microsoft?

[Salvance]

What I find interesting is that this article was submitted multiple times last night with Microsoft's name actually included Microsoft in the title (the Firehose is a pretty cool feature of being a subscriber BTW since you can see all submissions, not just accepted ones). Either the wording wasn't as concise/clear (I don't remember), or there was a little bias exhibitted by the editors.

Re:What About Microsoft?

[mpcooke3]

To be fair google and yahoo are the big search engine players, MSN search is under 15% of the market compared to say googles at around 45% and yahoo at around 30%.

Source: http://searchenginewatch.com/showPage.html?page=21 56431

Re:What About Microsoft?

[Karzz1]

I was the one who submitted this article and the reason MS was not mentioned is that they were not involved in the video interview that was released. I could have been a bit more specific with regards to the description though, so as not to ignore MS involvement in the project.

Re:What About Microsoft?

[Himring]

Its not the fact that we ignore the good things that Microsoft do...its the fact that that Microsoft rarely do a sumthing good that we get distracted by the bad things its doin....

...

ahh, my hed asplode....

Re:What About Microsoft?

[Anonymous Coward]

I could have been a bit more specific with regards to the description though, so as not to ignore MS involvement in the project.

This isn't your fault, don't apologize for anything. My original post about lack of Microsoft was actually targeted at the editors who select stories. I'm just trying to raise concern that the "Slashdot Criteria for story selection" now includes spin against Microsoft and they don't like stories that mention Microsoft doing good.

Re:What About Microsoft?

[Coward the Anonymous]

I'm confused--when Microsoft does something good, do we just ignore it? You know, I'm all for criticizing their evil plans for world domination in the software market but shouldn't news be subjective not objective even if it is only for nerds?

You got those backwards. Objective means without bias while the news you are complaining about is subjective, it is biased towards downplaying the good things Microsoft does.

Semantics, they'll get you everytime.

Re:What About Microsoft?

[joto]

In other words, MSN is bigger in the search market, than firefox is in the browser market.

Re:What About Microsoft?

[Anonymous Coward]

Side note, I'll bet this post hits rock bottom like any other post that says something positive about Microsoft.

Phaw! I bet you're just pissed because that comment broke your karma whore streak.

Re:What About Microsoft?

[ScentCone]

or there was a little bias exhibitted by the editors

Is that like being a little pregnant?

Re:What About Microsoft?

[Anonymous Coward]

You have the meanings of subjective and objective confused.

Re:What About Microsoft?

[_Sprocket_]

Side note, I'll bet this post hits rock bottom like any other post that says something positive about Microsoft

Sometimes I get this petty little feeling that there should be a "-1, Martyr Complex" mod option. But of course, this only feeds said complex. And that's the problem with a lot of moderations - sometimes its more effective responding with why an opinion might be missing something.

Having said that - responding is also only so effective. The linked example works well to demonstrate it. In the responses this down-modded post got, someone took the time to point out to you that "free" was not an issue of cost. Your entire post was derailed. Now you moan and groan about the injustices visited upon you due to anti-Microsoft sentiment. You're either refusing to listen, are slow on the pick-up, or are intentionally trolling.

Back on point...

Sure - there's a fair amount of anti-Microsoft sentiment here. I like it. You'll have to excuse me if I don't gush about Microsoft's latest venture like so much of the rest of the industry. But I will agree that we must be guarded to ensure we don't allow ourselves to get too carried away.

Next time - have the conviction in your opinion to state it without the cheap mod / martyrdom ploy.

Re:What About Microsoft?

[crumley]

Right, Firefox is a distant second in browsers, while microsoft is a distant third in search.

Re:What About Microsoft?

[mpcooke3]

Well if you look closely at the picture in the article you can see that Google is the only one to make it. MSN and Yahoo are just disposable booster rockets.

And yes, firefox market share is small.

Re:What About Microsoft?

[Anonymous Coward]

Sometimes I get this petty little feeling that there should be a "-1, Martyr Complex" mod option.

And I'm starting to wish there was a "-1, psychoanalysis bullshit" mod option.

What's wrong with pointing out that anything pro microsoft is bad on Slashdot? What's wrong with providing proof of that sentiment?

Re:What About Microsoft?

[_Sprocket_]

What's wrong with pointing out that anything pro microsoft is bad on Slashdot? What's wrong with providing proof of that sentiment?

Nothing. But claiming "I bet this gets down-modded" and then showing a previous example of a down-mod doesn't add anything to the point. Especially if that down-mod was shown to be accurate. Go back and re-read what I wrote.

It should be stressed that the "I bet I get modded" schtick applies to anything, not just the Slashdot-criticisms.

Re:What About Microsoft?

[pboulang]

That is completely new behavior on the part of the editors. In the past they have always had the utmost journalistic integrity and wrote the book on balanced reporting. Maybe they are being held captive kidnappers and this is their cry for help. You should call the FBI to come in and investigate.

Re:What About Microsoft?

[Phu5ion]

Its not the fact that we ignore the good things that Microsoft do...its the fact that that Microsoft rarely do a sumthing good that we get distracted by the bad things its doin....take XP for example....everytime Microsoft do something good, i still can't help but want to throw my computer out the window because XP is always crapping out on me. plus the fact that the good things that Microsoft *rarely* do are always so small and insignificant that no one cares.

Like, the Office grammar check crapping out on you?

Re:What About Microsoft?

[homey of my owney]

> shouldn't news be subjective not objective even if it is only for nerds? Umm, I think we all got your point, but Subjective would make the omission fine and expected. It's Objective we want...

Re:What About Microsoft?

[Anonymous Coward]

I was the one who submitted this article and the reason MS was not mentioned is that they were not involved in the video interview that was released. I could have been a bit more specific with regards to the description though, so as not to ignore MS involvement in the project.

So the quote from TFA is from Microsoft and they use Microsoft all over TFA that you linked. So you admit to not reading TFA at all, just watching the video and you're modded as +5 informative? Sounds just like typical moderator behavior on this site.

Re:What About Microsoft?

[HoldenManiaC]

no...thats the firefox spell check :D

The Department Raises a Valid Point

[eldavojohn]

from the my-robots.txt-thanks-you dept.

As we learned a short while ago, this initiative will make it that much easier for bots to detect what content a site has to offer. Is this good or bad for the end users of the internet--will it just increase the incentive for spiders and bots to crawl sites? What is the real purpose of this collaboration? To me it looks like an attempt for the search engines to get content providers to make the search engine's job that much easier.

Re:The Department Raises a Valid Point

[Rob T Firefly]

What is the real purpose of this collaboration? To me it looks like an attempt for the search engines to get content providers to make the search engine's job that much easier.

That makes sense, though. The whole reason for the web is the content provided by content providers, and they need the search engines to know what they have to offer just as badly as the search engines need the content to search for. It's all symbiotic, and it is just logical that one side is willing to help the other do something that ultimately helps both sides.

Re:The Department Raises a Valid Point

[kfg]

To me it looks like an attempt for the search engines to get content providers to make the search engine's job that much easier.

Yeeeeeeeeeeeah, and. . .?

KFG

Re:The Department Raises a Valid Point

[garcia]

Is this good or bad for the end users of the internet--will it just increase the incentive for spiders and bots to crawl sites?

I've been using Google's Sitemaps program for quite some time. I don't want the spiders crawling old and pointless content when there is new and more relevant stuff available for them to display to end users. Why would it increase spidering when they are being specifically told what and how important something is to spider?

I have noticed a significant decrease in the overall spidering of my site (thank god) but more targeted spidering, especially after Google is notified that I have a new sitemap available.

Re:The Department Raises a Valid Point

[juiceCake]

To me it looks like an attempt for the search engines to get content providers to make the search engine's job that much easier.

Which is wonderful. Web developers can use a standardized file to help optimize search engine support, makng their job that much easier, rather than developing these types of guidance or sitemap files separately for each search engine. Afterall, most of my clients, and I'm hazarding a guess here I know, but most clients want their site to be able to be found via a search engine from time to time. It benefits everyone it seems. Nice attempt indeed.

wow

[forrestf]

i can see it now, GooYahoo

Re:wow

[JCOTTON]

YAGOO

Re:wow

[harrythefish]

YaGoo(r)! Surely? Sounds like a Web 2.0 Bukkake party

Re:wow

[morgan_greywolf]

i can see it now, GooYahoo

Hey! That's MSGooHoo to you, bub!

Re:wow

[MORB]

Sounds like something that could have came out of Renault's car model name generator (Twingo, Kangoo, Logan, Koleos...)

Re:wow

[forrestf]

hahha, Mircosoft GooHoo Live!

Re:wow

[FuzzyFox]

Yahoogle!

Re:wow

[Supergibbs]

I like Yahgoosoft

Re:wow

[robbywalker]

If only a few more engines get involved we could have YahooglycosexcitavistapileSNbotomi

Re:wow

[reifman]

GooHoo?

Spinning

[CheeseburgerBrown]

This is obviously a tech-community spin to avoid tainting the news from the start.

Like if you were hosting a conference on global peace you might keep quiet about Dubya being a keynote speaker.

4th?

[Anonymous Coward]

So who is the fourth horseman of the apocalypse?

Typical Slashdot

[Volcane]

How typical of slashdot to not mention that Microsoft is also involved in the site.

Over compilicated!!!

[hey]

Why not just have a link from your main page to an HTML sitemap that links to all pages on your site.
Nice and easy. And usable by people and crawlers.

Re:Over compilicated!!!

[Anonymous Coward]

You don't manage a big site, do you.

what about robots?

[archen]

Lets say I have robots.txt set to deny everything, but I submit some pages to this thing for indexing. Does the spider obey robots.txt or what was submitted? Actually I'd find it handy to keep the spiders the hell off of my site but just submit a couple pages, but I don't see how that system could be trustworthy at all. Is it just me or is this just another form of meta tags?

Re:what about robots?

[Wesley Felter]

When robots.txt and the sitemap conflict, robots.txt takes precedence. This is because robots.txt is a hard restriction and sitemaps are just a hint.

Subjective...

[scombs]

Example Code from: http://www.sitemaps.org/protocol.html/

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9 ">
<url>
<loc>http://www.example.com/</loc>
<lastmod>2005-01-01</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</urlset>

Is it just me, or does the priority tag seem really objective and arbitrary? One webmaster's .5 could be another's .8...

Re:Subjective...

[Anonymous Coward]

Is it just me, or is the tag lacking its closing tag (or alternatively declaration of content-less)? They're even too stupid to check their very own samples against their very own DTDs!

Re:Subjective...

[LanMan04]

That's exactly right. The priority tag only applies to pages on your site, and is a relative measure. So (I would assume) that assigning a priority range of 0 to 0.5 would be the same to the search engine as a range of 0.5 to 1.

In other words, assigning a priority of 1 to all your pages will not affect their ranking vs. *other* sites that appear in the search results, only vs. other pages on your site. And if they're all 1, then you're telling the crawler that they're all equally important, just as if you had assigned them all a value of .5 (or anything else).

Re:Subjective...

[Zarniwoop_Editor]

It is only a priority relative to your own pages.
So I guess you could say it is very arbitrary but it is only used as a hint to show how the site owner would prefer his content to be spidered. If you had 10 million pages on your website but there were a few hundred you really wanted the spiders to be interested in then you would assign them a higher priority in your sitemap. It is relative to your own site only so it's ok that it works that way I think.

Re:Subjective...

[EqualSlash]

How could this be XML when they left out the closing tag?

Re:Subjective...

[scombs]

Wow, I'm awesome. The broken link is my own 1337 skills. Let's try this again: http://www.sitemaps.org/protocol.html Mmm, closing tags.

Re:Subjective...

[8-bitDesigner]

Actually, I believe that's priority within the site. So, for example, your homepage might have a "0.8" your "Contact Us" page, might have a "0.5", your "News" section might be a straight "1.0", and your privacy policy a "0.2".

Re:Subjective...

[Anonymous Coward]

The priority tag only applies to pages on your site, and is a relative measure. So (I would assume) that assigning a priority range of 0 to 0.5 would be the same to the search engine as a range of 0.5 to 1.

OK, but what if I go to 1000? 10000? 100000000000000000000000?
I think I see a business opportunity here.

Re:Subjective...

[Pollardito]

You see, most blokes, you know, will be marking all their pages at ten. You're on ten here, all the way up, all the way up, all the way up, you're on ten on your priority. Where can you go from there? Where?

I don't know.

Nowhere. Exactly. What we do is, if we need that extra push over the cliff, you know what we do?

Put it up to eleven.

Eleven. Exactly. One higher.

Re:Subjective...

[Jeff Molby]

In other words, assigning a priority of 1 to all your pages will not affect their ranking vs. *other* sites that appear in the search results

Are you sure?

If two pages from different site are determined to be of approximately equal relevance to the search, couldn't a search engine pick a favorite by using the internal priority ranking?

Wouldn't a page on widgets be more relevant coming from a widget-maker (who would give it a higer internal priority than his gadget pages) than a similar page coming from a gadget-maker (who would give it a lower internal priority than his gadget pages)?

I'm not saying this is necessarily a bad thing. It would probably yield better search results. But if things do go that way, I bet you'll find that content makers will quickly abandon the use of internal priorities.

That's only because

[harrythefish]

Microsoft are evil and they suck.

GooHoo! (Since GooTube isn't YouGoo)

[Tigwyk]

I'm for once interested in a joint project between search engines. If Google and Yahoo! can play nice, and Microsoft is mentioned as being part of this Web 2.0 menage-a-trois, perhaps something interesting will come of this. But right now it does just look like they want to make it easier to index pages. I've been attempting to submit my sitemaps to google for ages and have yet to see my sites listed when searching for my keywords, but perhaps that'll change in the future if this works out. Guess I'll keep my eyes and ears open.

Text Browsers

[poindextrose]

It's too bad that the specification only covers information relevant to search engines.

How about a <description> tag? I would take great interest in a sitemap specification that gives me enough information to navigate major parts of a site with a viewer plugin (of some sort) in a web browser.

There's nothing worse than fumbling around navigating page after page when the web server is slow, the pages are image- or ad-heavy, or the navigation on the page just plain sucks.

Re:Text Browsers

[dsaraujo]

You just wrote the principles of meta tags in web pages. They didn't work.

Re:Text Browsers

[poindextrose]

The difference here being that this would be the equivilant of consolodating that information into one file for all the major sections of your site, permitting easy navigation with a small download.

Don't be lazy

[Anonymous Coward]

Write (or snag) some generic xsl and use wget and xsltproc. All my sites have used xltproc via cron to regenerate a html sitemap from the google sitemap xml since the sitemap thing was launched.

If you're using a text mode browser, this should be obvious...

How do I submit a sitemap to Yahoo/Microsoft?

[lanfor]

So how can I submit my sitemap to Yahoo! and Microsoft/search.live.com? FAQ says something about sending a HTTP request to /ping?sitemap=http%3A%2F%2Fwww.y oursite.com%2Fsitemap.xml, but it doesn't say what are searchengine-specific urls to use.

Lukasz
Hikipedia - free database of hiking trails

Re:How do I submit a sitemap to Yahoo/Microsoft?

[FooAtWFU]

For Google, at least, there is this Google Sitemaps program which seems to predate this particular endeavour (I've been using it for months now). Yahoo... I dunno.

Re:How do I submit a sitemap to Yahoo/Microsoft?

[imroy]

Good question. I've been using Google's webmaster tools for a little while now, generating my own sitemap. Even though I have a pretty small site, the information is still interesting and useful. I tried the /ping URL on search.yahoo.com and got redirected (assuming a bad URL). I just tried it with search.live.com and got a 200 response but the text says 'Bad format while processing ping.' (Trust Microsoft to fuck up the HTTP standard). So I don't know what to do. Hopefully the FAQ will be updated to be more specific. So far it looks like it's mostly PR and little substance.

Re:How do I submit a sitemap to Yahoo/Microsoft?

[blacksmith_tb]

Worked for me by using

wget http://search.yahoo.com/ping?sitemap=http%3A%2F%2F www.mysite.com%2Fsitemap.gz

and

wget http://search.live.com/ping?sitemap=http%3A%2F%2Fw ww.mysite.com%2Fsitemap.gz

Though I have no idea if they're actually going to process it...

Re:How do I submit a sitemap to Yahoo/Microsoft?

[Anonymous Coward]

I am confused about which URL to use, too. They (G, Y, MS) should make clear which URL to use on their own web sites.

But as for the FAQ on www.sitemaps.org, I guess they are trying to make www.sitemaps.org neutral and open to any other search engines. In addition, I guess if they include their URLs there, some people will shout at Slastdot.org, "This is an unfair advertisement of the three major search engines in disguise of a new open standard!"

I for one...

[icedcool]

Google is going to take over the world... and if there company record is any indication of their rule, I for one welcome our new advanced search indexing overlords.

Parse error in example?

[caluml]

From the site:

Sample XML Sitemap

The following example shows a Sitemap that contains just one URL and uses all optional tags. The optional tags are in italics.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9 ">
<url>
<loc>http://www.example.com/</loc>
<lastmod>2005-01-01</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</urlset>

Are they missing a <url>

Why is this more than what robots.txt can do?

[Conficio]

Many here ask why this is more than robots.txt. For one it offers to add URLs that are driven by databases and parameters. Thing that the SEs do not index too well. It also adds last updated stamps and priority for re-visit.

Why is that important? So if I have one page where I always post the latest news, I can have the spider revisit every hour, so it get indexed ASAP. However the spider can go easy on the rest of my site otherwise. I also can train that spider for a burst, if I have for example an ongoing live event and post results ASAP.

Google was on to this already for a while. however Yahoo's facilities where not so comprehensive and MSN was missing altogether. Now we get a chance to create one single format and have all (big) SEs read it and may be the secondary spiders will catch on too.

I find it only said that they did not incorporate the ROR Resources of Resources RDF framework. I'm also missing a discovery mechanism, such as an extension to the robots.txt or a meta tag (or link rel="...") in the home page of a site.

Another use of this would be to download a full site, as you now know where is starts and what belongs to it.

Overall this is a good thing, where I come from.

This is New? Django has supported this for a while

[Derivin]

The Django web framework added support for 'google sitemaps' over a month ago. Google anounced the details of sitemaps over 3 months ago. Django Sitemaps: http://www.djangoproject.com/documentation/sitemap s/

Missing a few Sprockets.

[Anonymous Coward]

Nothing. But claiming "I bet this gets down-modded" and then showing a previous example of a down-mod doesn't add anything to the point. Especially if that down-mod was shown to be accurate. Go back and re-read what I wrote.

The original post was modded as flamebait. If it was modded off-topic, fine. It was modded as flamebait because I said something nice about Microsoft. Go back and re-read what I wrote & stop being so blind about basic anti-Microsoft sentiment on Slashdot.

Re:Missing a few Sprockets.

[_Sprocket_]

The original post was modded as flamebait. If it was modded off-topic, fine. It was modded as flamebait because I said something nice about Microsoft. Go back and re-read what I wrote & stop being so blind about basic anti-Microsoft sentiment on Slashdot.

You're right. One of the better mods should have been "off-topic" as most of the post was. The closest you seemed to come on that one was:

I know it's not open source and the license I got from them was super flaky. But in the interests of being able to use every technology available to me, I'll learn .NET. I'm more marketable to employers and, hell let's face it, even coding Microsoft libraries can be fun.

So you'll find some of their free (yes, free) software on my machine. Now, I had to pay for XP to be able to install that ... but I still feel like I paid for XP & not the Express tools.

And that's where it starts coming close to flamebait. The only question would be if you're intentionally trying to confuse the "free" meaning or not. Trolls willfully do it.

In any case, your post is a poor example of retribution for saying nice things. You even pointed out yourself that you should have gotten "off-topic" mods. I suppose if you did, it simply would have been more anti-Microsoft sentiment and not because of the post itself being poorly conceived.

Re:Missing a few Sprockets.

[Raenex]

My God but you are a karma whore. You resort to posting anonymously for follow-ups? Is there some prize for having a high karma history that I don't know about? Pathetic.

Re:Missing a few Sprockets.

[eldavojohn]

News flash, this site is about discussions. People set anonymous cowards to -7 so they don't have to read bullshit like the above. It isn't about being a karma whore, it's about making trite arguments invisible. It's also about being able to tell someone to go fuck themselves when they called you pathetic.

Understand genius?

Honestly, what did the above comments (or your own even) add to the fucking relevant discussion?

My God but you are a karma whore. You resort to posting anonymously for follow-ups? Is there some prize for having a high karma history that I don't know about? Pathetic.

My God but you are stupid. You resort to replying to anonymous follow-ups? Is there some prize for taking the conversation off-topic? Pathetic.

Re:Missing a few Sprockets.

[Raenex]

I replied to an anonmyous coward that wasn't really anonymous, and that was following the conversation. You're a karma whore, and it was fun to call you on it.

Re:This is New? Django has supported this for a wh

[deryckh]

While it's true that Django has support for the sitemaps protocol and sitemaps are nothing new for Google, Yahoo and MSN's support of the protocol is new.

Not Valid XML!

[rHBa]

Call me an XML nazi but the example usage has unclosed tags:

http://www.sitemaps.org/protocol.html

Re:Not Valid XML!

[insanecarbonbasedlif]

Really? It looks like it's all there (and in all the other examples they have posted):

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9 ">
      <url>
            <loc>http://www.example.com/</loc>
            <lastmod>2005-01-01</lastmod>
            <changefreq>monthly</changefreq>
            <priority>0.8</priority>
</urlset>

Re:Not Valid XML!

[rHBa]

The tag is left unclosed.

Re:Not Valid XML!

[rHBa]

Damn, that should read "The tag is unclosed"

Invalid XML

[rHBa]

As an XML specification that is likely to be used by people who aren't experts don't you think it would have been a good idea to used *valid* XML in the example usage?

Google Sitemaps

[Supersonic1425]

Google has had this for a while now. I had noticed that development has been healthy recently. whereas before it was a relatively unnecessary tool, now it's actually useful.

if it's as useful as Google Sitemaps, then I'm happy with today's news. the protocol does look pretty similar (and by pretty similar, I mean the XML structure is virtually identical). I'm guessing porting Google Sitemaps over to this new one will be painless.

PHP Class for Generating SiteMaps

[reifman]

This fall, I released free source code for people to use a PHP Class to generate SiteMaps for Google - and it seems like the standards group adapted Google's format. The code is perfect for dynamic database driven sites that can't readily use perl-scripts that sometimes perform this task. http://www.idealog.us/2006/09/google_sitemap_.html

How much is enough?

[Anonymous Coward]

Geez. First, there were meta tags. Then robots.txt. Then this. Plus proprietary indexing-control tags. Plus ridiculous weight factors (like, for example, giving "title" huge weight index). Plus citation index. How much is enough?

What's the problem? Well, let's see. Most of these things have nothing to do with the actual content of the website. Hence, creating bot-friendly website is not directly related to creating user-friendly website anymore. In fact, this two factors often conflict.

Where's theirs

[Anonymous Coward]

They're trying to sell the sitemap.xml idea, yet they themselves have no sitemap.xml file.

Re:Where's theirs

[Anonymous Coward]

Note that the file is not necessarily named as "sitemap.xml".

This may help AJAX apps to be more search-friendly

[piprog]

I just mused about the search-unfriendliness of AJAX apps yesterday and how that could be solved and today the big three are banging (almost) the same door. What do you think how could we go about solving the issue?

Sorry, LanMan04

[Jeff Molby]

I misread your post; I'm off-topic. Still, I can't imagine why a content provider would use different priorities. It could only hurt the rankings of some of the low-priority pages. Unless, of course, the search engines gave your high-priority pages an equivalent boost.

Lard Thunder 'n Jesus!

[ShieldW0lf]

Can you believe de nerve of dem bys down dere me son?

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9 ">
<url>
<loc>http://www.example.com/</loc>
<lastmod>2005-01-01</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</url>
<url>
<loc>http://www.example.com/catalog?item=12&amp;de sc=vacation_hawaii</loc>
<changefreq>weekly</changefreq>
</url>
<url>
<loc>http://www.example.com/catalog?item=73&amp;de sc=vacation_new_zealand</loc>
<lastmod>2004-12-23</lastmod>
<changefreq>weekly</changefreq>
</url>
<url>
<loc>http://www.example.com/catalog?item=74&amp;de sc=vacation_newfoundland</loc>
<lastmod>2004-12-23T18:00:15+00:00</lastmod>
<priority>0.3</priority>
</url>
<url>
<loc>http://www.example.com/catalog?item=83&amp;de sc=vacation_usa</loc>
<lastmod>2004-11-23</lastmod>
</url>
</urlset>

Take it from me, me bys, dere's no better place fer a vacation den up e're on de rock where de liquors hard and de sluts are everywhere!

I wish a more "open" protocol

[Anonymous Coward]

Ok, I welcome the new standard to improve search (crawling) quality without increasing (possibly reducing) load on web servers.

However, with the current protocol, a site owner can only send pings to the search engines she knows. Probably Google, Yahoo! Search and Live Search will cover more than 99% now (I am not sure), but it is not very satisfactory since it would block out other minor (or new) engines. (I am the same Coward as #16895296.)

I hope "Sitemap protocol 0.91" will include two additional features:
1. Autodiscovery (like feeds)
2. Some way for a search engine to "subscribe" the sitemap (like mailing lists, not like USENET groups; I am new to feeds and I do not know whether "subscription" of feeds has the same meaning as "subscription" of mailing lists).