Googlebot and Document.Write

With JavaScript/AJAX being used to place dynamic content in pages, I was wondering how Google indexed web page content that was placed in a page using the JavaScript "document.write" method. I created a page with six unique words in it. Two were in the plain HTML; two were in a script within the page document; and two were in a script that was externally sourced from a different server. The page appeared in the Google index late last night and I just wrote up the results.

Comments

Google request external JavaScript file?

[JAB Creations]

Check your access log to see if Google actually requested the external JavaScript file. If it didn't there would be no reason to assume Google is interested in non-(X)HTML based content.

Doesn't work; Good (kind of)

[The Amazing Fish Boy]

FTFA:

Why was I interested? Well, with all the "Web 2.0 technologies that rely on JavaScript (in the form of AJAX) to populate a page with content, it's important to know how it's treated to determine if the content is searchable.

Good. I am glad it doesn't work. Google's crawler should never support Javascript.

The model for websites is supposed to work something like this:

• (X)HTML holds the content
• CSS styles that content
• Javascript enhances that content (e.g. provides auto-fill for a textbox)

In other words, your web page should work for any browser that supports HTML. It should work regardless of whether CSS and/or Javascript is enabled.

So why would Google's crawler look at the Javascript? Javascript is supposed to enhance content, not add it.

Now, that's not saying many people don't (incorrectly) use Javascript to add content to their pages. But maybe when they find out search engines aren't indexing them, they'll change their practices.

The only problem I can see is with scam sites, where they might put content in the HTML, then remove/add to it with Javascript so the crawler sees something different than the end-user does. I think they already do this with CSS, either by hiding sections or by making the text the same color as the background. Does anyone know how Google deals with CSS that does this?

google.com/?q=slashdotting+in+google+dollars

[kale77in]

I think the actual experiment here is:

• Create a 6-odd-paragraph page saying what everybody already knows.
• Slashdot it, by suggesting something newsworthy is there.
• Pack the page with Google ads.
• Profit.

I look forward to the follow-up piece which details the financial results.

Re:google.com/?q=slashdotting+in+google+dollars

[Scarblac]

Exactly, this is the typical sort of fluff that Digg seems to love. As far as I know, Slashdot had avoided this particular type of adword blog post crap until now.

Re:How does document.write mess up your DOM tree?

[jesser]

Perhaps more importantly, document.write can't be used to modify a page that has already loaded, limiting its usefulness for AJAX-style features.

Re:If they weren't, then they're trying

[gregmac]

You have to also remember though, that often the content generated dynmically is going to be of no use to a search engine, it will often be user-specific - there's obviously some reason it's being generated that way.

And if pages are designed using AJAX and dynamic rendering just for the sake of using AJAX and dynamic rendering.. well, they deserve what they get :)