Monday, August 20, 2007

Search Engine Sitemaps and Google Dynamic Indexing

It was that long ago that Google, Yahoo, and Live (MSN) announced that they would cooperate on a sitemap protocol that would make it easy for webmasters to submit their xml sitemaps to the search engines. They collaborated on delivering some information at www.sitemaps.org. and, 'Ask' recently started 'looking for sitemaps' as well.

Regardless of what you've read, all four search engines can very easily be 'pinged' right from your browser address bar to alert them of changes to your sitemap (more on this in our next post).

Don't confuse your website's sitemap with the xml and/or text sitemap the search engines are looking for. This is the sitemap your visitors don't see but the search engines do. To some non-coder types, it may look confusing but there are plenty of websites and generators out there to help you.

While some banged their heads against the wall writing automated code, or manually delivering the sitemap info to the search engines in the early stages ... there have been many changes, and yes, it does help get you noticed.

While submitting a sitemap may not get you visited more often by the 'bots', it will help the search engines discover pages they didn't know about (and potentially rank them).

Some complain that delivering sitemaps is doing the search engines work for them. We see it as one of the greatest (and easiest) steps towards true 'White Hat' SEO since verification.

Here's some stuff you may not know ...
1) Be sure to reference your sitemap in the robots.txt file! The syntax is easy and in some cases, it really makes a difference.
2) It's probably a good idea to directly submit and validate your sitemap to Google's Webmaster Central. A slew of new features have been added including one that let's you mess with your robots.txt file ...
3) Yahoo's sitemap home is at their 'Siteexplorer' page and allows you to upload simple text files (If you aren't xml-ready yet?). It also allows you to validate your site.

MSN appears to be allowing submission via Moreover. While these seems a little funky and we've always ranked well with Live Search .... we'll keep you posted on the progress here.

Be sure to try the various browser based ping addresses (be sure it's for search engine sitemaps, not websites, blogs, or feeds!). Do at least one submission and see if you get 'sitemap accepted'.

In other news .... Recently, Matt Cutts posted a piece to his blog about Google indexing 'current event' pages in HOURS. Not just news stories, but webpages and blog posts. We found this totally fascinating so we tested it out. We posted a unique (and yes, relevant) news story to one of our blogs, pointed it at the story pages, pinged 'the majors' and checked back about four hours later. Honestly, we're not sure if it's because we used Blogger (owned by Google) or fed the RSS feed through Feedburner (owned by Google), but the story was indexed in the TOP-10 under numerous phrases. Very cool stuff with lots of possiblities for all of us.

That's another 'interim' post as we get this place ready for a clean and formal relaunch soon .....

No comments: