Come on Google...

So having got the new Beetlebrow site live and looking great, and made a few tweaks to make it run faster and work better, I thought I'd better nip along to our Google webmaster account to upload the new XML sitemap and make sure the transition went as smoothly as possible.

The sitemap is built with a product we developed ourselves from a simple Limi template and a UML diagram. It's not complicated, but it doesn't need to be - it just lists everything on the site of the types of pages we want Google to index, while allowing us to specify pages to exclude. So far so good but...

There's always a but. This time it was that, for some reason Google didn't like the sitemap - it was missing a tag. How very strange. As this was the first time we'd used the new product, and I was also including it on a couple of recent sites we'd built - www.siefersharrison.com and http://www.richardmax.co.uk - I didn't know what kind of problems we might face with it, but I had to sort it out.

Looking at the sitemap in Firefox, it looked fine, and the W3C validator said it was well-formed XML, but then in Safari all I got was an empty set urlset tags. What's going on? Well it's obvious innit? Schoolboy error. I was logged in to the site as a manager in Firefox, but I wasn't logged in in Safari, and Google, being an anonymous visitor was getting the empty urlset tag experience too. And unless you specifically give the scripts which access all the configuration settings for the sitemap the same rights as a site manager, if you're not logged in, they can't read the settings. Dur.

So, having given myself a stern telling off, I gave the scripts the necessary permissions, and hey presto, the sitemap work in Safari, and Google stopped complaining. So I submitted the three sitemaps, for Beetlebrow, Richard Max and Siefers Harrison, and waited...

Twenty four hours later and all of Siefers Harrison and all of Richard Max are indexed, but Beetlebrow hasn't been touched. Another 24 hours after that, still zero files indexed for Beetlebrow. Is Google punishing me for my stupidity? If so, they're even cleverer than I thought. I know it can take days, weeks even, for Google to get round to indexing you, but to do our two client sites so fast and leave us hanging on? It's just not fair.

On the other hand, I did find out that the old Beetlebrow site was on page 1 for "web design and hosting" (see http://www.google.com/search?q=web+design+and+hosting), which is pretty darned cool, cos it's not like there's much competition between web design firms for good Google rankings or anything. On the other hand, that'll probably disappear when it does get round to indexing us. Curse it and crush it.

At the same time Richard Max is NUMBER 1 in Google for "planning lawyer" - not blimmin' bad for such a generic term - see http://www.google.com/search?q=planning%2Blawyer. So while I'm disappointed Google hasn't seen all of our lovely new Beetlebrow content yet, I'm exceedingly delighted with our SEO efforts on Richard Max. Hurray for Google!