This week, Google took that idea and extended it into an independent online tool, making it available for anyone on any browser. It uses a version of WebKit installed on a server to run the test (via the Chrome version of Page Speed Im guessing) to test your site and suggest improvements.

It seems to work in exactly the same way, and suggests the same kinds of things to improve your score, and has the same issues I pointed out last time, in that it marks you down for incorrect caching on Google Services you include on your page (Analytics in my case).

When I first ran the test on my site, I had dropped to 76/100, this was down to the javascript includes I had slowing things down, and the image caching wasn’t long enough. A couple of tweaks later and I’m now back up to 96/100 which is 3 above where I got to last year.

Read the Google post or start testing your site, please note tho, that as its being hosted on the Google Labs site, this could (as many have done before) disappear without warning. If it does I’ll update this post to reflect that.

This new system basically allows them to push anything Googlebot finds into the main index in seconds rather than weeks.

Previously Googlebot would crawl the web, find a site, then submit its content to another system for processing. This other system would then sort and process the content and work out what its all about. It would then pass it to another system for inclusion in the index. This was a time consuming process and was also done in batches, which means your site would have to wait for all the other sites in the same batch to be processed before being submitted to the index.

The new system is different, rather than doing things in large batches, it will work on smaller portions, more portions at the same time and submit them all to the index straight away, allowing users of the search engine to see information much much quicker, and this is just the start!

Image copyright of Google

There are several rumours floating around about what lies in store next for Google users and their shot of caffeine.  But for now we will have to wait and just enjoy our new “real-time” search results.

You can real the full article on the Google Webmaster Blog

First thing the post suggest is to check out Site Performance in Webmaster tools, so off I went. When this first appeared in the Labs section of webmaster tools it was updating pretty regularly and was quite useful and as the graph shows helped me tweak a few things and improve the load time of the server. But it hasn’t been updated in a while now so isn’t very relevant or useful to me.

Next thing it suggests is to install Page Speed for Firebug and test my site. So thats what I did and I got Page Speed Score: 93/100! which is pretty impressive. So I thought next, 7% to go, what can I do to get to 100/100. Nothing it seems… All the suggestions the tool gave me were unsolvable and prevent anyone from getting top marks.

The first suggestion is to Leverage browser caching,
the file it advises me need to do this too is “http://www.google-analytics.com/ga.js” which I cant do anything about.

The next suggestion is to Defer loading of JavaScript, “66.4% of the JavaScript loaded by this page had not been invoked by the time the onload handler completed” is smugly informs me.
- http://gavtaylor.co.uk  24 functions uncalled of 25 total functions
- http://www.google-analytics.com/ga.js 154 functions uncalled of 243 total functions.

On closer examination the functions it reports on my site are actually from Firebug, which must be open for the test to run? catch 22 there!
And the analytic code again is from Google and I cant do anything about that.

The final suggestion is to Use efficient CSS selectors,
http://gavtaylor.co.uk/core/styles/styles.css has 12 very inefficient rules, 27 inefficient rules, and 0 potentially inefficient uses of :hover out of 101 total rules.

This I can do something about and when I get a moment I will be re-coding my CSS file. But this wont improve my results more than 1 or 2 points.

So this unreachable target of 100/100 appears to be impossible. I could remove the Google code from my site which would resolve the Google plug-in issue but I cant run the test without having Firebug open so there will always be 24 out of 25 uncalled functions reported keeping me away from that 100/100 score.

So… will I be sitting in that tree, doing things I should not be?… No, not until they fix the issues with their testing tools anyway…

Update 4/4/11:

On Thursday (31st March), Google released an online version of their page speed tool, This new test works in the same way to the Firebug plugin but uses a version of webkit installed on a server. I have written a quick follow up post with links and my initial thoughts here.

not bad results there really, not much to done on searches for my name, going to focus on php developer I think

interestingly on the web search for ‘gav taylor’, my twitter feed came top.
again searches for my name do quite well, confirms I need to improve seo for ‘php developer’

Will be interesting to see where I end up for these searches in Feb 2011

On the reseller server I had access to grey-listing and SpamAssassin to filter mail as it arrived and I got maybe 2 or 3 a week, but the day after I moved onto the vps this shot up to between 20 and 50 a day!  I have my emails pushed straight to my phone as well so this fast became a bit of a pain.

My VPS did come with SpamAssassin pre-installed as part of Plesk but due to the licence I have installed I couldn’t use it so this caused two problems, 1) emails are not filtered for spam and 2) SpamAssassin is using up precious memory while not actually helping.

The first thing I did was to turn off SpamAssassin, it was using a lot of memory as even tho it wasn’t filtering any mail  it was still scanning every message that arrived on the server, On advice of my hosting providers tech support I decided not to un-install it in case it caused problems so I just stopped the service from running and then disabled it from auto starting on system reboot to preventing it restarting if I ever need to reboot the server.

I then tried to find the grey-listing software used on my old reseller server, this seemed to work really well so I wanted it on my server too. Support told me they were using a script put together by Brent Meisher written to work specifically with Plesk but  unfortunately the repo had been removed and I couldn’t download the files.

I was speaking with a support engineer about it and he mentioned a new tool he was playing with called SpamDyke and how it was looking promising as a replacement to their current grey-listing solution, so I decided to give it a try.

The good news is, that since I started writing this blog post, SpamDyke has been added into the atomic repositories, so to install it is as simple as using apt get or yum. Configuration is just as simple, the default settings are actually quite effective and I’ve been using them for two weeks and haven’t received a single piece of spam.

All the information you need can be found on the SpamDyke website and the README file is really useful.
This is something I defiantly recommend to everyone who is running their own mail server.

The first is called ‘Fetch as Googlebot’ as allows you to view the information that Googlebot retrieves when it visits ont of the pages on your site. This is what people, webmasters and SEO ‘experts’, are most likley to be interested in. At fist glance it appears to be just the source code of a page, and while yes it is, it does also contain the server headers that were detected, so you can use the two to determin if googlebot sees your page as you want it to.

The second feature is a Malware status page, this feature has been around for a while outside the webmaster tools, if you knew where to find it, and its a useful tool is your site does contain some dodgy code, its good to see this is now part of the webmaster tools and easier to find.
There is a good post on the Google Security Blog that can tell you more about Malware detection so rather than write about it myself, I’ll refer you to it.

It includes not just a new design but also a re-think on all sections of the current webmaster tools, how they are grouped together, how you navigate between them and also the information that they give you.

Highlights Include

• One-stop Dashboard
• More top search queries:
• Sitemap tracking for multiple users:
• Message subscription:
• Improved menu and navigation:
• Smarter help:
• Removal of the enhanced Image Search option:

For a full list of updates and more visit Googles ‘Whats New’ page or read the Webmaster Central blog post.

This is a traditional sitemap, one that you will find on most sites across the internet, there is nothing wrong with this and does its job very well, it lists all my pages within the site allowing people or search engine spiders to navigate around and find the content they are looking for.

There is however a new growing trend, well… I say new but it has been around for a while, but more and more sites seem to be using it recently which has resulted in the “modern sitemap” phrase being termed by the web design and developer community. I’m talking about footer sitemaps, instead of having a single page listing all the pages within a site, you have a smaller “related links” sitemap in the footer of every page on a site, a few examples for you;

• http://www.mozilla.com/
• http://www.apple.com/mac/
• http://www.rednoseday.com/

I can see the benefits of using a sitemap in this way, the first that comes to mind is that you associate the links with some content, search engines can use the content to grab keywords and then serve your links if the keywords are used in a query, this is the big failure of the traditional plain text listing sitemap in terms of SEO, with no content to associate the links to means search engines cant add any weight to the links. Other benefits of this type of sitemap range from providing your visitors a list of related pages they may find useful to simply a way of padding out the footer of your site if you don’t have much content.

As with anything there are downsides to having a sitemap like this, the main one being there isn’t a single point listing all pages within the site, in some cases you may end up overloading a page with links, a big no-no for SEO, not to mention some website designs simply wouldn’t look right with a sitemap in its footer.

I think that while the benifits of having the associated content shouldn’t be ignored, this type of sitemap only works with certain site designs so trying to get a nice balance is what you should aim for, combining the two styles of sitemap to allow your visitors to easily find the content they are looking for. Having a list of all the pages in your site is a valuble tool and allows search engines to index all the pages on your site, having “related links” on a page allows you to show visitors content they may be interested in but wouldnt normally find.

A “modern sitemap” to me is one that users find useful, this could be the traditional sitemap or its younger brother the related links sitemap, the important thing is your visitors can find the content they are looking for.

“A rival to google” is a big claim to make and this has been claimed by the makers of every search engine released since the boys and girls at the mountain view chocolate factory positioned themselves top of the pile, remember cuil.com anyone?? So we will have to wait and see what happens with Wolfram Alpha, but i do hope they come up with a better name…

you can read the blog post @ blog.wolfram.com
and view the new search engine (you cant search yet though) and signup for latest news @ www.wolframalpha.com

The best way I have found is to alert the search engine to an update, you can do this by sending a ping request to the search engine via an api. This api alerts the index that new content is available and to instruct the crawler to visit the site,  below are the uri’s of the top search engines sitemap ping api’s

• Google – http://www.google.com/webmasters/tools/ping?sitemap=
• Yahoo – http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=
• Live.com – http://webmaster.live.com/webmaster/ping.aspx?siteMap=
• Ask.com – http://submissions.ask.com/ping?sitemap=

Just append the url with your sitemap location and off you go. I have modified my sites cms to send a request via cURL to the api’s when ever a new page is added or deleted from my site. This ensures search engines are alerted to any changes in my sitemap.