On the reseller server I had access to grey-listing and SpamAssassin to filter mail as it arrived and I got maybe 2 or 3 a week, but the day after I moved onto the vps this shot up to between 20 and 50 a day!  I have my emails pushed straight to my phone as well so this fast became a bit of a pain.

My VPS did come with SpamAssassin pre-installed as part of Plesk but due to the licence I have installed I couldn’t use it so this caused two problems, 1) emails are not filtered for spam and 2) SpamAssassin is using up precious memory while not actually helping.

The first thing I did was to turn off SpamAssassin, it was using a lot of memory as even tho it wasn’t filtering any mail  it was still scanning every message that arrived on the server, On advice of my hosting providers tech support I decided not to un-install it in case it caused problems so I just stopped the service from running and then disabled it from auto starting on system reboot to preventing it restarting if I ever need to reboot the server.

I then tried to find the grey-listing software used on my old reseller server, this seemed to work really well so I wanted it on my server too. Support told me they were using a script put together by Brent Meisher written to work specifically with Plesk but  unfortunately the repo had been removed and I couldn’t download the files.

I was speaking with a support engineer about it and he mentioned a new tool he was playing with called SpamDyke and how it was looking promising as a replacement to their current grey-listing solution, so I decided to give it a try.

The good news is, that since I started writing this blog post, SpamDyke has been added into the atomic repositories, so to install it is as simple as using apt get or yum. Configuration is just as simple, the default settings are actually quite effective and I’ve been using them for two weeks and haven’t received a single piece of spam.

All the information you need can be found on the SpamDyke website and the README file is really useful.
This is something I defiantly recommend to everyone who is running their own mail server.

The first is called ‘Fetch as Googlebot’ as allows you to view the information that Googlebot retrieves when it visits ont of the pages on your site. This is what people, webmasters and SEO ‘experts’, are most likley to be interested in. At fist glance it appears to be just the source code of a page, and while yes it is, it does also contain the server headers that were detected, so you can use the two to determin if googlebot sees your page as you want it to.

The second feature is a Malware status page, this feature has been around for a while outside the webmaster tools, if you knew where to find it, and its a useful tool is your site does contain some dodgy code, its good to see this is now part of the webmaster tools and easier to find.
There is a good post on the Google Security Blog that can tell you more about Malware detection so rather than write about it myself, I’ll refer you to it.

It includes not just a new design but also a re-think on all sections of the current webmaster tools, how they are grouped together, how you navigate between them and also the information that they give you.

Highlights Include

• One-stop Dashboard
• More top search queries:
• Sitemap tracking for multiple users:
• Message subscription:
• Improved menu and navigation:
• Smarter help:
• Removal of the enhanced Image Search option:

For a full list of updates and more visit Googles ‘Whats New’ page or read the Webmaster Central blog post.

This is a traditional sitemap, one that you will find on most sites across the internet, there is nothing wrong with this and does its job very well, it lists all my pages within the site allowing people or search engine spiders to navigate around and find the content they are looking for.

There is however a new growing trend, well… I say new but it has been around for a while, but more and more sites seem to be using it recently which has resulted in the “modern sitemap” phrase being termed by the web design and developer community. I’m talking about footer sitemaps, instead of having a single page listing all the pages within a site, you have a smaller “related links” sitemap in the footer of every page on a site, a few examples for you;

• http://www.mozilla.com/
• http://www.apple.com/mac/
• http://www.rednoseday.com/

I can see the benefits of using a sitemap in this way, the first that comes to mind is that you associate the links with some content, search engines can use the content to grab keywords and then serve your links if the keywords are used in a query, this is the big failure of the traditional plain text listing sitemap in terms of SEO, with no content to associate the links to means search engines cant add any weight to the links. Other benefits of this type of sitemap range from providing your visitors a list of related pages they may find useful to simply a way of padding out the footer of your site if you don’t have much content.

As with anything there are downsides to having a sitemap like this, the main one being there isn’t a single point listing all pages within the site, in some cases you may end up overloading a page with links, a big no-no for SEO, not to mention some website designs simply wouldn’t look right with a sitemap in its footer.

I think that while the benifits of having the associated content shouldn’t be ignored, this type of sitemap only works with certain site designs so trying to get a nice balance is what you should aim for, combining the two styles of sitemap to allow your visitors to easily find the content they are looking for. Having a list of all the pages in your site is a valuble tool and allows search engines to index all the pages on your site, having “related links” on a page allows you to show visitors content they may be interested in but wouldnt normally find.

A “modern sitemap” to me is one that users find useful, this could be the traditional sitemap or its younger brother the related links sitemap, the important thing is your visitors can find the content they are looking for.

“A rival to google” is a big claim to make and this has been claimed by the makers of every search engine released since the boys and girls at the mountain view chocolate factory positioned themselves top of the pile, remember cuil.com anyone?? So we will have to wait and see what happens with Wolfram Alpha, but i do hope they come up with a better name…

you can read the blog post @ blog.wolfram.com
and view the new search engine (you cant search yet though) and signup for latest news @ www.wolframalpha.com

The best way I have found is to alert the search engine to an update, you can do this by sending a ping request to the search engine via an api. This api alerts the index that new content is available and to instruct the crawler to visit the site,  below are the uri’s of the top search engines sitemap ping api’s

• Google – http://www.google.com/webmasters/tools/ping?sitemap=
• Yahoo – http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=
• Live.com – http://webmaster.live.com/webmaster/ping.aspx?siteMap=
• Ask.com – http://submissions.ask.com/ping?sitemap=

Just append the url with your sitemap location and off you go. I have modified my sites cms to send a request via cURL to the api’s when ever a new page is added or deleted from my site. This ensures search engines are alerted to any changes in my sitemap.