The importance of website speed to search engine crawling

  • Wednesday, 12th February, 2014
  • 21:26pm

As the internet is growing, one of the differentiating aspects of  content can be the speed that information is served and how well your web host is helping you to be found.
The importance of website speed to search engine crawling
This screenshot from Google Webmaster Tools, to the right, is a great demonstration of the importance of website speed to search engine crawling. The three sections are:

  • Pages crawled per day
  • Kilobytes downloaded per day
  • Time spent downloading a page (in milliseconds)

The importance of website speed to search engine crawling is emphasised in the Time spent downloading a page (in milliseconds) section that shows that up to mid November the site in question was responding at around 2000 milliseconds. At this time the "Pages crawled per day" were settled at around 50 and Kilobytes downloaded per day at around 500 Kb

After some server reconfiguration was dealt with towards the end of November which unfortunately made the response peak at around 6000 milliseconds. At this point Google didn't seem to make any difference between Pages crawled and Kilobytes downloaded.

From the start of December the reconfiguration had settled and was then responding between 700 and 800 milliseconds. This did get the attention of Google where the speed increase can be seen to have a dramatic affect on amount of information that Google wanted to know for the site.

When the speed increased the Pages Crawled rose from a steady 50 pages per day to over 340 which is a 680% increase. The Kilobytes downloaded per day is even better before the changes Google were interested in around 500 Kb per day then after the upgrade this rose to 3500 giving a huge 700% increase in the need for information about the site.

So, in conclusion, What is the importance of website speed to search engine crawling? Google gained a lot of interest in the site when the response time dropped below 1000 milliseconds.

I understand that this is a brief snapshot of a single site for a short period of time but it goes with reason that with the amount of content that is being produced and published on a daily basis one difference has to be the speed at which that information can get to the end-user: You the reader.

If you want to information on how we can help to improve your site you can just contact us and we will run through a full analysis of your site and how we can help.

What you are thoughts, let us know in the comments below.

« Back
Venatour Sports Travel

Encode have continually surpassed themselves in developing our new website. Their patience in learning and understanding what we needed from a website to provide the best service to our customers has been beneficial on so many levels. I would highly recommend them to anyone looking for intelligent solutions to their web, seo or business needs.

Lisa Cooley

Positive: Professionalism, Quality, Responsiveness, Value

Working with Neil is a dream; he's fast, efficient, responsive and full of website knowledge and advice. He won't baffle you with IT over load and will be honest about the best working solution.

Highly recommended!

Ellie Stevenson Author

Positive: Professionalism, Value

Encode and I had a very useful discussion about Wordpress and related issues at no cost to myself. He gave me some very helpful information and answered the questions I had.

I learnt quite a lot! Thank you.

Silvia Boev

Positive: Professionalism, Quality, Responsiveness, Value

I am very excited to work again with Neil. His attention to detail is second to none. I have seen his work in action in the past and he is great. I love his calm approach and his patient guidance to non-tech people like me. I would highly highly reccommend him!

Dr Fuz

Neil is the best! He has sorted out all our business emails and everything runs a great deal better now. Website issues are now a thing of the past as Neil has introduced a maintenance programme to ensure the site is always working optimally and any content changes are actioned pretty much immediately.

Highly recommended!!

H Scott-Brown

'Encode' provided my hosting and domain transfers. They were able to deal with the technicalities of the transfer in a particularly speedy manner, taking all the worry of the transfer, domain issues DNS and nameserver headaches as well as the migration. A couple of emails, and phone call meant it was done smoothly, swiftly and efficiently. All I can say is I'm really please they were recommended to me. Thank you