The importance of website speed to search engine crawling
As the internet is growing, one of the differentiating aspects of content can be the speed that information is served and how well your web host is helping you to be found.
This screenshot from Google Webmaster Tools, to the right, is a great demonstration of the importance of website speed to search engine crawling. The three sections are:
- Pages crawled per day
- Kilobytes downloaded per day
- Time spent downloading a page (in milliseconds)
The importance of website speed to search engine crawling is emphasised in the Time spent downloading a page (in milliseconds) section that shows that up to mid November the site in question was responding at around 2000 milliseconds. At this time the "Pages crawled per day" were settled at around 50 and Kilobytes downloaded per day at around 500 Kb
After some server reconfiguration was dealt with towards the end of November which unfortunately made the response peak at around 6000 milliseconds. At this point Google didn't seem to make any difference between Pages crawled and Kilobytes downloaded.
From the start of December the reconfiguration had settled and was then responding between 700 and 800 milliseconds. This did get the attention of Google where the speed increase can be seen to have a dramatic affect on amount of information that Google wanted to know for the site.
When the speed increased the Pages Crawled rose from a steady 50 pages per day to over 340 which is a 680% increase. The Kilobytes downloaded per day is even better before the changes Google were interested in around 500 Kb per day then after the upgrade this rose to 3500 giving a huge 700% increase in the need for information about the site.
So, in conclusion, What is the importance of website speed to search engine crawling? Google gained a lot of interest in the site when the response time dropped below 1000 milliseconds.
I understand that this is a brief snapshot of a single site for a short period of time but it goes with reason that with the amount of content that is being produced and published on a daily basis one difference has to be the speed at which that information can get to the end-user: You the reader.
If you want to information on how we can help to improve your site you can just contact us and we will run through a full analysis of your site and how we can help.
What you are thoughts, let us know in the comments below.
Wednesday, February 12, 2014