Search Engines and Performance

Search Engines such as Google, Yahoo and Microsoft Live Search (or Bing) are very useful at getting your site noticed. However, sometimes the search engine 'robot' that finds your site and catalogues your web pages can be a little bit over ambitious. On sites with lots of content that is fetched from a database, the attention of a search engine robot can cause significant load on a database as the robot repeatedly pulls all the content from your database.

To improve performance, we recommend that every site has a 'robots.txt' file which indicates to the search engine where it can search and where it can not. This not only helps with your performance, reducing CPU load, but can also improve your search ranking as the results in the search engine do not contain irrelevant pages.

The following article provides many useful examples on how to optimize for search engines with popular packages such as phpBB and Wordpress.


  • 0 Users Found This Useful
Was this answer helpful?

Related Articles

Optimizing your website via cPanel

You can quickly and simply improve the performance of your website by going to your cPanel at...

Performance tips from Google

Google have collected together lots of very useful tips and tutorials in optimizing your...

Using Expires Headers to improve performance

Adding an Expires header to your website with a date in the future makes the files cacheable,...

High Server Load

High server load can be caused by a number of problems such as high CPU load, high memory usage...

Improving Wordpress performance

Wordpress can be a significant drain on a server's resources if badly configured. Here's some...