Testing How Crawl Priority Works
Tue, Feb 23, 2010
SEOmoz tests have showed various techniques used by search engines to determine crawl priority paths.
It has been said that the more pages linked, the greater the chance that the web page will be indexed. Generally, the further away a page is from the homepage, the less likely the page will be indexed.
We then see a slight decrease in the indexation of pagination pages 2, 3 and 4. This is most likely because additional pages have been created and are yet to be properly indexed. Therefore, there is a shift in crawl priority for the deeper URLs.
If you want deep pages indexed, the pages should be additionally linked to from other pages. Such as through “similar products”, “see also”, “related categories” and so on.
Conclusion:
When creating large amounts of pages, a deep URL structure can sometimes prevent high crawl priority if not implemented correctly. The key solution to the problem is found in basic web usability techniques.
I.E Click here for “similar products” or See Also: Jack in the box.
If the average user finds it difficult to find and click the deep URLs, you can bet search engines will also find it difficult.
No related posts.
Tags: crawl priority, search engines, SEO



Leave a Reply