All Collections
FAQs
How can I ensure that my sitemap can complete crawling all the estimated pages even after I reached my plan's limits?
How can I ensure that my sitemap can complete crawling all the estimated pages even after I reached my plan's limits?

Crawling Tips

Jeff avatar
Written by Jeff
Updated over a week ago

Easy. Simply DO NOT set the Max Page limits.
​
Because, If you do set this limit, the crawler will cold-stop when it reaches it, and you will not be allowed to restart the crawl from where it stopped.
​
By not specifying this setting, when the crawler reaches the plan's Max Sitemap Size limit, it will auto-pause the crawl and give you the option to upgrade to a larger plan that offers larger-sized maps.
​
After the upgrade, simply return to your paused sitemap and click to resume the crawl. At this juncture, you will also have the ability to change some crawl settings like Max Depth and add Max Page limits.

Did this answer your question?