All Collections
Tutorials
How to know the Total Scope/Pages of a site prior to crawling.
How to know the Total Scope/Pages of a site prior to crawling.

Pro-Tip

Jeff avatar
Written by Jeff
Updated over a week ago

Option 1:
Google site:yourdomain.com
See total search results found


Option 2:
Most sites, have a nifty little file called 'sitemap.xml` - you can always find it on the top- level :

somesite.com/sitemap.xml

for example: https://www.apple.com/sitemap.xml or http://visualsitemaps.com/sitemap.xml

With that file ( which is just a bunch of links that tell Search Engines what to index ).. you can count them all to get an idea of scope.

We don't recommend you manually count them of course.. let the browser do that for you!

Step 1: Open Google's Chrome Browser

Step 2: Open the Sitemap.xml url in your browser

Step 3: Command +F (mac) or Cntrl+F (pc) to bring up Search in page.

Step 4: enter 'http:' and voila.. wait for the nodes to be counted!

Did this answer your question?