Skip to main content

How to know the Total Scope/Pages of a site prior to crawling.

Pro-Tip

Written by Jeff

Option 1:
​Google site:yourdomain.com
See total search results found


Option 2:
​Most sites, have a nifty little file called 'sitemap.xml` - you can always find it on the top- level :
​
​somesite.com/sitemap.xml
​
​for example: https://www.apple.com/sitemap.xml or http://visualsitemaps.com/sitemap.xml

With that file ( which is just a bunch of links that tell Search Engines what to index ).. you can count them all to get an idea of scope.
​
We don't recommend you manually count them of course.. let the browser do that for you!
​
​Step 1: Open Google's Chrome Browser

Step 2: Open the Sitemap.xml url in your browser

Step 3: Command +F (mac) or Cntrl+F (pc) to bring up Search in page.
​
​Step 4: enter 'http:' and voila.. wait for the nodes to be counted!
​

Did this answer your question?