We do our best to try to access servers to run our crawls - but there will always be edge-cases where you will need to apply a residential proxy to your crawl.
STEP 1: Buy some proxies!
Below are some proxy providers we recommend - We recommend the Rotating Residential Proxies type for ideal performance.
https://www.webshare.io/
*Estimate cost for 100 pages = $1.40
There are some other proxy providers below *prices will vary.
https://soax.com/proxies/residential
STEP 2: Apply your purchased proxies
1. go to your Account Settings
2. enter your proxy in this exact format: protocol://user:pass@host:port
example: http://umguadan:[email protected]:6654
IMPORTANT
Sometimes the proxy provider will give write it like this: 31.59.20.176:6754:umguadan:uajt9fg6op0j
formatted like: host:port:username:password
In that case just rewrite it in this sequence: protocol://user:pass@host:port
*make sure to add the protocol = http://
if you are having issues - just ping us on live chat.
What are Residential Proxies?
Residential proxies allow our crawlers to access websites through real residential IP addresses instead of standard data-center IPs. Some websites, CDNs, firewalls, and bot-protection systems may block or limit automated crawlers when they detect traffic coming from hosting providers or known cloud infrastructure. By routing crawl requests through residential proxy networks, VisualSitemaps can appear more like a normal visitor from a real internet connection, which can improve crawl success for sites with stricter security rules, geo-based access controls, or aggressive bot filtering. This does not bypass private logins, paywalls, or permissions, but it can help reduce false blocks and make crawls more reliable for websites that are otherwise accessible to regular users.