All Collections
How to whitelist our crawlers
How to whitelist our crawlers
Jeff avatar
Written by Jeff
Updated over a week ago

Our IPs are ultra-dynamic, and therefore constantly changing.
We also use backup residential IPs for those hard-to-access servers.

You can however use our handy custom headers to identify our crawlers and let them through ;-)

1. Click Advanced Settings > Include Custom ##### HTTP Header
2. this will show your UNIQE header ID.
3. copy/paste that into your server configs.

Did this answer your question?