Yes.
Our goal is to provide you with accurate and full-scoped data. If we go too fast, some servers will react by shutting down access. Who wants that right?
Based on our research and constant benchmark testing, we've determined the safest acceptable rate limits and implemented dynamically throttled limits to avoid server overload.
Crawling
Written by Artur M
Updated over a week ago