yes!
β
However, you need to be very careful there because our crawler will CLICK ON EVERY LINK IT FINDS!
β
So if you set it up to crawl as an ADMIN user, it could trigger links on the page that are available to admins only.
β
Therefore you can either :
a) use a different limited-restriction user ( non-admin )
b) filter out specific links you want our crawler to skip.
**WE ARE NOT RESPONSIBLE FOR ANY CHANGES TO YOUR SITE.**
*PLEASE CRAWL RESPONSIBLY AND ALWAYS BACKUP YOUR SITE JUST IN CASE*
β