I am trying to scrape information off of a website (on Chrome) where I need to click on each subpage to get the information I need. After about 7, I get blocked by the website. I think if I was able to switch IPs either each time or once I get blocked, that would work.
I am using Selenium to open the site and navigate to the subpages. I have tried using a try-catch block so far and a while loop but I am getting errors I do not know how to address.
Does anyone have an alternative approach or previous success doing this?
You can use rotating proxies to change your IP per request or with a time interval but If you don't want to use any proxy you can restart your router to get a new IP address from your ISP but If you have a static IP from your ISP even through you restar your router your IP will stay the same.
In python with selenium, how do you rotate IP addresses?