Suppose you want to call some API (open without any KEY needed) which has some rate limit per second (10 requests max per second). Suppose now, I am calling a function (with multiprocessing.Pool) which request data from API and process it. Is there some way to switch ip in order to not get blocked? Maybe a list of ip/ proxies. Can somebody tell me a way to get this done?
Thanks!
There certainly are a lot of hacks you can use to get around rate limiting, but you should take a moment and ask yourself 'should I?' You'd likely be violating the terms of service of the service, and in some jurisdictions you could be opening yourself to legal action if you are caught. Additionally, many services implement smart bot detection which can identify bot behavior from request patterns and block multiple IPs. In extreme cases, I've seen teams block the IP range for an entire country to punish botting.
Python Multiprocessing Rate Limit