I am having problem in scrapy concurrency with captcha
Use a blocking HTTP client library (e.g. Requests) for your API calls.
I have been searching all over internet for solution but couldn't find any. I am using captcha solving api to solve captcha in scrapy-splash but when captcha page appears, scrapy sends request to api while trying to scrape next page. This is happening due to concurrency. I have tried CONCURRENT_REQUESTS=1 but same result. I have tried setting wait and pausing engine but in most cases, two or more threads arrive exactly at same line and so while first thread pause engine, 2 or more thread also pauses engine and tries to solve captcha. Can anyone please suggest anything to solve this issue.
sorry if you didn't understand, Please comment any questions you have, I reply in seconds