I have been trying to get this web scraping script working properly, and am not sure what to try next. Hoping someone here knows what I should do. 
I am using BS4 and the problem is whenever a URL takes a long time to load it skips over that URL (leaving an output file with fewer inputs in times of high page load times). I have been trying to add on a timer so that it only skips over the url if it doesn't load in x seconds. 
Can anyone point me in the right direction?
Thanks!
Try using multi threading or multiprocessing to spawn threads, i think it will spawn a thread for every request and it won't skip over the url if it's taking too long.
Python BS4 Scraping Script Timer