Threading or multiprocessing for webscraping with selenium
The main factor in CPython for choosing between Threads of Processes is based on your type of workload.
If you have a I/O bound type of workload, where most of your application time is spent waiting for data to come in or to go out, then your best choice is using Threads.
If, instead, your application is spending great time using the CPU, then Processes are your tool of choice.
This is due to the fact that, in CPython (the most commonly used interpreter) only one Thread at a time can make use of the CPU cores. For more information regarding this limitation just read about the Global Interpreter Lock (GIL).
There is another advantage when using Processes which is usually overlooked: Processes allow to achieve a greater degree of isolation. This means that if you have some unstable code (in your case could be the scraping logic) which might hang or crash badly, encapsulating it in a separate Process allows your service to detect the anomaly and recover (kill the Process and restart it).
I need to use selenium for a scraping job with a heap javascript generated webpages. I can open several instances of the webdriver at a time and pass the websites to the instances using queue.
It can be done in multiple way.s though. I've experimented with both the Threading module and the Pool- and Process-ways from the  multiprocessing module.
All work and will do the job quite fast.
This leaves me wondering: Which module is generalky prefered in a situation like this?