Selenium not freeing up memory even after calling close/quit
I have experienced similar issue and destroying that driver my self (i.e setting driver to None) prevent those memory leaks for me
So I've been working on scraper that goes on 10k+pages and scrapes data from it.
The issue is that over time, memory consumption raises drastically. So to overcome this - instead of closing driver instance only at the end of scrape - the scraper is updated so that it closes the instance after every page is loaded and data extracted.
But ram memory still gets populated for some reason.
I tried using PhantomJS but it doesn't load data properly for some reason.
I also tried with the initial version of the scraper to limit cache in Firefox to 100mb, but that also did not work.
Note: I run tests with both chromedriver and firefox, and unfortunately I can't use libraries such as requests, mechanize, etc... instead of selenium.
Any help is appreciated since I've been trying to figure this out for a week now. Thanks.