Quick question @apaszke and @jtremblay :
Did you implement DQN with the same memory configuration as stated in the original paper (1 Mio. transitions)?
I am currently facing a similar issue, where python starts to use up gigabytes of memory in an end-to-end learning setting an the python process gets killed at around 114 thousand transitions in memory.
I’m working on a MacbookPro with 16 gigs of RAM.