I’m trying to implement a multi-thread and single-GPU Actor-Critic training, where 1 thread is doing simulation and 1 thread is doing training.
The two threads will share a replay buffer (on CPU) and a model (on GPU).
An example code is here in the gist https://gist.github.com/zihangdai/fc8f76fbb8a0f6323a6b31e6d98ceb50 59
If you run the code, it will occasionally give Segmentation fault. But real tasks, the Segmentation fault happens must faster.
When I use the net_lock to lock the model, i.e., only one thread can use the model at a time, the problem goes away. However, this will be obviously less efficient. My hope is that both threads can use the model at the same time, and I don’t worry about the undetermined behavior problem for now.
Any idea why this is happening?