Ok, I’ll test it.

If you push a copy of the current model to GPU9 and execute the evaluation method, it should run asynchronously while your other GPUs are training.

Could this be implemented using torch.multiprocessing or what can you recommend?