Great, thank you for sharing!
If you have access to an IBM AC922 8335-GTH model server (4 GV-100 GPUs 32GB each, dual core Power9 CPUs, 1TB system memory), can you tell me if the following would scenario for over-provisioning physically available GPU memory would work?
Let’s assume that I’m running Ubuntu-18.04 LTS on the AC922 with support for NVIDIA docker runtime support. The system is configured with 1TB of physical system memory. I have a docker image which runs a DNN model that requires 16GB of GPU memory.
Now, the total physical available GPU memory is 32GB x 4 = 128GB.
Will I be able to run and launch 16 docker containers, creating a total demand for 16x16GB = 256GB of GPU memory? This is 128GB more than available physical GPU RAM.
Will I still be able to run the docker containers, or will PyTorch complain that it has insufficient GPU resources for some of the docker container instances?
This is for a distributed reinforcement learning environment setup and simulation within one physical node, before considering distribution of the workload over multiple nodes over an infiniband network.