Hi,
VMware passthrough mode for V100 GPUs is a dedicated mode as well 1:1 (GPU:VM). There is no way today to control the GPU memory or no of used CORES. What you can do is to limit your application to use more then x amount of HBM2 (like in TF). You can have multiple containers running on a single GPU using nvidia-docker2 and NVIDIA_VISIBLE_DEVICES option let’s say:
4x containers to NVIDIA_VISIBLE_DEVICES=0
4x containers to NVIDIA_VISIBLE_DEVICES=1
4x containers to NVIDIA_VISIBLE_DEVICES=2
4x containers to NVIDIA_VISIBLE_DEVICES=3
but the HBM2 utilization of GPU’s will be a big issue if you don’t limit the GPU memory usage from Pytorch (i.e. 7GB /session/container) ; you may get very rapidly CUDA OMM and all your training sessions will fail.
With Pytorch LMS you have two option to control how LMS will work:

torch.cuda.set_limit_lms(limit) - defines the soft limit in bytes on GPU memory allocated for tensors )
torch.cuda.set_size_lms(size) - defines the minimum tensor size in bytes that is eligible for LMS swapping

If today your Pytorch GPU process is cc 7GB it may worth trying (assuming V100 with 32GB HBM2 in AC922) but because Pytorch can’t cap the HBM2 usage to a specific value let’s say 7GB HBM2, you have very high risk for OMM.
Rgds,
FM