Hello,
I have a laptop with:
OS Name - Microsoft Windows 10 Enterprise 2016 LTSB
Processor - Intel® Core™ i7-6700HQ CPU @ 2.60GHz, 2592 Mhz, 4 Core(s), 8 Logical Processor(s)
|Installed Physical Memory (RAM)|16.0 GB|
|Total Physical Memory|15.8 GB|
|Available Physical Memory|8.92 GB|
|Total Virtual Memory|18.3 GB|
|Available Virtual Memory|8.75 GB|
|Page File Space|2.53 GB|
Adapter Description	Intel® HD Graphics 530
Adapter Description	NVIDIA Quadro M2000M
The NVIDIA GPU has only 4GB RAM
I have a model which successfully inferenced by the PyTorch C++ interface using the torch::jit::script::Module.
When the application  runs as a stand alone process in the system everything is working fine but when I add an additional CUDA based applications which consume also parts of the GPU memory, the following operation fails:
std::shared_ptr<torch::jit::script::Module> module;
output = module->forward(inputs).toTensor();
CUDA(cudaDeviceSynchronize());

I assume that the failure root cause is unavailable GPU memory.
Questions:


Is there any way to control this dynamic failure?


Is there a way to statically know how much GPU memory will be required for inference operation based on the model?


Is there a way to tell the PyTorch to use the CPU memory in case no more available GPU memory exist?
As the Tensorflow C++ interface provide:
For example:


tensorflow::SessionOptions sessionOptions;
m_sessionOptions.config.mutable_gpu_options()->set_allow_growth(true);

Or:
sessionOptions.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0.8);

Thanks,