In tensorflow, there is a function called tf.config.experimental.set_memory_growth (Details 1) which allocates as much memory to the process as needed. Moreover, it doesn’t release the memory till the process runs. This prevents any other process from concurrently occupying the memory.
This is a very useful functionality for cases when the GPU is a shared resource and your process has high but dynamic memory requirement.
Could someone tell if there is a similar option in pytorch to limit the number of processes which can share memory concurrently?