Hi!
I am trying to perform Monte Carlo sampling to estimate the uncertainty of my network. To do so, I have dropout enabled and I am executing following loop:
outputs = [self._model(inputs) for i in range(X)]

Where “inputs” is just a single frame.
First I tried this code in Google Colab, and everything was working fine, even with X = 40 and batch_size = 100 (frames) at once.
When I switch back to a personal computer with a 6GB dedicated memory GPU, if X is greater than 5 (with one single frame), I get the famous error “CUDA Out of memory”. Even if I split the code such that:
outputs = [self._model(inputs) for i in range(X/2)]
outputs = [self._model(inputs) for i in range(X/2)]

The code gives me same results.
Why is this happening? Is the for loop being parallelized by CUDA API and thus collapsing the memory? How can I solve this?
Thank you in advance