Hello,
I am training a fully convolutional network on GPU. Using the same network, trained on the same data, with the same training loop, changing all of the filter sizes from 5 to 3 results in an OOM error (CUDA out of memory. Tried to allocate 102.75 MiB (GPU 0; 7.91 GiB total capacity; 4.79 GiB already allocated; 66.81 MiB free; 14.80 MiB cached)).
For some strange reason changing the filter sizes to 7 does not incur in the same issue, while it occurs again if I switch to kernel sizes of (5,5,1). The filters do not affect the size of the layers as I’m using padding.
I also noticed that when testing on random data all networks work just as fine, but they do stop working on my real data. Could this depend on having different input sizes? This still would not explain why a 5x5x5 filter would work, while a 3x3x3 would not.