Hello there,
Thanks for the tips. Reducing to float16 is essential as the memory reduction is significant - enough of a difference to make it run at all. Interesting about the gradient though. I’m on a 2080Ti at the moment. I see what support it has.
Filter size I can change. I believe I’m using interpolation but I’ll check again. Maxpool is still used but I can replace with a stride. In-place relu I’ve not tried. I’ll give that a shot.
I’m still very surprised that the reduction in parameters has had very little effect on the memory use. Very odd indeed.
B