ResNet is ~44M pars, LSTM is ~6M pars. How much VRAM is that?
Assuming nb_channels=3 and byter per element (does element=pixel?) the data volume comes up to ~16Gb.
The video classifiers I’ve seen were trained on datasets like UCF101 with very short video. I have videos that have a few very different scenes in them, and I can’t extract 1 frame/minute, because LSTM won’t be able to learn the context, so it must be roghly 1FPS, which in total comes up to 5K frames, and they have to be large (e.g. 512x512).
1 video=1 data point.
One more thing I don’t quite understand about DataLoader: if I have a minibatch_size=1, once that one datapoint (video) is fed forward, can I remove it from the GPU to load a new one, so the amount of used VRAM remains the same? This didn’t seem to work that way, but maybe I did something wrong?