I think so, as long as it fits in your GPU.
If you do it each batch, the time spent on transferring data from memory to GPU can be relatively large. That’s why you may observe a low GPU-Util with nvidia-smi.