I found some tools here: pytorch.org/docs/stable/cuda.html
under “Memory management”.
Figured out my problem was that I was using the outputs for some statistical calculations and forgot to put them back to cpu before performing those. Somehow that was causing the issue of tensors being duplicated on the device they resided. Running .to(‘cpu’) on all before sending those to stats fixed it pronto.
Edit: This ended up just putting the memory overflow on the cpu. I changed it to .detach() with no need to remove from the GPU and that seems to resolve the issue.