Hi all,
When I load the model for inference. The model always consumes a lot of memory even the model size is small. For example: model size is 70MB (Encoder + Decoder + attention with Resnet 50 as backbone for encoder) but it occupies approximately 1GB GPU memory.
Can we have any ways to reduce the GPU memory when loading the model for inference?
Thank you!