Hi,
If your model is relatively small, it is not abnormal for it only to use 100MB.
What takes a lot of memory are all the intermdiate results needed for gradient computation. You can check the max allocated memory to see the peak usage.