The model parameters are only a small part of the total memory that you need to run a model. Even thought the model parameters only requieres 500M, to run the model you also need to keep in memory the values passed in every layer. For example, if you have a simple first layer Conv2d with a kernel size 3x3 that acts over a RGB image and gives 2 output channels i.e. (Cin,Cout,H,W)=(3,2,3,3), in this case the first layer only have 54 parameters. However if the input image size is (C,H,W)=3x128x128, the intermediate first layer will have 128x128x2=32768 values (assuming padding to keep the input size). Moreover, depending on the optimizer that you are using it will also increase the memory, for instance Adam saves two vectors with the same size that the parameters of the model, so that 2x500M. However, the increase of memory after several epoch is strange.