The memory usage depends not only on the stored parameters and activations but also on the used libraries. E.g. if you are using cudnn with its benchmark mode (via torch.backends.cudnn.benchmark = True), the first iteration will profile different algorithms and will select the fastest one, which might use more memory.
The available kernels depend on the cudnn version as well as the used GPU, so I think the best way to profile the model would be to check the max. batch size for your current setup.
That being said, if you disable e.g. cudnn and use the native implementations, you might be able to calculate the approx. memory usage for the current PyTorch version.