Most likely the main part of the memory allocation after initializing the first model will be grabbed by the CUDA context for this device.
The second model will thus just use the necessary space for all parameters.