Are the GPUs completely empty in both setups or are you using e.g. a display or other applications in the Windows machine?
If they are empty (check it via nvidia-smi), are you seeing the larger memory usage in all models on the Windows system?