Is the GPU memory in use during training? How much? nvidia-smi has a Memory-Usage field.
How much GPU-util does it use during validation and for how long?
You can use the command watch -n 0.1 nvidia-smi to run the nvidia-smi every 0.1 second
Reboot again ;)?