No, I am not doing anything actually. This nvdia-smi output was there on another cmd prompt. This behavior happens sometimes in validation part and sometimes in training part. The observed behavior is that initially in an epoch which takes 25-30 min it might not happen, but after happening for the first time it keeps on repeating very often.