Hi,
I add these lines at the beginning of my code, and the main.py of my code goes like this:
import torch
import torch.nn as nn
import numpy as np
import random
import my_model
import my_dataset

torch.manual_seed(123)
torch.cuda.manual_seed(123)
np.ranom.seed(123)
random.seed(123)
torch.backends.cudnn.enabled=False
torch.backends.cudnn.deterministic=True

def train():
...

if __name__ == '__main__':
    train()

I run my program for two times without modifying the code. The loss of the first iter are both 9.044713973999023, but the loss of the second iter becomes different with one to be 9.045238494873047 and the other 9.045231819152832.
The reason why I am so concerned about these random behavior is that I find the evaluation results of running the same code for two times can become as large as around 0.5%. I trained my model on cityscapes for 80k iter, and I believe it should be enough for the model to be converged. Thus I do not quite understand why there is such a gap between running two times, and why the loss goes different even at the second iter? How could I fix the random behavior to make two results to be identical as expected please?