ptrblck:

ow are you comparing these different approaches? Are you seeding the code or reusing the same state_dicts? Is each approach deterministic but yields different results when compared to each other?
In the latter case how large are the differences?


I’m seeding the code using the following:
    seed=1024
    torch.manual_seed(seed)
    torch.cuda.manual_seed(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False
    np.random.seed(seed)
    random.seed(seed)