Vignesh_Baskaran:

The first one is: Ideally I would like to assign different random seeds to:

transformer dropout: random_seed(1)
linear dropout: random_seed(2)
linear initialization: random_seed(3)



You could set the seed using torch.manual_seed before running these operations. Note however that seeding the code before applying a random operation (such as using dropout) would use the same mask for the same input shape, so this operation isn’t “random” anymore.



 Vignesh_Baskaran:

My second question is: How can I selectively apply random seed to any of them. To elaborate, I would like to do the following:

transformer dropout - Set random seed - So deterministic behaviour
linear dropout - No random seed - So random behaviour
linear initialization - Set random seed - So deterministic behaviour

Very importantly: Remove random seed so everything else following the model has non-deterministic behaviour



You cannot switch between deterministic and random behavior easily, since all operations using the pseudorandom number generator would use the already set seed.
E.g. if you set the seed in operation 1 (transformer dropout), the same drop mask will be samples, but also the following operations using the PRNG would use the same “random” numbers unless you reseed the code to a new (random) seed.