I assume the problem is about the order of making use of the sequence of generated pseudo random numbers. e.g. After generating a sequence of fake pseudo random number [1, 5, 4, 6, 7], conv1, conv2, dataloader may pick up 1,5,4 respectively; But if you added another layer, conv3, then conv1, conv2, conv3, dataloader will pick up 1,5,4,6 respectively. The base seed of dataloader will be different in these two situations.