Hey Guys,
I am using multiple nn.BatchNorm1d() layers. I noticed that the activation of the same inputs change slightly when I include the input in different batches. Even when I am using model.eval().
So I have input A and I feed it to the model in the same batch as the the inputs B and C. The activation of A will differ to the Activation of A when I fed it through the network together with B C, D, E.
Is this due to the BatchNorm I feel like model.eval(), should prevent this behavior. But maybe I am messed up somewhere.
Would there be a option to prevent this behavior