The inputs shouldn’t be all the same, they are images of shoe sole impressions, which are mostly white with a bit of black where the shoe sole came in contact the ground.
For the first few batches, most of the time just 1-2 but sometimes 3 or 4, it seems to work as intended.
So maybe there is a problem with the Dataloader? But how can I debug this except from running through it step by step? Is there a way to get the current statistics of the batchnorm layers?