i tried another sanity check, just flattened the input of the entire net (a 1,64,64,32 tensor) and inserted into a linear layer that ends at one neuron that goes into MSE loss.
still same behavior of static loss. important to note that i’m giving the same input everytime.
so maybe it means it’s not the net itself after all?