Solved the case with only 1 sample! My expectations were based on SGD, but I was using Adam, which oscillates before converging.
But why with 2 samples in the dataset, does the network end up outputting the same strange value for both of them, instead of converging to each of their targets? I still don’t understand this.