In the logs, the output starts below the target value, then the optimizer pushes the parameters to make the output increase and eventually it passes the target.
At this point, the optimizer should change the parameters so that the output will be in the opposite direction (decreasing it). Gradient descent should guarantee this decrease if the dataset has 1 sample only.
But instead, the output just keeps increasing and increasing for multiple steps before eventually coming back down and converging. I don’t understand how this could happen – optimizer makes it continue increasing for multiple steps after passing target and loss goes up and up – with only 1 sample.