Thank you for the response. I tried adding different activation functions and got the similar results.
One other thing I tried which may be telling are:

My network can reach 0 loss on a single training example



However as soon as I try to train a batch of two inputs, it predicts the same output:

As you can see the inputs are different but they both predict -6.5, when the actual targets are -15 and 2.
Any idea of what this means and/or how I can try to debug it? Thank you!