I would recommend to add non-linearities between the linear layers and try to overfit a small data sample (e.g. just 10 samples) by playing around with the hyper parameters, such as the learning rate.
Once your model can successfully overfit the small data sample, you could try to scale up the problem again.