The code looks alright, so I would recommend to play around with some hyper-parameters and e.g. lower the learning rate.
Your dataset is already small, but it might also be helpful for the sake of debugging to just use a few samples (e.g. 10 samples) and make sure the loss decreases and your model is able to overfit this small data sample.