Make sure your data is split appropriately between the train and the validation set. Use shuffle with stratified sampling if possible.
Experiment with learning rate (lr). The spikes in your dev_mse curve could be due to lr.
Which optimizer are you using? Adam or RAdam optimizer should work well in most cases.
Since you’ve already used experimented with dropout, you can also try using L1/L2 regularizations.