It your model is not training properly I would generally recommend to try to overfit a small dataset, e.g. just 10 samples, and make sure your model is able to do so by playing around with some hyperparameters.
If this is not possible, you might have some bugs in the code, e.g. forgetting to zero out the gradients.
PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier