It looks like your model is overfitting.
There are not many solutions to this other than adding regularization such as weight decay, dropout layers, etc or increase your training data.
It is a hot subject within ML and data science and thus there are many blogs and articles on the matter.
I would start with adding a nn.dropout() and if that doesn’t improve much try using a bathnormalization layer as this layer also can works as a regulator. (nn.BathNorm2d()).