Are you using the same data to evaluate your model?
How was the training and validation error?
Did you make sure to call model.eval() before evaluating the model? Your current model probably doesn’t need it, because it doesn’t contain any Dropout or BatchNorm, but it’s recommended anyway.