Yes, if you would like to evaluate your model, you should call model.eval(), as this will e.g. disable dropout and use the running estimates in batch norm layers.
Did the additional dropout change anything at all or are you seeing approx. the same accuracies nevertheless?