tset0401:

you mean that I should keep dropout in my model for better result, instead of removing it (in case 1, gives better performance)?


Kind of both :). What I mean is in your particular architecture, Dropout may not work well, which is why you get a better test performance if you training a model without dropout (i.e., model.eval() disables dropout during training).
Usually, (as per Hinton; dropout came from his group), for dropout you want to make your neural network slightly overcomplex (the capacity is larger than needed for a given problem). So, I was suggesting to add parameters to your network and then try to see if that, together with dropout, gives  you even better results than training the network you have without dropout. Also, maybe your dropout is too strong for the given architecture, and like @vmirly1 suggested, I would also try smaller dropouts probas.