You do not need to remove the Dropout layers in testing but you need to call model.eval() before testing. Calling this will change the behavior of layers such as Dropout, BatchNorm, etc. so that Dropout layers, for example, will not affect the result.