Dropout has a training mode and a test mode. The loss of a network with dropout in test mode should be about the same, on average, as dropout in training mode. I’m not saying that training and test losses should be the same; I’m saying that, using a network trained with dropout, taking the mean of the loss should yield about the same value whether you turn on dropout or you turn it off (and by turning it off I mean, putting it in test mode).
I hope that makes sense.