I believe that using dropout should speed up training a lot, because the model stops computing parts of the model. However, empirically it seems not to be the case. The training time is roughly the same if I use really high dropout or not.