[Dropout] why  there is no dropout for the dev and eval?
I do not see any dropout layer after `get_pooled_output()`  in the tf version referred to [here](https://github.com/google-research/bert/blob/master/run_classifier.py#L590). Why do you add a dropout layer in your implemention?
@thomwolf  Thanks!<|||||>line 604<|||||>> line 604

Thanks so much. my mistake.

Do you know why there is no dropout for the dev and eval?
<|||||>first of all, no one uses dropout at evaluation stage as it's a regularizer. The difference of implementation is due to the fact that a dropout layer in pytorch behaves differently once you turn on a evaluation model (i.e., model.eval()). So even though it has a dropout layer it doesn't take any effect (p=0) at evaluation.<|||||>> first of all, no one uses dropout at evaluation stage as it's a regularizer. The difference of implementation is due to the fact that a dropout layer in pytorch behaves differently once you turn on a evaluation model (i.e., model.eval()). So even though it has a dropout layer it doesn't take any effect (p=0) at evaluation.

Thanks for help me clarify this problem.