If dropout is going to remove neurons, why are those neurons built? I know that Dropout will remove neurons randomly to reduce over-fitting.
If Dropout is going to remove neurons, why are those neurons built? We could remove those neurons from the architecture.
Why don't we do that?

The neurons are only dropped temporarily during training. They are not dropped from the network altogether. It is just that it turns out that we get better weights if we randomly set them to zero, temporarily, so the other neurons "think" they cannot "rely" on the other neurons and have to "perform well themselves". The neural network that you get out at the end contains all the neurons.
