If dropout is going to remove neurons, why are those neurons built? I know that Dropout will remove neurons randomly to reduce over-fitting.
If Dropout is going to remove neurons, why are those neurons built? We could remove those neurons from the architecture.
Why don't we do that?

The goal of dropout isn't to disable neurons permanently. It is to prevent the network from splitting into separate sections functioning in parallel and not utilizing interconnections between them. An extreme, undesired case would be a network relying on only a single neuron processing certain feature.
Dropout disables neurons randomly during training which forces the consecutive layer to optimize weights coming from other inputs, and thus making a better use of all the possible connections available in the architecture.
