markdaoust:

Also the dropout-like implementation… this only works to kill the output before the add on a residual branch, right?


You are right. This implementation of Stochastic Depth is only cutting the outputs at the residual block.



 markdaoust:

Does the dropout 1/keep_prob scaling still make sense used like this?


No, I am not scaling the dropped out blocks with the inverse dropout probabilities. They are simply not dropped during inference.



 markdaoust:

self.wrapped = wrapped


Is wrapped the block we are applying?