michaelklachko:

open a bug about p not being zeroed


No.  p = something just doesn’t overwrite elements of p, but instead assigns a new thing to the name p. That’s inherent in how Python works, you want p.copy_(...).
There are extremely few reasons to use p.data, and chances are you’re doing it wrong if you’re using it. (And people are getting serious about removing it properly, so hopefully it’ll go away soon.)
For the scaling, I don’t know. From a cursory look at the Gal and Ghahramani paper, maybe they also use the plain Bernoulli. I’d probably multiply with torch.bernoulli(weight, 1-drop_prob) instead of using dropout and scaling.
Best regards
Thomas