Solved by KuanS in post #2 
              
              
                Hi 
I am doing the same thing currently 
here is my solution 
class SparseDropout(torch.nn.Module):
    def __init__(self, dprob=0.5):
        super(SparseDropout, self).__init__()
        # dprob is ratio of dropout
        # convert to keep probability
        self.kprob=1-dprob

    def forward(s…