thank you very much for your fast reply! please excuse me, but I’m rather new to Pytorch, nonetheless your code is very readable!
So the class that you wrote would go, say:
self.fc1 = nn.Linear(input_size, hidden_size)
self.relu = nn.ReLU()
here!
and what I’m guessing is that self.masker(self.weight) gets the weights from it’s input (the layer before it) and It’ll send them to the (custom) Dropout function, where it will return whatever it does? Oh! and:
return F.linear(input, masked_weight, self.bias), is masked_weight the, whatever activation function i use, times the dropout mask?, and why do I return F.linear(input…) why input? don’t I need to return only the output of the layer?
Thank you!