Thank you so much for the answer @ptrblck
Also another solution I could come up was with this where I would create a dropout mask similar to yours but with one’s
dropout_mask = F.dropout(torch.ones((batch_size, length, observation , 1)), p=0.6)
and then multiply the data with it, one benefit of this I feel from the other approach would be I will not have to scale the activation, the F.dropout will do it for me.
Another is that this will act like a proper dropout where I might be able to get both the features and none of them as well
What are your thoughts about it?