Hello,
I am working on Semantic Role Labeling, I have tensors which are the result of concatenating the BERT embedding of a word and a bit which indicates where the predicate is in the sentence, so I have tensors of shape (batch_size, sequence_len, bert_embedding+1). I pass these to a dropout layer but the fact that the indicator bit will be zeroed is not desirable. Is there a way to tell dropout not to apply to that index?
Thanks in advance.