A: a 5x11x4 tensor where there are 5 sentences in a batch and 11 is the vocab size for log softmax.
B: a 5x4 tensor which represents the target sentence
I have the lengths of sentences in my batch to be 3,2,1,4 [Max is 4]
Now i want to prevent backprop happening in the portion of sentences beyond the length.
I can think of a way:
I can have a 5x11x4 tensor constructed such that the third dimension is 1 for the length of sentence times and 0 otherwise.
Then i can multiply this element wise with my original tensor and then apply the NLLloss function. This way backprop cannot affect the gradient in those elements.
Does this seem fine or is any other method recommended?