Hi, I have a dataset with 2D samples of shape (X, 100) each, where X is an arbitrary number.
I want to feed a neural network using batches, but the samples can have variable sequence length and for this reason I used pad_sequence method to create the batch tensor with padded samples (of shape (batch_size, sequence_length, 100)).
Example (here each sample has shape (x, 3)):
L = [tensor([[ 0.7900, -0.7206,  1.2947]]),          tensor([[ 0.5230, -0.3547, -1.8458],                     [ 0.2626,  0.0604,  0.1883]])]
L is the list of my samples, as you can see, the first has shape (1,3) and the second has shape (2,3)
Applying pad_sequence method the result is:
tensor([[[ 0.7900, -0.7206,  1.2947],                [ 0.0000,  0.0000,  0.0000]],               [[ 0.5230, -0.3547, -1.8458],                [ 0.2626,  0.0604,  0.1883]]])
The neural network has two nested networks which results are summed together and concatenated to output a score. Does the network’s result depend on padding? How can not consider it?
I know that Keras has a masking layer, but I don’t know how to implement it on PyTorch.
Thank you in advance.