Hi everyone,
I am using minibatch to accelerate my rnn network. The size of my minibatch is 50.
Now I get the following code:
hiddens, hidden = self.rnn(vids_embeddings_sorted_packed, hidden)
hidden goes well. It owns all 50 results.
result:
hidden
tensor([[[ 0.7852, -0.7907,  0.0129,  …, -0.5274, -0.6168,  0.8388],
[-0.1980,  0.5532, -0.4744,  …,  0.3106, -0.3343,  0.1624],
[ 0.2504,  0.1677, -0.5666,  …, -0.6386,  0.0573,  0.2041],
…,
[ 0.9051, -0.7725,  0.5131,  …,  0.8861,  0.7480, -0.1111],
[ 0.7791, -0.8842, -0.6890,  …, -0.2500, -0.7887,  0.4737],
[-0.1205,  0.2851, -0.5888,  …,  0.2452, -0.1945, -0.8261]]])
hiddens should own all the intermediate results. But it only owns the intermediate results of the first one, but without the remaining 49 ones.
result:
hiddens
PackedSequence(data=tensor([[ 0.8669, -0.2405, -0.2589,  …, -0.2067,  0.3336,  0.6909],
[-0.3318,  0.8954, -0.5930,  …, -0.3042,  0.3690, -0.4933],
[ 0.7121,  0.0110,  0.2848,  …,  0.5082,  0.3286, -0.2838],
…,
[ 0.1060, -0.9560,  0.0281,  …,  0.7583,  0.7072,  0.8017],
[ 0.8482,  0.2802, -0.2152,  …,  0.8695,  0.1836, -0.9567],
[ 0.7852, -0.7907,  0.0129,  …, -0.5274, -0.6168,  0.8388]]), batch_sizes=tensor([ 293,  293,  111,   46,   21,   12,    7,    4,    3,    3,
3,    2,    2,    1,    1,    1,    1,    1,    1,    1,
1]))
You may find that the last row of hiddens should end in -0.8261, but it only contains the result of the first row of hidden, but neglect the result of the remaining 49 row of hidden.