Oh, linear layer works on last dimension, thus you can concatenate on last dimension and proceed. But in your code, tensor lengths vary before last dimension, thus you can’t do this. You can use .view(4,-1) or .view(4*40,-1) on inputs, but varying first dimension (treated as batch(=independent) dimension) makes no sense. Maybe you need permute(). Also, I’m not sure where 62720 comes from, replace it with concatenated length.