Your sequence length seems to be 12 so you need to use every 12th other target in your target tensor, not the entire thing. Or you could make your input in the shape of (batch, seq, *inputshape) and output in the shape of (batch, *targetshape) and remove the view step in your forward