Oh then I misunderstood. What you ask is still doable pretty much the way I described, for example you can save the outputs at each step in a list, and then create a tensor from that, and pad it to a standard length if you need to calculate your loss on the entire sequence. In many applications, this isn’t needed though, for example in language models you usually calculate the loss on each step (greedy training), so in that case you dont need to deal with variable sizes at all.