Thanks @ptrblck.
If I call cat on a list of tensors repeatedly, won’t each cat call require a new malloc + copy? The list would have a new element for every input token in the sequence.
I was planning on making a much larger (say, 500 column tensor) and fill in columns of that tensor one by one. And resize to a larger tensor when my original one was full.