Yea my plan was to just preallocate a tensor with say 500 rows and double it in size every time I need to resize so that I don’t end up doing quite as many copies. I was looking for the cheapest way to do that.
Unfortunately at each input step I need to construct the cat’ed tensor ( in order to compute the attention scores at that timestep) so I can’t just use a list and cat it once at the end.