k will be a new tensor and thus use new memory for the concatenated tensors.
PyTorch uses a caching allocator internally, which should reuse the GPU memory (assuming you are using the GPU), but this operation would still copy the tensors.
You could append the tensors to a list and call torch.cat or torch.stack at the end, if possible.