torch.cat works for me:
a = tc.empty(1, 10)
for i in range(6):
    b = tc.empty(1, 10)
    a = tc.cat((a, b))

I suggest appending them to a list and stack them. You can test which way is faster on your GPU.