I seem to have figured it out.
torch::cat({x, y}) seem to work for me, not 100% sure about the requirements on the dimensionality. It looks like there is a version with another parameter to specify the dimension but it says I can only use -1 or 0, and I am not sure what that means. At least it works as expected when I just concatenate two 1D tensors together.