Really thanks, I was using python list multiplication with torch.stack, but this is much faster.
On CPU:
In [22]: %timeit torch.stack([torch.arange(1000)] * 2000, 1)                                   
42.3 ms ± 125 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [23]: %timeit torch.arange(1000).unsqueeze(1).expand(1000, 2000)                            
13.1 µs ± 21.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)


On GPU:
In [27]: %timeit torch.stack([torch.arange(1000).to(device)] * 2000, 1).to(device)             
2.15 ms ± 6.66 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [30]: %timeit torch.arange(1000).to(device).unsqueeze(1).expand(1000, 2000).to(device)    
43.8 µs ± 152 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)