If I add those lines to my example and run, I get the following output:
tensor(-0.0064, device='cuda:1') tensor(-0.0003, device='cuda:1') tensor(0.0064, device='cuda:1')
tensor(-0.0040, device='cuda:0') tensor(0.0005, device='cuda:0') tensor(0.0056, device='cuda:0')
tensor(-0.0064, device='cuda:0') tensor(-0.0003, device='cuda:0') tensor(0.0064, device='cuda:0')
tensor(2.0009, device='cuda:1')
tensor(3.6893e+19, device='cuda:1')

So, they are not the same.
Even more curiously, I find that if I re-run the last two lines you sent without killing the kernel, I get:
tensor(3.6893e+19, device='cuda:1')
tensor(3.6893e+19, device='cuda:1')

So, for some reason re-running I get a different answer for (output - output1.to("cuda:1")).abs().max().
Not sure where we go from here, please do let me know what other tests I should run, or if you need more information.