I think the reason for this non-deterministic behavior on CUDA tensors is, because you are using repeating indices in i and thus also in i_gpu (you can check it with i_gpu.unique().nelement()).
Since the operation is performed in parallel on the GPU, overlapping indices will take the value of the last kernel which writes at this position.
You can also trigger this behavior with a single value:
c = torch.zeros(1, dtype=torch.long)
i = torch.zeros(10000, dtype=torch.long)
s = torch.arange(10000)

c[i] = s  # On CPU will take last value (9999)
print(c)

c_gpu = torch.zeros(1, dtype=torch.long, device='cuda')
i_gpu = i.to('cuda')
s_gpu = s.to('cuda')

c_gpu[i_gpu] = s_gpu
print(c_gpu)  # Prints "random" values

I’m not sure if this is a bug (and the values should be the last value in s) or if this is just undefined behavior.