Clearly, these randomizers do not generate the same sequence; here’s an example:
torch.cuda.manual_seed(60)

torch.cuda.FloatTensor(1).normal_()
Out[94]: tensor([ 0.7700], device='cuda:0')

torch.cuda.FloatTensor(1).normal_()
Out[95]: tensor([ 0.5048], device='cuda:0')

torch.cuda.manual_seed(60)

torch.cuda.FloatTensor(1).normal_()
Out[97]: tensor([ 0.7700], device='cuda:0')

torch.manual_seed(60)
Out[98]: <torch._C.Generator at 0x7f537c0320b0>

torch.randn(1)
Out[99]: tensor([ 0.7534])

torch.randn(1)
Out[100]: tensor([ 1.8541])

torch.manual_seed(60)
Out[101]: <torch._C.Generator at 0x7f537c0320b0>

torch.randn(1)
Out[102]: tensor([ 0.7534])

Or, I am missing something here?