torch.manual_seed(seed) will set the same seed on all devices.
However, setting the same seed on CPU and on GPU doesn’t mean they spew the same random number sequences – they both have fairly different PRNGs (for efficiency reasons).
As guidance, treat the PyTorch PRNGs as determinism helpers, but not so much as having PRNG consistency across device types.