So I did some digging: To me it looks like manual_seed, which ends up calling createGeneratorState only sets the MTGP32 states (=that of the RNG used in THC code), but not the seed + offset for Philox (=the RNG used in native code).
I filed a bug 63.
@wandering007 Thank you for sharing this with us and providing the testing code! A reproducing example is gold. This should be fixed soon. As a workaround I would recommend using torch.cuda.set_rng_state.