The attributes of a module will be moved to the device, if they are registered as buffers (i.e. they don’t need gradients) or as an nn.Parameter (i.e. they should be updated).
Since you defined num as torch.int8, an nn.Parameter won’t work, as only floating point tensors can require gradients.
If you just register num as self.num = torch.zeros((a,), dtype=torch.int8) it won’t be moved to the device.