If you need to create a tensor in the forward method, I would recommend to use the device attribute of any parameter of the model. Otherwise you might run into problems e.g. if you are using data parallel.
However, in your approach you are pushing each tensor in self.C to the same device in each forward pass, which seems not to be necessary, as you could also do it in the __init__ method once.
Is self.C containing tensors with a variable shapes? If not, you could just use torch.stack to create a single tensor.