If you need to create these tensors inside create_question_char_embedding, you could try to use the device attribute of a model parameter and create the new tensors on the same device:
char_tensor = torch.zeros(50, 45, device=self.parameter.device)