It looks like hidden_and_cell_state was initialized on the CPU.
You could sample it from the model and push it to the GPU in the training loop or probably even better set the device inside initialize_hidden_and_cell_states:
def initialize_hidden_and_cell_states(self):
    device = next(self.parameters()).device
    # The hidden and cell state at the start are all zeros
    return (torch.zeros(self.batch_size, self.hidden_size, device=device),
            torch.zeros(self.batch_size, self.hidden_size, device=device))