When using a standard nn.GRU() for performing RNNs, how should one initialize the hidden state?  Currently, I do the following:
def init_hidden(self, batch_size):
    # variable of size [num_layers*num_directions, b_sz, hidden_sz]
    return Variable(torch.zeros(self.num_directions, batch_size, self.hidden_size)).cuda() 

This works - however, does this assume the network learns a different hidden state for each element in the batch?  Should I somehow be replicating the same initial hidden state for every element in the batch, so that the network doesn’t need to learn batch_sz hidden states, but rather one hidden state?