I think I can access the init_hidden() method using net.module.init_hidden(), at least it runs past that line now.
It still does not work though, because with batch_size=64, I want to give each of my four GPUs 16 training samples, but instead I get this error:
Traceback (most recent call last):
  File "char-rnn.py", line 185, in <module>
    output, hidden = net(c, hidden, batch_size)
  File "/root/miniconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 61, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/root/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/data_parallel.py", line 71, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs)
  File "/root/miniconda2/lib/python2.7/site-packages/torch/nn/parallel/parallel_apply.py", line 45, in parallel_apply
    raise output
RuntimeError: size '[64 x 1]' is invalid for input of with 16 elements at /py/conda-bld/pytorch_1490981920203/work/torch/lib/TH/THStorage.c:59

So it seems like each GPU (or just one GPU, not sure) gets full batch of 64 samples. Any ideas?