I’m looking at a lstm tutorial. In this tutorial, the author seems to initialize the hidden state randomly before performing the forward path.
        hidden_a = torch.randn(self.hparams.nb_lstm_layers, self.batch_size, self.nb_lstm_units)
        hidden_b = torch.randn(self.hparams.nb_lstm_layers, self.batch_size, self.nb_lstm_units)

it makes more sense to me to initialize the hidden state with zeros.

  
      gist.github.com
  
  
    https://gist.github.com/williamFalcon/f27c7b90e34b4ba88ced042d9ef33edd 42
Pytorch_LSTM_variable_mini_batches.py
import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn import functional as F

"""
Blog post:
Taming LSTMs: Variable-sized mini-batches and why PyTorch is good for your health:
https://medium.com/@_willfalcon/taming-lstms-variable-sized-mini-batches-and-why-pytorch-is-good-for-your-health-61d35642972e
"""
This file has been truncated. show original




  
  
    
    
  
  


is random initialization the correct practice?