Solved by Abhilash_Srivastava in post #2 
              
              
                You can do it for each epoch too (instead of each batch), which I think is better. You’ll need to do it for both training and testing (and validation). 

Yes, the initialization should match the batch_size of the input. 

hidden should be passed as the input the forward function, not initialized in…