Can you give me suggestions on my current problem I am facing with bidirectional multilayered GRU?

  
    
    
    Bidirectional GRU with multilayers (what are the inputs to the 2nd layer) C++
  
  
    [35610748-dd41a23a-06a5-11e8-803e-c577359c372b]
 As per my understanding, GRU pytorch working as in the 2nd picture.  After each layer,  both forward and reverse directiona outputs are concatenated. Correct me if i was wrong. 

The initial hidden state of first layer initialised with zeroes. 
Is The last hidden state of 1st layer is the initial hidden state of the 2nd layer?. 
And also I want to know,  the second layer takes the concatenated 1st layer output as a input and operates on them, as …