Solved by miturian in post #2 
              
              
                I found the problem myself. 
I was not changing the correct variables in the GRU layer. I had not understood the variables-names properly, as they are described in GRU — PyTorch 1.7.1 documentation. 
Changing to 
gru.bias_ih_l0[:]=0
gru.bias_ih_l0_reverse[:]=0
gru.bias_hh_l0[:]=0
gru.bias_hh_l0_reve…