Take a look:
def forward(self, out):
        out = self.fc1(out)
        out = torch.transpose(out,0,1)
        hidden = self.__init__hidden(batch)
        out,hidden = self.gru(out, hidden)
        out = self.fc3(hidden)
        out = out.reshape(batch,3)
        return out
    
    def __init__hidden(self, batch):
        hidden = torch.randn(1, batch, 512).to(device)
        return hidden

When I call outputs = net (wn), def forward is triggered.
This happens for every sequence in batch.
And each time we initialize the hidden variable, which calls def __init__hidden. I want def __init__hidden to work the first time, and for the second and next passes hidden should be remembered from the previous sequence (out,hidden = self.gru(out, hidden)).