I don’t know how to use GRU
It moves but doesn’t learn
This is the first time I’ve used an RNN layer in pytorch, so I think this is the cause.
Please tell me if there is not enough information I will add it
code
#Definition
self.rnn=nn.GRU(self.hidden_size1,gru_hidden)
self.norm=nn.LayerNorm(gru_hidden)
self.Relu=swish(0.7)
#runtime
x=self.Linear(x)
x,hidden=self.rnn(x,hidden)
onry_x=self.Relu(self.norm(x))
Causes i can think of
1
Insufficient understanding of gru and proper initialization (for example, he、Set bias to 0)
I don’t even know how to access weight bias
2
There is a tanh in the GRU (in the official description)
3
Something is missing
​