Hi @ptrblck : As a small follow up question on that, I was wondering how we could use an if statement to initialize the kernel and recurrent base separately for an LSTM in Pytorch as keras has orthogonal initialization for the recurrent layers and glorot for the kernel
Something like
 def init_weights(model):
    if type(model) == nn.RNNBase:
       # Do Orthogonal
    elif type(model) == nn.Linear:
       # Do glorot

rnn = nn.LSTM(10, 20, 2)
rnn.apply(init_weights) 


I was trying to understand the right conditions from torch.nn.modules.rnn — PyTorch 1.8.1 documentation 1 but wasn’t able to figure it out.