ptrblck:

def initialize_weights(model): if type(model) in [nn.Linear]: nn.init.xavier_uniform_(model.weight) nn.init.zeros_(model.bias) elif type(model) in [nn.LSTM, nn.RNN, nn.GRU]: nn.init.orthogonal_(model.weight_hh_l0) nn.init.xavier_uniform_(model.weight_ih_l0) nn.init.zeros_(model.bias_hh_l0) nn.init.zeros_(model.bias_ih_l0)


@ptrblck
Thanks for your help, i’ve ran it twice, for both:
Pearson: 0.250
Spearman: 0.250
So it definitely improved, but unfortunately it’s still not that…
Appreciate the help. I can definitely provide more code/ dataset if needed ( As mentioned the dataset is publicly available it’s the SemEval 2014 - task 3, the sentence2phrase files ).
I’m hoping that before anyone takes time to try really running the code, a mistake / suggestion can be tried just based on pasted code. Like yours.