How to exclude Embedding layer from Model.parameters()?
I use an embedding layer to project one-hot indices to continuous space. However, during the training, I don’t want to update the weight of it. How could I do that?
ShawnGuo
you can set the weight of the embedding layer to not require grad. <SCODE>m = nn.Embedding(...)
m.weight.requires_grad=False
<ECODE>
smth
Oh, I see. Thank you very much.
ShawnGuo
<SCODE>ValueError: optimizing a parameter that doesn't require gradients
<ECODE> The optimizer is used in the following way: <SCODE>self.optimizer = optim.Adadelta(self.model.parameters(), lr=args.learning_rate)
<ECODE> And, model is defined as follow: <SCODE>class DecomposableModel(nn.Module):
    def __init__(self, word_embedding, config):
        super(DecomposableModel, self).__init__()
        self.name = 'DecomposableModel'

        self.drop_p = config['drop_p']

        self.word_dim = word_embedding.embeddings.size(1)
        self.embedding = nn.Embedding(word_embedding.embeddings.size(0), self.word_dim)
        self.embedding.weight = nn.Parameter(word_embedding.embeddings)
        self.embedding.weight.requires_grad = False
        # self.embedding_normalize()

        self.F = nn.Linear(self.word_dim, config['F_dim'])
        self.G = nn.Linear(2 * self.word_dim, config['G_dim'])
        self.H = nn.Linear(2 * config['G_dim'], config['relation_num'])

        self.cuda_flag = config['cuda_flag']

    def forward(self, p_ids, h_ids):
        ......
<ECODE>
ShawnGuo
Hi,
albanD