How to use the parameters of a certain layer
print(bert.embeddings.LayerNorm.bias)  #Run successfully
print(bert.encoder.layer.0.attention.self.query.weight)     #invalid syntax appears in "layer.0."
so how to use the parameters of a certain layer?
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>@mengruwg 

The general way to access any parameter of a PyTorch model via its name is

`parameter = model.state_dict()[<parameter_name>].`

Since huggingface models override PyTorch nn.Module you can use the above method for any model. [Reference](https://discuss.pytorch.org/t/how-to-manipulate-layer-parameters-by-its-names/1282)
<|||||>I see. Thank you very much for your prompt reply.
















At 2021-08-28 01:53:47, "Rahul Shiv Chand" ***@***.***> wrote:

@mengruwg

The general way to access any parameter of a PyTorch model via its name is

parameter = model.state_dict()[<parameter_name>].

Since huggingface models override PyTorch nn.Module you can use the above method for any model. Reference

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.