hi @sshleifer,
what counts as a “layer” for freezing?
I’ve created a model based on BertModel (with a custom head), and I’d like to freeze the embedding layer and some of the attention layers.   I’ve applied
ct = 0
for child in model.children():
ct += 1
if ct < 11:          # this freezes layers 1-10
for param in child.parameters():
param.requires_grad = False
but I’m not sure what that will have frozen.
How can I tell ?