guoziyuan:

or 10 of the pretrained


This is usually what I did:
original_model = BartForConditionalGeneration.from_pretrained('facebook/bart-base')
my_model = BartForConditionalGeneration.from_pretrained('facebook/bart-base', encoder_layers=2)
## for example, I want my model's last encoder layer's weight = original model's last encoder's weight
model.model.encoder.layers[1].load_state_dict(original_model.model.encoder.layers[5].state_dict())