Hi, I am looking for a solution to reset a layer in the pre-trained model. For example, like in a BART model, if I am going to reset the last layer of the decoder, how should I implement it?
I notice we have the _init_weights(), which should be helpful. So I am wondering if the code should be like:
# load the pre-trained model
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large")
# reset a specific layer
model._init_weights(model.get_decoder().layers[n:n+1])

But I don’t think I make it correct because the fine-tuning result doesn’t change. Any ideas on this implementation? Thank you!