Use the fine-tuned model for another task
Hi, I am currently using this code to research the transferability of those pre-trained models and I wonder how could I apply the fine-tuned parameter of a model to another model. For example, I fine-tuned the **BertForMultipleChoice** and got the **pytorch_model.bin**, and what if I want to use the parameters weight above in the **BertForMaskedLM**. 

I believed there should exist a way to do that since they just differ in the linear layer. However, simply use the BertForMaskedLM.from_pretrained method is problematic. 
Hi!

If you saved the model `BertForMultipleChoice` to a directory, you can then load the weights for the `BertForMaskedLM` by simply using the `from_pretrained(dir_name)` method. The transformer weights will be re-used by the `BertForMaskedLM` and the weights corresponding to the multiple-choice classifier will be ignored.<|||||>Hi! Thanks for answering me. And this is what I have done at first, which resulted in the following:
![image](https://user-images.githubusercontent.com/20436061/62506225-4eb6f700-b832-11e9-9438-ab22047cd9e8.png)
As you can see, the output tensors are all zeros, which seems to be really weird! 

Although this might happen, I still want to confirm that I am doing the right thing, I basically calculating each masked word's probability. And some of them are zero which results in the final sentence zero probs.
![image](https://user-images.githubusercontent.com/20436061/62506407-24196e00-b833-11e9-9968-67ef4a47a368.png)
<|||||>Could you share a code snippet that reproduces what you're trying to do so that I can try and see on my side?<|||||>For sure!
```
import torch
from pytorch_pretrained_bert import BertTokenizer, BertModel, BertForMaskedLM
import numpy as np
import math

# OPTIONAL: if you want to have more information on what's happening, activate the logger as follows
import logging
logging.basicConfig(level=logging.INFO)

def predict(text, bert_model, bert_tokenizer):
    # Tokenized input
    # text = "[CLS] I got restricted because Tom reported my reply [SEP]"
    text = "[CLS] " + text + " [SEP]"
    tokenized_text = bert_tokenizer.tokenize(text)
    # text = "[CLS] Stir the mixture until it is done [SEP]"
        #masked_index = 4
    sentence_prob = 1
    for masked_index in range(1,len(tokenized_text)-1):
        # Mask a token that we will try to predict back with `BertForMaskedLM`
        masked_word = tokenized_text[masked_index]
        #tokenized_text[masked_index] = '[MASK]'
        # assert tokenized_text == ['[CLS]', 'who', 'was', 'jim', 'henson', '?', '[SEP]', 'jim', '[MASK]', 'was', 'a', 'puppet', '##eer', '[SEP]']
        # print (tokenized_text)

        # Convert token to vocabulary indices
        indexed_tokens = bert_tokenizer.convert_tokens_to_ids(tokenized_text)
        # Define sentence A and B indices associated to 1st and 2nd sentences (see paper)
        # segments_ids = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]
        length = len(tokenized_text)
        segments_ids = [0 for _ in range(length)]
        # Convert inputs to PyTorch tensors
        tokens_tensor = torch.tensor([indexed_tokens])
        segments_tensors = torch.tensor([segments_ids])

        # If you have a GPU, put everything on cuda
        tokens_tensor = tokens_tensor.to('cuda')
        segments_tensors = segments_tensors.to('cuda')

        # Load pre-trained model (weights)
        # bert_model = BertForMaskedLM.from_pretrained('bert-large-uncased')
        # bert_model.eval()

        # If you have a GPU, put everything on cuda
        tokens_tensor = tokens_tensor.to('cuda')
        segments_tensors = segments_tensors.to('cuda')
        bert_model.to('cuda')

        # Predict all tokens
        with torch.no_grad():
            predictions = bert_model(tokens_tensor, segments_tensors)

        predictions = torch.nn.functional.softmax(predictions, -1)

        index = bert_tokenizer.convert_tokens_to_ids([masked_word])[0]

        curr_prob = predictions[0, masked_index][index]
        
        if curr_prob.item()!=0:
            #print(curr_prob.item())
            sentence_prob *= curr_prob.item()
        # predict_list = predictions[0, masked_index]
        
        #tokenized_text[masked_index] = masked_word
    #return math.pow(sentence_prob, 1/(len(tokenized_text)-3))
    return sentence_prob

# Load pre-trained model tokenizer (vocabulary)
tokenizer = BertTokenizer.from_pretrained('./tmp/swag_output')
# Load pre-trained model (weights)
model = BertForMaskedLM.from_pretrained('./tmp/swag_output')
model.eval()

# prob = predict(sentence_1, bert_model=model, bert_tokenizer=tokenizer)

with open("Sentence4leyang.txt", "r") as f:
    file = f.readlines()

num = len(file)
count = 0
curr = 0
for i in file:
    label, sentence_1, sentence_2, sentence_3 = i.split("\001")
    
    print (label[0])
    prob_1 = predict(sentence_1, bert_model=model, bert_tokenizer=tokenizer)
    prob_2 = predict(sentence_2, bert_model=model, bert_tokenizer=tokenizer)
    prob_3 = predict(sentence_3, bert_model=model, bert_tokenizer=tokenizer)
    answer = max(prob_1, prob_2, prob_3)
    print(prob_1, prob_2, prob_3)

```
For the txt file, you could just create some sentences to replace it.
We used the weight after fine-tuning the Bert with official run_swag.py example.<|||||>If you finetuned a `BertForMultipleChoice` and load it in `BertForMaskedLM`some weights will be initialized randomly and not trained.

This is indicated in this part of your output:
![image](https://user-images.githubusercontent.com/7353373/62710120-fe3cc680-b9f6-11e9-9c7a-45c5a50590a8.png)

If you use this model with un-trained weights you will have random output. You need to train these weights on a down-stream task.<|||||>Hi, Thanks for the response. @thomwolf 
However, from my perspective, even if you use the vanilla `Bert-base-uncased` model, the `BertForMaskedLM` still runs perfectly without any random initialization. And I assume `BertForMultipleChoice` is simply the original `Bert-base-uncased` model with an additional linear classifier layer.
Therefore, I think there should be a way to only keep the 'Bert model' but without the linear layer after fine-tuning. I think this feature could be really helpful for researchers to investigate the transferability of the models.<|||||>No unfortunately.

So the model used for pretraining bert and the one we provide on our AWS S3 bucket is `BertForPretraining` which has 2 heads: (i) the masked lm head and (ii) the next sentence prediction head.

`BertForMaskedLM` is a sub-set of `BertForPretraining` which keeps only the first head => all the weights are initialized with pretrained weights if you initialize it from the provided weights, you can use it out-of-the-box.

`BertForMultipleChoice` does NOT have a masked lm head and has instead a multiple-choice head => if you train this model and use it to initialize a `BertForMaskedLM` you won't initialize the language model head.

If you don't remember: just look at the log during model initialization. If it's written `Weights from XXX not initialized from pretrained model` it means you have to train the model before using it.<|||||>We will make the documentation more clear on that.

For your specific use-case, a solution could be to make a model your-self similarly to the way they are made in the library and keep the language modeling head as well as the other heads you want. And then fine-tune the newly added head on your dataset.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
