Some weights of GPT2LMHeadModel were not initialized from the model checkpoint at 'yyy' and are newly initialized
Hi Everyone,

I am trying to use a fine tuned GPT Neo Model for my inferencing, am getting the below warning - 

### **_Some weights of GPT2LMHeadModel were not initialized from the model checkpoint at 'yyy' and are newly initialized..._**

I have used deep speed to fine tune the model. Because of this warning, **_the generated text during inference is giving some random meaningless text_**. Please suggest how to load those weights.


### Inference code is below (highlighted in bold is the line of code)

from transformers import GPT2Tokenizer, GPT2LMHeadModel
import torch

device = 'cuda' if torch.cuda.is_available() else 'cpu'

tokenizer = GPT2Tokenizer.from_pretrained('finetuned')
tokenizer.padding_side = "left"
tokenizer.pad_token = tokenizer.eos_token
### model = GPTNeoForCausalLM.from_pretrained("finetuned").half().to("cuda")

Thanks,
Balaji
Hi, I'm not very familiar with GPT Neo, but should [this](https://huggingface.co/transformers/model_doc/gpt_neo.html#transformers.GPTNeoForCausalLM) be the correct model for fine-tuned GPT Neo?<|||||>Apologies for pasting the wrong code, below is the one. Thanks @qqaatw for pointing it out. 
This is the line of code where am seeing the warning.

### model = GPTNeoForCausalLM.from_pretrained("finetuned").half().to("cuda")<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.