CPM LM
# 🌟 New model addition （2.6B params）

## Model description

CPM(Chinese Pre-Trained Language Models), which has 2.6B parameters, can be used for zero shot、one shot、few shot learning.
Code and model is available.

## Open source status

* [✅] the model implementation is available:   [CPM-Generate pytorch](https://github.com/TsinghuaAI/CPM-Generate) [CPM-LM-TF2](https://github.com/qhduan/CPM-LM-TF2)
* [✅] the model weights are available: can be found [here](https://github.com/TsinghuaAI/CPM-Generate)
* [✅] who are the authors: (Research team of Beijing Zhiyuan Institute of artificial intelligence and Tsinghua University @ TsinghuaAI)

@JetRunner might be interested in that!<|||||>Hi, I didn't notice this before I finished the translation to `Transformer` just now. May [this script](https://github.com/mymusise/CPM-TF2Transformer/blob/main/transfor_CMP.ipynb) help.

BTW, I met some problems when uploading. #8662<|||||>> @JetRunner might be interested in that!

Yes I was working on it but it seems @mymusise has already worked it out!

@mymusise I will assist you through the uploading process!<|||||>@mymusise I think the generated result from your repo is a little buggy here. Any idea why? 
```
[{'generated_text': '你好 ▁ , ▁我 ▁是 ▁ 个 ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk>'}]
```
https://github.com/mymusise/CPM-TF2Transformer/blob/e5ea4799603f19ab7f92596f7ad7472198c505c6/transfor_CMP.ipynb#L881<|||||>|```
|[{'generated_text': '你好 ▁ , ▁我 ▁是 ▁ 个 ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk> ▁ <unk>'}]
|```

@JetRunner  Hi, because I use  `BertTokenizer` here, didn't use the `bpe` method. 
And seems `GPT2Tokenizer` does not yet support other languages such as Chinese, the [byte_encoder](https://github.com/huggingface/transformers/blob/dd52804f5fce0a568ffbb3dc7fd088d2de0a0e56/src/transformers/models/gpt2/tokenization_gpt2.py#L246) here will encode other languages to unknown token. Any advice?<|||||>I see. It's not a big problem since we can now specify the tokenizer type in the configuration file. I can take care of those once you've uploaded the model file. Let's wait for @julien-c to solve the big file uploading problem first.<|||||>OK, thank you, guy.<|||||>Hi, the model has been uploaded, see: https://huggingface.co/mymusise/CPM-Third-Party<|||||>@mymusise Awesome news! Let me take care of the rest and I will keep you updated.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>😄 Is there any news? 
I have tested this before and the result is different from the official repo. 
Has this problem been solved? (I don't have a card now so I can't test it, sorry😢.)
If so, I will close this issue. @mymusise @JetRunner <|||||>> I have tested this before and the result is different from the official repo.

Hi, there are a few problems with the old one. I've recreated the model card, please try the new one: [mymusise/CPM-GPT2](https://huggingface.co/mymusise/CPM-GPT2)

and the FP16 version has been uploaded already: [mymusise/CPM-GPT2-FP16](https://huggingface.co/mymusise/CPM-GPT2-FP16)
<|||||>Thanks, it works perfectly! 😄