readme: ---
  language: zh
  datasets: jinyong
  inference:
    parameters:
      max_length: 108
      num_return_sequences: 1
      do_sample: True
  widget: 
  - text: "杨过朗声说道：今番良晤，豪兴不浅，他日江湖相逢，再当杯酒言欢。咱们就此别过。 -"
    example_title: "神雕侠侣"
  - text: "乱世之际，人不如狗。 -"
    example_title: "射雕英雄传"
  ---
  
  
  
  
  # 飞雪连天射白鹿，笑书神侠倚碧鸳
  
  ## Model description
  
    AI生成金庸小说，给出开头续写。
  
  ## How to use
  使用 pipeline 调用模型:
  
  ```python
  >>> # 调用微调后的模型
  >>> senc="这些雪花落下来,多么白,多么好看.过几天太阳出来,每一片 雪花都变得无影无踪.到得明年冬天,又有许很多多雪花,只不过已不是 今年这些雪花罢了。"
  >>> model_id="jinyong-gpt2-finetuning"
  >>> from transformers import AutoTokenizer, GPT2LMHeadModel, TextGenerationPipeline
  
  >>> tokenizer = AutoTokenizer.from_pretrained(model_id) 
  >>> model = GPT2LMHeadModel.from_pretrained(model_id)
  >>> text_generator = TextGenerationPipeline(model, tokenizer)   
  >>> text_generator.model.config.pad_token_id = text_generator.model.config.eos_token_id
  >>> text_generator( senc,max_length=108, do_sample=True)
  [{'generated_text': '这些雪花落下来,多么白,多么好看.过几天太阳出来,每一片 雪花都变得无影无踪.到得明年冬天,又有许很多多雪花,只不过已不是 今年这些雪花罢了。 反正 老天爷 有眼 ， 不知 哪里 是甚么 风 险 ？” 正 说到此处 ， 突然 听得 谢逊 啸声 渐近 ， 忍不住 张口 惊呼 ， 一齐 向他 扑去 ， 只听 谢逊 一声 怒吼 ， 跟着 左手 用力 拍 出一掌 ， 以 掌力 化开 。 众人 吃了一惊 ， 同时 从 海 道 中 跃出 ， 双双 倒退 。 张翠山和殷素素 对望一眼 ， 均想 以 这两 大高手 之力 如何 抵挡 ， 以 今日 之力 如何 攻敌 之'}]
  >>> 
  ```
  Here is how to use this model to get the features of a given text in PyTorch:
  
  ```python
  from transformers import AutoTokenizer, AutoModelForCausalLM
  
  tokenizer = AutoTokenizer.from_pretrained("supermy/jinyong-gpt2")
  
  model = AutoModelForCausalLM.from_pretrained("supermy/jinyong-gpt2")
  ```
  
  
  
  ## Training data
  
  此数据集基于金庸的【飞雪连天射白鹿，笑书神侠倚碧鸳】小说集训练。
  
  ## 统计信息
  
  ```
  ```
  
  ## Training procedure
  
  基于模型：[GPT2](https://huggingface.co/gpt2) 
  训练环境：英伟达16G显卡
  
  bpe分词："vocab_size"=30000
  ```
  [INFO|trainer.py:1608] 2022-12-02 19:52:59,024 >> ***** Running training *****
  [INFO|trainer.py:1609] 2022-12-02 19:52:59,024 >>   Num examples = 9443
  [INFO|trainer.py:1610] 2022-12-02 19:52:59,024 >>   Num Epochs = 108
  [INFO|trainer.py:1611] 2022-12-02 19:52:59,024 >>   Instantaneous batch size per device = 12
  [INFO|trainer.py:1612] 2022-12-02 19:52:59,024 >>   Total train batch size (w. parallel, distributed & accumulation) = 12
  [INFO|trainer.py:1613] 2022-12-02 19:52:59,024 >>   Gradient Accumulation steps = 1
  [INFO|trainer.py:1614] 2022-12-02 19:52:59,024 >>   Total optimization steps = 84996
  [INFO|trainer.py:1616] 2022-12-02 19:52:59,025 >>   Number of trainable parameters = 124439808
  
  [INFO|trainer.py:1608] 2022-12-03 21:44:00,182 >> ***** Running training *****
  [INFO|trainer.py:1609] 2022-12-03 21:44:00,182 >>   Num examples = 9443
  [INFO|trainer.py:1610] 2022-12-03 21:44:00,182 >>   Num Epochs = 216
  [INFO|trainer.py:1611] 2022-12-03 21:44:00,182 >>   Instantaneous batch size per device = 12
  [INFO|trainer.py:1612] 2022-12-03 21:44:00,182 >>   Total train batch size (w. parallel, distributed & accumulation) = 12
  [INFO|trainer.py:1613] 2022-12-03 21:44:00,182 >>   Gradient Accumulation steps = 1
  [INFO|trainer.py:1614] 2022-12-03 21:44:00,182 >>   Total optimization steps = 169992
  [INFO|trainer.py:1616] 2022-12-03 21:44:00,183 >>   Number of trainable parameters = 124439808
  [INFO|trainer.py:1637] 2022-12-03 21:44:00,184 >>   Continuing training from checkpoint, will skip to saved global_step
  [INFO|trainer.py:1638] 2022-12-03 21:44:00,184 >>   Continuing training from epoch 107
  [INFO|trainer.py:1639] 2022-12-03 21:44:00,184 >>   Continuing training from global step 84500
  
  [INFO|trainer.py:1608] 2022-12-05 07:36:13,626 >> ***** Running training *****
  [INFO|trainer.py:1609] 2022-12-05 07:36:13,626 >>   Num examples = 9443
  [INFO|trainer.py:1610] 2022-12-05 07:36:13,626 >>   Num Epochs = 368
  [INFO|trainer.py:1611] 2022-12-05 07:36:13,626 >>   Instantaneous batch size per device = 12
  [INFO|trainer.py:1612] 2022-12-05 07:36:13,626 >>   Total train batch size (w. parallel, distributed & accumulation) = 12
  [INFO|trainer.py:1613] 2022-12-05 07:36:13,626 >>   Gradient Accumulation steps = 1
  [INFO|trainer.py:1614] 2022-12-05 07:36:13,626 >>   Total optimization steps = 289616
  [INFO|trainer.py:1616] 2022-12-05 07:36:13,627 >>   Number of trainable parameters = 124439808
  [INFO|trainer.py:1637] 2022-12-05 07:36:13,628 >>   Continuing training from checkpoint, will skip to saved global_step
  [INFO|trainer.py:1638] 2022-12-05 07:36:13,628 >>   Continuing training from epoch 255
  [INFO|trainer.py:1639] 2022-12-05 07:36:13,628 >>   Continuing training from global step 201000
  
  {'loss': 8.0431, 'learning_rate': 4.970998635229893e-05, 'epoch': 0.64}
  {'loss': 7.4867, 'learning_rate': 4.94158548637583e-05, 'epoch': 1.27}
  {'loss': 7.322, 'learning_rate': 4.912172337521766e-05, 'epoch': 1.91}
  ......
  {'loss': 3.901, 'learning_rate': 2.5010882865076008e-05, 'epoch': 108.01}
  {'loss': 3.8959, 'learning_rate': 2.4863817120805686e-05, 'epoch': 108.64}
  ......
  {'loss': 3.1625, 'learning_rate': 4.6090404254317857e-07, 'epoch': 214.1}
  {'loss': 3.1592, 'learning_rate': 3.1413242976140055e-07, 'epoch': 214.74}
  {'loss': 3.1625, 'learning_rate': 1.6706668549108195e-07, 'epoch': 