readme: # Punctuator for Simplified Chinese
  
  The model is fine-tuned based on `DistilBertForTokenClassification` for adding punctuations to plain text (simplified Chinese). The model is fine-tuned based on distilled model `bert-base-chinese`.
  
  ## Usage
  
  ```python
  from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast
  
  model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_zh")
  tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_zh")
  ```
  
  ## Model Overview
  
  ### Training data
  Combination of following three dataset:
  
  - News articles of People's Daily 2014. [Reference](https://github.com/InsaneLife/ChineseNLPCorpus)
  
  ### Model Performance
  - Validation with MSRA training dataset. [Reference](https://github.com/InsaneLife/ChineseNLPCorpus/tree/master/NER/MSRA)
  - Metrics Report:
      |                  | precision | recall | f1-score | support |
      |:----------------:|:---------:|:------:|:--------:|:-------:|
      |      C_COMMA     |    0.67   |  0.59  |   0.63   |  91566  |
      |     C_DUNHAO     |    0.50   |  0.37  |   0.42   |  21013  |
      | C_EXLAMATIONMARK |    0.23   |  0.06  |   0.09   |   399   |
      |     C_PERIOD     |    0.84   |  0.99  |   0.91   |  44258  |
      |  C_QUESTIONMARK  |    0.00   |  1.00  |   0.00   |    0    |
      |     micro avg    |    0.71   |  0.67  |   0.69   |  157236 |
      |     macro avg    |    0.45   |  0.60  |   0.41   |  157236 |
      |   weighted avg   |    0.69   |  0.67  |   0.68   |  157236 |
  
  
  