readme: ---
  language: en
  datasets: COCA
  ---
  # docusco-bert
  
  ## Model description
  
  **docusco-bert** is a fine-tuned BERT model that is ready to use for **token classification**. The model was trained on data sampled from the Corpus of Contemporary American English ([COCA](https://www.english-corpora.org/coca/)) and classifies tokens and token sequences according to a system developed for the [**DocuScope**](https://www.cmu.edu/dietrich/english/research-and-publications/docuscope.html) dictionary-based tagger. Descriptions of the categories are included in a table below. 
  
  ## About DocuScope
  DocuScope is a dicitonary-based tagger that has been developed at Carnegie Mellon University by **David Kaufer** and **Suguru Ishizaki** since the early 2000s. Its categories are rhetorical in their orientation (as opposed to part-of-speech tags, for example, which are morphosyntactic).
  
  DocuScope has been been used in [a wide variety of studies](https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=docuscope&btnG=). Here, for example, is [a short analysis of King Lear](https://graphics.cs.wisc.edu/WP/vep/2017/02/14/guest-post-data-mining-king-lear/), and here is [a published study of Tweets](https://journals.sagepub.com/doi/full/10.1177/2055207619844865).
  
  
  ## Intended uses & limitations
  
  #### How to use
  
  The model was trained on data with tags formatted using [IOB](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)), like those used in common tasks like Named Entity Recogition (NER). Thus, you can use this model with a Transformers NER *pipeline*.
  
  ```python
  from transformers import AutoTokenizer, AutoModelForTokenClassification
  from transformers import pipeline
  tokenizer = AutoTokenizer.from_pretrained("browndw/docusco-bert")
  model = AutoModelForTokenClassification.from_pretrained("browndw/docusco-bert")
  nlp = pipeline("ner", model=model, tokenizer=tokenizer)
  example = "Globalization is the process of interaction and integration among people, companies, and governments worldwide."
  ds_results = nlp(example)
  print(ds_results)
  ```
  
  #### Limitations and bias
  
  This model is limited by its training dataset of American English texts. Moreover, the current version is trained on only a small subset of the corpus. The goal is to train later versions on more data, which should increase accuracy. 
  
  ## Training data
  
  This model was fine-tuned on data from the Corpus of Contemporary American English ([COCA](https://www.english-corpora.org/coca/)). The training data contain chunks of text randomly sampled of 5 text-types: Academic, Fiction, Magazine, News, and Spoken.
  
  Typically, BERT models are trained on sentence segments. However, DocuScope tags can span setences. Thus, data were split into chunks that don't split **B + I** sequences and end with sentence-final punctuation marks (i.e., period, quesiton mark or exclamaiton point).
  
  Additionally, the order of the chunks was randomized prior to sampling, and statified sampling was used to provide enough training data for low-frequency caegories. The resulting training data consist of:
  
  * 21,460,177 tokens
  * 15,796,305 chunks
  
  The specific counts for each category appear in the following table.
  
  Category|Count
  -|-
  O|3528038
  Syntactic Complexity|2032808
  Character|1413771
  Description|1224744
  Narrative|1159201
  Negative|651012
  Academic Terms|620932
  Interactive|594908
  Information Exposition|578228
  Positive|463914
  Force Stressed|432631
  Information Topics|394155
  First Person|249744
  Metadiscourse Cohesive|240822
  Strategic|238255
  Public Terms|234213
  Reasoning|213775
  Information Place|187249
  Information States|173146
  Information ReportVerbs|119092
  Confidence High|112861
  Confidence Hedged|110008
  Future|96101
  Inquiry|94995
  Contingent|94860
  Information Change|89063
  Metadiscourse Interactive|84033
  Updates|81424
  Citation|71241
  Facilitate|50451
  Uncertainty|35644
  Academic WritingMoves|29352
  Information ChangePositive|28475
  Responsibility|25362
  Citation Authority|22414
  Information ChangeNegative|15612
  Confidence Low|2876
  Citation Hedged|895
  -|-
  Total|15796305
  
  ## Training procedure
  
  This model was trained on a single 2.3 GHz Dual-Core Intel Core i5 with recommended hyperparameters from the [original BERT paper](https://arxiv.org/pdf/1810.04805). 
  
  ## Eval results
  ### Overall
  metric|test
  -|-
  f1 |.927
  accuracy |.943
  
  ### By category
  category|precision|recall|f1-score|support
  -|-|-|-|-
  AcademicTerms|0.91|0.92|0.92|486399
  AcademicWritingMoves|0.76|0.82|0.79|20017
  Character|0.94|0.95|0.94|1260272
  Citation|0.92|0.94|0.93|50812
  CitationAuthority|0.86|0.88|0.87|17798
  CitationHedged|0.91|0.94|0.92|632
  ConfidenceHedged|0.94|0.96|0.95|90393
  ConfidenceHigh|0.92|0.94|0.93|113569
  ConfidenceLow|0.79|0.81|0.80|2556
  Contingent|0.92|0.94|0.93|81366
  Description|0.87|0.89|0.88|1098598
  Facilitate|0.87|0.90|0.89|41760
  FirstPerson|0.96|0.98|0.97|330658
  ForceStressed|0.93|0.94|0.93|436188
  Future|0.90|0.93|0.92|93365
  InformationChange|0.88|0.91|0.89|72813
  InformationChangeNegative|0.83|0.85|0.84|12740
  I