readme: # GPT-NYC-nontoxic
  
  ## About
  
  GPT2 (small version on HF) fine-tuned on questions and responses from https://reddit.com/r/asknyc
  
  I filtered comments to ones with scores >= 3, and responding directly
  to the original post ( = ignoring responses to other commenters).
  I also added many tokens which were common on /r/AskNYC but missing from
  GPT2.
  
  Additional <Toxic> and <NonToxic> tokens control following output.
  Toxic comments (about 5.5% of input data) are those which were flagged
  by [Perspective API](https://developers.perspectiveapi.com) with toxicity > 0.7,
  or by [English DeHateBERT](https://huggingface.co/Hate-speech-CNERG/dehatebert-mono-english),
  with <NonToxic> tagging for all comments related to LGBTQ identity 
  to avoid false positives / more aggressive censorship from these classifiers.
  
  Try prompting with ```question? - additional info %% <Toxic> ```
  Or ```question? - additional info %% <NonToxic>```
  
  ## Other options
  
  The [gpt-nyc-small](https://huggingface.co/monsoon-nlp/gpt-nyc-small) repo is based
  on GPT2 [small] but without the <Toxic> and <NonToxic> tags. It is the most
  directly comparable model to this one.
  
  The main [gpt-nyc](https://huggingface.co/monsoon-nlp/gpt-nyc) repo is based
  on GPT2-Medium and comes off more accurate. It does not have Toxic/NonToxic tagging.
  
  ## Blog
  
  Initial model: https://mapmeld.medium.com/gpt-nyc-part-1-9cb698b2e3d
  
  ## Notebooks
  
  ### Data processing / new tokens
  
  https://colab.research.google.com/drive/13BOw0uekoAYB4jjQtaXTn6J_VHatiRLu
  
  ### Fine-tuning GPT2 (small)
  
  https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR
  
  ### Predictive text and probabilities
  
  Scroll to end of
  
  https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR
  
  to see how to install git-lfs and trick ecco into loading this.
  