pytorch jax gpt2 lm-head causal-lm transformers text-generation
.gitattributes README.md added_tokens.json config.json flax_model.msgpack merges.txt pytorch_model.bin special_tokens_map.json tokenizer_config.json vocab.json
<h1>BreitBot</h1><h2>Timothy W. Dooley</h2>___________________________________________________<h3>GitHub</h3>The GitHub for the project can be found [here](https://github.com/twdooley/election_news)<h3>Model</h3><br>This model was trained on about 16,000 headlines from Breitbart.com spannning March 2019- 11 November 2020. The purpose of this project was to better understand how strongly polarized news crafts a narrative through Natural Language Processing. The BreitBot model was specifically created to understand the 'clickbaity' nature of a Breitbart headline. Many of the results are 'reasonable' within the scope of Breitbart's production. I will leave it to the user to make further interpretation. The full project noted that over 70% of Breitbart's articles from month to month have a negative sentiment score. Subjectively, I believe this is shown through the headlines generated.<br><h3>Training</h3><br>BreitBot is a finetuned on GPT2 with about 16,000 headlines. The maximum length allowed in the tokenizer was the length of the longest headline (~50 tokens). A huge credit goes to Richard Bownes, PhD whose article ["Fine Tuning GPT-2 for Magic the Gathering Flavour Text Generation"](https://medium.com/swlh/fine-tuning-gpt-2-for-magic-the-gathering-flavour-text-generation-3bafd0f9bb93) provided incredible direction and help in training this model. It was trained using a GPU on Google Colab. 