🌟 Mirostat decoding algorithm
# 🌟 Mirostat: A Perplexity-Controlled Neural Text Decoding Algorithm

## Description

Paper : https://arxiv.org/pdf/2007.14966.pdf

Abstract : 

> [...] We use this analysis to design a feedbackbased adaptive top-k text decoding algorithm called mirostat that generates text (of any length) with a redetermined value of perplexity, and thereby high-quality text without any tuning. [...] Mirostat avoids both traps: experiments show that cross-entropy has a near-linear relation with repetition in generated text. This relation is almost independent of the sampling method but slightly dependent on the model used. Hence, for a given language model, control over perplexity also gives control over repetitions.

## Open source status

* [x] the model implementation is available: https://github.com/basusourya/mirostat
* [x] the model weights are available: _Not applicable_
* [x] who are the authors: @basusourya 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
