Refactor generation sampling parameters (e.g. top k, temperature) into "Sampling" classes 
#4164 has a full description of the intention here. Basically, to avoid exploding generate(...) with more arguments, I've added one generic Sampler parameter that allows for arbitrary transformations of the generation probability distribution conditioned on the past. This allows users to specify custom ways of sampling (e.g. insert a specific token after a previous one, etc.)

In the process, I've added some basic tests around these samplers; existing tests pass otherwise.
@sshleifer thanks for taking a look. The run against the tests you mentioned (bart/t5/marian) passed when I gave them a kick. When you say performance, this approach should have the same amount of compute (each enabled Sampler runs once per generation loop) since it is just moving code around unless I missed something. Let me do a rebase and see if that CI failure goes away -- let me know if you have any other concerns! <|||||>@turtlesoupy - thanks a lot for the PR! Cool design choice! 

The `generate` method definitely needs a bigger refactor sooner or later and this is a cool idea on how to make it easier to add new probability distribution wrap functions. With this design I'm a bit worried that we restrict beam search too much in a sense that only the log_softmax of the "next_tokens" distribution can "wrapped" but not the summed distribution of the `next_token_scorers + beam_scores`. Here this will break the beam search + sampling case (if I understood the code correctly).

I guess a method that adapts the `_beam_scores + next_token_scores` could also be used in "greedy" beam search in the future and this design choice would block us a bit. But I'm not sure whether there are many use cases one would like to adapt `_beam_scores + next_token_scores` before appling `top_k` for "greedy" beam search...what are your thoughts on this? @turtlesoupy @yjernite @sshleifer <|||||>@patrickvonplaten I'm un-opinionated since my use cases weren't using beam search; the goal of this PR was so that I could introduce a my own sampler that enforced rules without having to fork the generate function.

For beam search, one approach could be to apply the warp to (`next_token_scores + beam_scores`) and then perform sampling afterwards. Then it is sampling from a consistent space and the hypothesis scores would be modified appropriately <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
