Top-k sampling and top-p sampling for generating phrases on batches with GPT-2?
How can I generate on batches with GPT-2 if I want to make use of these awesome sampling techniques: [top-k sampling and top-p sampling](https://huggingface.co/blog/how-to-generate)?. 

There is this implementation for generating phrases on batches already in issue [#3021](https://github.com/huggingface/transformers/issues/3021). 

Any advice? thanks!

@patrickvonplaten 

It's on our ToDo-List :-) Currently batch generation with GPT2 is not possible, so you will have to rely on the code in https://github.com/huggingface/transformers/issues/3021<|||||>Can top-k and top-p sampling be implemented in batches?<|||||>Sure, the provided `top-k-top-p sampling` function provided that :-) 