Contrastive Search does not work at all for Llama 7B
### System Info

latest transofrmers and everything, rtx 3090

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

@yxuansu 

I tried it with Llama 7B and results are very bad, it does not generate anything. Also i tried with T5 and results are very bad, but it is probably not intended to work with T5.


    import transformers
    import torch
    from transformers import AutoTokenizer,AutoModelForCausalLM,LlamaTokenizer  
    tokenizer = AutoTokenizer.from_pretrained("huggyllama/llama-7b")
    model = AutoModelForCausalLM.from_pretrained(
            "huggyllama/llama-7b",
            load_in_8bit=False,
            torch_dtype=torch.float16,
            device_map="auto",
        )

    input_ids = tokenizer.encode('I enjoy walking with my cute dog', return_tensors='pt').to("cuda")
    beam_output = model.generate(
    input_ids, 
    penalty_alpha=0.6, 
    top_k=4,
    max_length=100, 
    )

    print(tokenizer.decode(beam_output[0], skip_special_tokens=True))    

This generates just a dot.

    I enjoy walking with my cute dog.










### Expected behavior

It should generate text actually
Actually it does work very well for text generation. It just does not generate anything with this prompt for some reason.<|||||>Thanks for investigating! <|||||>As always thanks for hard work on implementing such cool thing as this one. I will share how much it helped, seem like it increases accuracy, but I will need to check much more examples to tell.