Generation: only search for eos_token if set
In generation, the current check for `unfinished_sequences.max()`, which is to find sequences that have ended early via `eos_token_id`, creates a synchronization point even when there is no `eos_token`, which slows inference down.

This pull request moves that calculation to inside the condition checking for an `eos_token`, so that such slowdown may be removed by disabling this token.

On my old system with `iommu=soft`, this change is saving me 6 seconds per token on a large model by setting `model.config.eos_token_id = None`.

@gante 
_The documentation is not available anymore as the PR was closed or merged._<|||||>> 1. If you rebase with `main`, you'll see that there is another decoding method with the same pattern. Would you be able to rebase and push the change there as well?

Done. I have not reviewed the rest of the function for synchronization points but observe the impact of this change will be much smaller with the nested assistant loop. Nice new feature.

> 2. How's the system where you noticed the big speed change? How are you running `.generate()`? I'd be interested in knowing more about it :)

This is an old Asus KGPE-D16 motherboard. They've been popular with some independent developers as higher-end hackable hardware. It has two old K80s in it and is running the Dasharo third-party bios firmware, which adds support for newer PCI cards.

The firmware is not fully polished and I was observing corruption when transferring data between cards; the solution from nvidia's forums was to pass `iommu=soft` to the kernel. This fixes the issue in a pinch but makes data transfer very slow, and points where data is transferred became the biggest bottlenecks.

The generation call I'm presently using is roughly ````model.generate(input_ids,
            do_sample=False,
            min_length=10,
            max_length=50,
            top_p=0.95,
            temperature=0.0)```` from the cuda branch of the gptq llama repository.

The model approaches 40GB in size and is spread across all 4 logical cards using huggingface accelerate `device_map="auto"`. (I've made another small patch to transformers, not submitted yet (it would affect every model, hard to test), to additionally reduce the need to transfer data between the cards inside the model layer loop, when running with accelerate. The attention mask and position ids are not properly moved off the first card to prepare for layers on other cards, with the vanilla code.)<|||||>@xloem thank you for the explanation! 💛 