<h3 dir="auto">Feature request</h3>
<p dir="auto"><g-emoji class="g-emoji" alias="hugs" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f917.png">🤗</g-emoji> Accelerate has a wrapper to help with distributed metric calculation (a tough problem!), and the <code class="notranslate">no_trainer</code> scripts should be updated to include it!</p>
<p dir="auto">An example can be seen <a href="https://github.com/huggingface/accelerate/blob/main/examples/nlp_example.py#L163-L169">here</a>, below is an example diff of what the integration would look like:</p>
<div class="highlight highlight-source-diff notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="-             predictions, references = accelerator.gather((predictions, batch[&quot;labels&quot;]))
-             # If we are in a multiprocess environment, the last batch has duplicates
-             if accelerator.num_processes &gt; 1:
-                 if step == len(eval_dataloader) - 1:
-                     predictions = predictions[: len(eval_dataloader.dataset) - samples_seen]
-                     references = references[: len(eval_dataloader.dataset) - samples_seen]
-                 else:
-                     samples_seen += references.shape[0]
+             predictions, references = accelerator.gather_for_metrics((predictions, batch[&quot;labels&quot;])) "><pre class="notranslate"><span class="pl-md"><span class="pl-md">-</span>             predictions, references = accelerator.gather((predictions, batch["labels"]))</span>
<span class="pl-md"><span class="pl-md">-</span>             # If we are in a multiprocess environment, the last batch has duplicates</span>
<span class="pl-md"><span class="pl-md">-</span>             if accelerator.num_processes &gt; 1:</span>
<span class="pl-md"><span class="pl-md">-</span>                 if step == len(eval_dataloader) - 1:</span>
<span class="pl-md"><span class="pl-md">-</span>                     predictions = predictions[: len(eval_dataloader.dataset) - samples_seen]</span>
<span class="pl-md"><span class="pl-md">-</span>                     references = references[: len(eval_dataloader.dataset) - samples_seen]</span>
<span class="pl-md"><span class="pl-md">-</span>                 else:</span>
<span class="pl-md"><span class="pl-md">-</span>                     samples_seen += references.shape[0]</span>
<span class="pl-mi1"><span class="pl-mi1">+</span>             predictions, references = accelerator.gather_for_metrics((predictions, batch["labels"])) </span></pre></div>
<p dir="auto">The list of available scripts to update include:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/image-classification/run_image_classification_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/language-modeling/run_clm_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/language-modeling/run_mlm_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/multiple-choice/run_swag_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/question-answering/run_qa_beam_search_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/question_answering/run_qa_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/semantic-segmentation/run_semantic_segmentation_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/speech-pretraining/run_wav2vec2_pretraining_no_trainer.py</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> examples/pytorch/summarization/run_summarization_no_trainer.py</li>
</ul>
<h3 dir="auto">Motivation</h3>
<p dir="auto">This is a great first issue for someone who wants to learn how to use some of the latest bits in Accelerate and get an easy beginner contribution to the library <g-emoji class="g-emoji" alias="hugs" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f917.png">🤗</g-emoji></p>
<h3 dir="auto">Your contribution</h3>
<p dir="auto">If you decide to pick up this issue, feel free to ping myself (<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/muellerzr/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/muellerzr">@muellerzr</a>), <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a>, or <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pacman100/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pacman100">@pacman100</a> to review <g-emoji class="g-emoji" alias="hugs" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f917.png">🤗</g-emoji></p>
<h1 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji> Feature request</h1>
<p dir="auto"><code class="notranslate">T5ForConditionalGeneration</code> inherits <code class="notranslate">generate</code> from <code class="notranslate">PreTrainedModel</code> via <code class="notranslate">GenerationMixin</code>.</p>
<p dir="auto">Currently in our docs if we write:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="for more info see :meth:`transformers.T5ForConditionalGeneration.generate`"><pre class="notranslate"><code class="notranslate">for more info see :meth:`transformers.T5ForConditionalGeneration.generate`
</code></pre></div>
<p dir="auto">it doesn't get resolved in the html docs (it doesn't get a xref link, but leaves it as a text)</p>
<p dir="auto">Neither <code class="notranslate">transformers.PreTrainedModel.generate</code> gets resolved - which is the super-class that has this method.</p>
<p dir="auto">But since it's a mixin and actually comes from <code class="notranslate">transformers.generation_utils.GenerationMixin</code> the only way to get the cross-reference turned into a working link is to write:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="for more info see :meth:`transformers.generation_utils.GenerationMixin.generate`"><pre class="notranslate"><code class="notranslate">for more info see :meth:`transformers.generation_utils.GenerationMixin.generate`
</code></pre></div>
<p dir="auto">which is slow and complicated to figure out and non-intuitive. And this was just one example.</p>
<p dir="auto">Both <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a> and I tried to find a sphinx configuration to make it resolve <code class="notranslate">transformers.T5ForConditionalGeneration.generate</code> to <code class="notranslate">transformers.generation_utils.GenerationMixin.generate</code> and point to the right doc entry, but didn't succeed.</p>
<p dir="auto">This could be related to the fact that we don't generate docs for inherited methods and only document them in the parent class. We want to keep it that way but to be able to cross-reference the inherited methods in the docstrings.</p>
<p dir="auto">If you perhaps have an expertise in sphinx and could help us resolve this it'd be absolutely amazing!</p>
<p dir="auto">T5 was just an example, this is needed for dozens of models.</p>
<p dir="auto">If there is no way to currently do it you could also submit a feature request with sphinx and take a lead on following through for this new feature to be added.</p>
<p dir="auto">Bonus:</p>
<p dir="auto">As explained above we don't duplicate inherited methods docs, but it'd be really useful to list the inherited methods which didn't get documented in the sub-classes, e.g.: <a href="https://huggingface.co/transformers/model_doc/t5.html#tft5forconditionalgeneration" rel="nofollow">https://huggingface.co/transformers/model_doc/t5.html#tft5forconditionalgeneration</a> could have a brief entry:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Inherited methods: generate, greedy_search, sample, beam_search, beam_sample, group_beam_search"><pre class="notranslate"><code class="notranslate">Inherited methods: generate, greedy_search, sample, beam_search, beam_sample, group_beam_search
</code></pre></div>
<p dir="auto">with cross-references to the corresponding full doc entries in the super-class. Of course, while this could be created manually, it'd be very difficult to maintain - so we need to this to be autogenerated.</p>
<p dir="auto">Thank you!</p>
<p dir="auto">Related issues/prs:</p>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="849737172" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/11049" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/11049/hovercard" href="https://github.com/huggingface/transformers/pull/11049">#11049</a></li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="771239511" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/9202" data-hovercard-type="issue" data-hovercard-url="/huggingface/transformers/issues/9202/hovercard" href="https://github.com/huggingface/transformers/issues/9202">#9202</a></li>
</ul>