<h1 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji> Feature request</h1>
<p dir="auto">We have:</p>
<ol dir="auto">
<li><code class="notranslate">finetune_trainer.py</code> has</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    n_train: Optional[int] = field(default=-1, metadata={&quot;help&quot;: &quot;# training examples. -1 means use all.&quot;})
    n_val: Optional[int] = field(default=-1, metadata={&quot;help&quot;: &quot;# validation examples. -1 means use all.&quot;})
    n_test: Optional[int] = field(default=-1, metadata={&quot;help&quot;: &quot;# test examples. -1 means use all.&quot;})"><pre class="notranslate"><code class="notranslate">    n_train: Optional[int] = field(default=-1, metadata={"help": "# training examples. -1 means use all."})
    n_val: Optional[int] = field(default=-1, metadata={"help": "# validation examples. -1 means use all."})
    n_test: Optional[int] = field(default=-1, metadata={"help": "# test examples. -1 means use all."})
</code></pre></div>
<ol start="2" dir="auto">
<li>some other <code class="notranslate">run_</code> scripts use <code class="notranslate">--n_obs</code></li>
<li><code class="notranslate">--max_steps</code> in the main trainer - which works only on the train_dataset - no ability to limit items on eval_dataset</li>
</ol>
<p dir="auto">Requests/Questions:</p>
<ol dir="auto">
<li>How does one use <code class="notranslate">--max_steps</code> if one needs to use a different number of items for train and eval?</li>
<li>Can we have a consistent way across examples to do this same thing?</li>
</ol>
<p dir="auto">Thank you.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a></p>
<p dir="auto">The <code class="notranslate">examples</code> have an incosistency of how the cl args are defined and parsed. Some rely on PL's main args as <code class="notranslate">finetune.py</code> does: <a href="https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune.py#L410">https://github.com/huggingface/transformers/blob/master/examples/seq2seq/finetune.py#L410</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    parser = argparse.ArgumentParser()
    parser = pl.Trainer.add_argparse_args(parser)"><pre class="notranslate"><code class="notranslate">    parser = argparse.ArgumentParser()
    parser = pl.Trainer.add_argparse_args(parser)
</code></pre></div>
<p dir="auto">others like <code class="notranslate">run_pl_glue.py</code> rely on <code class="notranslate">lightening_base.py</code>'s main args: <a href="https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_pl_glue.py#L176">https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_pl_glue.py#L176</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    parser = argparse.ArgumentParser()
    add_generic_args(parser, os.getcwd())"><pre class="notranslate"><code class="notranslate">    parser = argparse.ArgumentParser()
    add_generic_args(parser, os.getcwd())
</code></pre></div>
<p dir="auto">now that we pushed <code class="notranslate">--gpus</code> into  <code class="notranslate">lightening_base.py</code>'s main args the scripts that run PL's main args collide and we have:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="fail.argparse.ArgumentError: argument --gpus: conflicting option string: --gpus"><pre class="notranslate"><code class="notranslate">fail.argparse.ArgumentError: argument --gpus: conflicting option string: --gpus
</code></pre></div>
<p dir="auto">i.e. PL already supplies <code class="notranslate">--gpus</code> and many other args that some of the scripts in <code class="notranslate">examples</code> re-define.</p>
<p dir="auto">So either the example scripts need to stop using <code class="notranslate">pl.Trainer.add_argparse_args(parser)</code> and rely exclusively on <code class="notranslate">lightning_base.add_generic_args</code>, or we need a different clean approach. It appears that different scripts have different needs arg-wise. But they all use <code class="notranslate">lightning_base</code>.</p>
<p dir="auto">The problem got exposed in: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="665494213" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/6027" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/6027/hovercard" href="https://github.com/huggingface/transformers/pull/6027">#6027</a> and <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="674625886" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/6307" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/6307/hovercard" href="https://github.com/huggingface/transformers/pull/6307">#6307</a></p>