<h2 dir="auto"><g-emoji class="g-emoji" alias="question" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/2753.png">❓</g-emoji> Questions &amp; Help</h2>
<p dir="auto">I am trying to fine-tune XLM on the SQuAD dataset.<br>
The command is as following:<br>
[CUDA_VISIBLE_DEVICES=0 python run_squad.py --model_type xlm --model_name_or_path xlm-mlm-tlm-xnli15-1024 --do_train --do_eval --train_file $SQUAD_DIR/train-v1.1.json --predict_file $SQUAD_DIR/dev-v1.1.json --per_gpu_train_batch_size 12 --learning_rate 3e-5 --num_train_epochs 100 --max_seq_length 384 --doc_stride 128 --eval_all_checkpoints --save_steps 50 --evaluate_during_training --output_dir /home/weihua/Sqad/transformers/xlm_out/ ]<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/43492059/68640277-2da52700-0542-11ea-981b-c748e918e1d8.png"><img src="https://user-images.githubusercontent.com/43492059/68640277-2da52700-0542-11ea-981b-c748e918e1d8.png" alt="WeChat Image_20191112113346" style="max-width: 100%;"></a></p>
<p dir="auto">All the parameters I set are in accordance with the example in the pytorch-transformers document. But Exact and F1 scores have barely increased. And the loss of each epoch also drops very slowly. Each epoch drops by about 0.01.<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/43492059/68640296-3f86ca00-0542-11ea-93b3-db1bd64539b2.png"><img src="https://user-images.githubusercontent.com/43492059/68640296-3f86ca00-0542-11ea-93b3-db1bd64539b2.png" alt="68477603-d6b10080-0268-11ea-947f-995b157da8d3" style="max-width: 100%;"></a></p>
<p dir="auto">Is there a problem with my parameter settings? Or do I need to adjust some parts of the model when I use XLM?</p>
<h3 dir="auto">Feature request</h3>
<p dir="auto">Similarly to the <code class="notranslate">max_new_tokens</code>, a <code class="notranslate">min_new_tokens</code> option would count only the newly generated tokens, ignoring the tokens of the input sequence (prompt) in decoder only models.</p>
<h3 dir="auto">Motivation</h3>
<p dir="auto">The option <code class="notranslate">min_length</code> of the <code class="notranslate">generate()</code> method might be ambiguous for decoder only models. It is not clear if decoder only models consider the length of the input (prompt) for the <code class="notranslate">min_length</code> condition or only the newly generated tokens.<br>
In Encoder Decoder (seq2seq) it is clear though.</p>
<h3 dir="auto">Your contribution</h3>
<p dir="auto">Not that I remember. But I could test it.</p>