[run_translation.py] out of range integral type conversion attempted
spitting off from https://github.com/huggingface/transformers/issues/22571 as it was a secondary problem reported there:

### Reproduction

```
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=src python examples/pytorch/translation/run_translation.py \
--model_name_or_path t5-base --do_train --do_eval --source_lang en \
--target_lang de --source_prefix 'translate English to German: ' \
--dataset_name stas/wmt14-en-de-pre-processed --output_dir \
/tmp/tst-translation --num_train_epochs 1 --per_device_train_batch_size=1 \
--max_train_samples 10 --overwrite_output_dir --seed 1137 \
--per_device_eval_batch_size 1 --predict_with_generate --fp16 \
--max_eval_samples 10
```

fails inside eval:

```
[INFO|trainer.py:3126] 2023-04-04 09:28:07,548 >> ***** Running Evaluation *****
[INFO|trainer.py:3128] 2023-04-04 09:28:07,548 >>   Num examples = 10
[INFO|trainer.py:3131] 2023-04-04 09:28:07,548 >>   Batch size = 1
[INFO|configuration_utils.py:575] 2023-04-04 09:28:07,552 >> Generate config GenerationConfig {
  "_from_model_config": true,
  "decoder_start_token_id": 0,
  "eos_token_id": 1,
  "pad_token_id": 0,
  "transformers_version": "4.28.0.dev0"
}

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00,  3.72it/s]Traceback (most recent call last):
  File "examples/pytorch/translation/run_translation.py", line 664, in <module>
    main()
  File "examples/pytorch/translation/run_translation.py", line 605, in main
    metrics = trainer.evaluate(max_length=max_length, num_beams=num_beams, metric_key_prefix="eval")
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 159, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 2990, in evaluate
    output = eval_loop(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 3278, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
  File "examples/pytorch/translation/run_translation.py", line 546, in compute_metrics
    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3445, in batch_decode
    return [
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3446, in <listcomp>
    self.decode(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_base.py", line 3485, in decode
    return self._decode(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
OverflowError: out of range integral type conversion attempted
```

@sgugger 
cc @ArthurZucker Seems like the model and tokenizer have mismatched length<|||||>Yeah, but :
- the tokenizer has 100 additional special tokens so even if the model predicts something above 32000 (the model's vocab size) you get an extra id (until 32099)
- the tokenizer has an `unk_token` so when you go above `32099`, the fast simply outputs `''` while the slow ` '<extra_id_-29>'` (which is a bit strange I'll give you that 😅 
snippet:
```python 
>>> from transformers import T5Tokenizer, T5TokenizerFast
>>> tokenizer_slow = T5Tokenizer.from_pretrained("t5-base")
>>> tokenizer_slow.decode(32140) # above vocab size
'<extra_id_-3167901>'
>>> tokenizer_fast = T5TokenizerFast.from_pretrained("t5-base")
''
```
The issue is different. This is a integer overflow in rust: 
```python 
>>> tokenizer_fast.decode(3200000000000)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
Cell In[29], line 1
----> 1 tokenizer_fast.decode(3200000000000)

File ~/Work/transformers/src/transformers/tokenization_utils_base.py:3485, in PreTrainedTokenizerBase.decode(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces, **kwargs)
   3482 # Convert inputs to python lists
   3483 token_ids = to_py_obj(token_ids)
-> 3485 return self._decode(
   3486     token_ids=token_ids,
   3487     skip_special_tokens=skip_special_tokens,
   3488     clean_up_tokenization_spaces=clean_up_tokenization_spaces,
   3489     **kwargs,
   3490 )

File ~/Work/transformers/src/transformers/tokenization_utils_fast.py:549, in PreTrainedTokenizerFast._decode(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces, **kwargs)
    547 if isinstance(token_ids, int):
    548     token_ids = [token_ids]
--> 549 text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
    551 clean_up_tokenization_spaces = (
    552     clean_up_tokenization_spaces
    553     if clean_up_tokenization_spaces is not None
    554     else self.clean_up_tokenization_spaces
    555 )
    556 if clean_up_tokenization_spaces:

OverflowError: out of range integral type conversion attempted
``` 
That means you are juste giving a huge huge number to decode is there a reason ?<|||||>Please note I've only relayed the errors reported on the pytorch Issued by a user trying to use `torch.compile`. <|||||>Hi guys,

I have the same problem with the `run_seq2seq_qa.py` script and it turns out, that `preds` are passed to the `decode` function, with the following content:

```
[[     0 250099   1013 ...   -100   -100   -100]  
 [     0 250099   1013 ...   -100   -100   -100]                          
 [     0 250099   1013 ...   -100   -100   -100]                          
 ...                                                                                     
 [     0 250099    260 ...   -100   -100   -100]              
 [     0 250099    442 ...   -100   -100   -100]
 [     0 250099   3883 ...   -100   -100   -100]]
```

So the problematic thing here is `-100` I guess, because I can reproduce the error with:

```
>>> tokenizer.decode(-100)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/transformers/src/transformers/tokenization_utils_base.py", line 3485, in decode
    return self._decode(
  File "/home/ubuntu/transformers/src/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
OverflowError: out of range integral type conversion attempted
```<|||||>Awsome thanks for providing this! Indeed these should be converted to padding<|||||>Could it be similar to this fix? https://github.com/huggingface/transformers/pull/18592
The hardcoded -100 doesn't seem to always do the right thing.<|||||>I tried with another model arch and it's breaks too but in another way. so eval is quite broken in many ways.

```
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=src python examples/pytorch/translation/run_translation.py --model_name_or_path  'facebook/wmt19-en-ru'  --do_train --do_eval --source_lang en --target_lang de --source_prefix 'translate English to German: ' --dataset_name stas/wmt14-en-de-pre-processed --output_dir /tmp/tst-translation --num_train_epochs 1 --per_device_train_batch_size=1 --max_train_samples 10 --overwrite_output_dir --seed 1137 --per_device_eval_batch_size 1 --predict_with_generate --fp16 --max_eval_samples 10


Traceback (most recent call last):
  File "examples/pytorch/translation/run_translation.py", line 664, in <module>
    main()
  File "examples/pytorch/translation/run_translation.py", line 605, in main
    metrics = trainer.evaluate(max_length=max_length, num_beams=num_beams, metric_key_prefix="eval")
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 159, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 2993, in evaluate
    output = eval_loop(
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer.py", line 3174, in evaluation_loop
    loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/trainer_seq2seq.py", line 290, in prediction_step
    outputs = model(**inputs)
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/mnt/nvme0/code/huggingface/transformers-master/src/transformers/models/fsmt/modeling_fsmt.py", line 1251, in forward
    masked_lm_loss = loss_fct(lm_logits.view(-1, self.config.tgt_vocab_size), labels.view(-1))
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 1174, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "/home/stas/anaconda3/envs/py38-pt20/lib/python3.8/site-packages/torch/nn/functional.py", line 3029, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
ValueError: Expected input batch_size (56) to match target batch_size (48).
```<|||||>@stas00  I am facing the same issue while fine-tuning t5-small using `examples/pytorch/summarization/run_summarization.py`
And I can see `preds` has `-100` and so decode fails with the below error:

```
Traceback (most recent call last):
  File "examples/pytorch/summarization/run_summarization.py", line 751, in <module> main()
  File "examples/pytorch/summarization/run_summarization.py", line 705, in main
    predict_results = trainer.predict(predict_dataset, metric_key_prefix="predict")
  File "src/transformers/trainer_seq2seq.py", line 216, in predict
    return super().predict(test_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
  File "src/transformers/trainer.py", line 3069, in predict
    output = eval_loop(
  File "src/transformers/trainer.py", line 3281, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
  File "examples/pytorch/summarization/run_summarization.py", line 635, in compute_metrics
    decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
  File "src/transformers/tokenization_utils_base.py", line 3446, in batch_decode
    return [
  File "src//transformers/tokenization_utils_base.py", line 3447, in <listcomp>
    self.decode(
  File "src/transformers/tokenization_utils_base.py", line 3486, in decode
    return self._decode(
  File "src/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
OverflowError: out of range integral type conversion attempted
```

<|||||>The first issue is addressed in #22693

The second issue with FSMT is due to [this line](https://github.com/huggingface/transformers/blob/151425ddb29d4ad1a121e8cce62000a2ac52d3ba/src/transformers/trainer_seq2seq.py#L270) added by @gante . The `decoder_input_ids` not passed to `generate` result in generations that have the same length as the inputs and not the targets.<|||||>@sgugger thanks for the fix. I can see the same issue in  line 718 https://github.com/huggingface/transformers/blob/main/examples/pytorch/summarization/run_summarization.py#L718

possible fix:
preds= np.where(predict_results.predictions != -100, predict_results.predictions, tokenizer.pad_token_id)
predictions = tokenizer.batch_decode(preds, skip_special_tokens=True, clean_up_tokenization_spaces=True)
<|||||>Good catch, adding this too in the PR.<|||||>Thinking more, I think this is also a result of the recent changes in generate, which used to be the one padding the result with `tokenizer.pad_token_id`, and it's now the `Trainer` padding them with -100. cc @gante <|||||>Hey everyone -- the last issues should be gone with #22772, but feel free to comment/reopen if any related problem persists!<|||||>Hi! since a couple of weeks I also stumbled on this error. It was working just fine before. I am pretty sure I have transformer installed from source so the PR with the fix is there as well. I am using Bart-large and the Trainer class.
I first define rouge as training evaluation function:
```python
    def compute_rouge(pred): 
        predictions, labels = pred
        #decode the predictions
        decode_predictions = tokenizer.batch_decode(predictions, skip_special_tokens=True)
        #decode labels
        decode_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)

        #compute results
        res = rouge.compute(predictions=decode_predictions, references=decode_labels, use_stemmer=True)
        #get %
        return res
``` 
And give it to the trainer
```python
       trainer = Seq2SeqTrainer(
                    model, 
                    args,
                    train_dataset=tokenized_dataset['train'],
                    eval_dataset=tokenized_dataset['valid'],
                    data_collator=collator,
                    tokenizer=tokenizer,
                    compute_metrics=compute_rouge
                )
``` 
Then the script breaks in Trainer.train, while decoding for dev set evaluation:


``` 
Traceback (most recent call last):
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/finetunemodel.py", line 226, in <module>
    main(args)
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/finetunemodel.py", line 149, in main
    trainer.train()
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 1662, in train
    return inner_training_loop(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 2022, in _inner_training_loop
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 2288, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer_seq2seq.py", line 159, in evaluate
    return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 2994, in evaluate
    output = eval_loop(
             ^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 3283, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/finetunemodel.py", line 103, in compute_rouge
    decode_predictions = tokenizer.batch_decode(predictions, skip_special_tokens=True)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3456, in batch_decode
    return [
           ^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3457, in <listcomp>
    self.decode(
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3496, in decode
    return self._decode(
            ^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: out of range integral type conversion attempted

```
Interestingly enough, on a similar formatted dataset (but longer text) while using Longformer (led), I get the same error but this time at prediction time, thus the trained is completed successfully: 
```
Traceback (most recent call last):
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/LED_4_DWIE.py", line 236, in <module>
    main(args)
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/LED_4_DWIE.py", line 161, in main
    preds, labels, metrics = trainer.predict(tokenized_dataset['test'], num_beams=5, min_length=50, max_length=max_target, no_repeat_ngram_size=2, early_stopping=True)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer_seq2seq.py", line 216, in predict
    return super().predict(test_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 3070, in predict
    output = eval_loop(
             ^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/trainer.py", line 3283, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/kg2Narrative/KGNarrative2/script4trainingLLM/LED_4_DWIE.py", line 103, in compute_rouge
    decode_predictions = tokenizer.batch_decode(predictions, skip_special_tokens=True)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3456, in batch_decode
    return [
           ^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3457, in <listcomp>
    self.decode(
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3496, in decode
    return self._decode(
           ^^^^^^^^^^^^^
  File "/home/ghoogerw/.conda/envs/kg2Narrative/lib/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 549, in _decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OverflowError: out of range integral type conversion attempted
```




<|||||>Hey @GabHoo -- could you share with us a short stand-alone script to reproduce the issue? :)<|||||>Thank you for time. Here is a standaone version of the script. I hope it is the case,

```python

 from transformers import AutoTokenizer,AutoModelForSeq2SeqLM,DataCollatorForSeq2Seq,Seq2SeqTrainingArguments,Seq2SeqTrainer
import os
from datasets import load_dataset
import numpy as np
from utils import *
import torch
import evaluate
import sys
import json
import time
import argparse

def tokenize_for_evaluation(tokenizer,preds,labels):


    predicted_text = []
    golden_labels = []

    for pred, label in zip(preds, labels):

        gen = tokenizer.decode(pred, skip_special_tokens=True)
        gen = str(gen)
        predicted_text.append(gen)

        gold = tokenizer.decode(label, skip_special_tokens=True)
        gold = str(gold)
        golden_labels.append(gold)

        return predicted_text,golden_labels
    
def process_data_BART(data_to_process,tokenizer,max_input,max_target,typeKG ):

    #get the dialogue text
    inputs = [graph for graph in data_to_process[f'{typeKG}']]
    #tokenize text
    model_inputs = tokenizer(inputs,  max_length=max_input, padding='max_length', truncation=True)

    #tokenize labels
    #with tokenizer.as_target_tokenizer():
    targets = [target for target in data_to_process['story']]
    model_targets = tokenizer(targets, max_length=max_target, padding='max_length', truncation=True)
    

    #reuturns input_ids, attention_masks, labels
    
    data_to_process["input_ids"] = model_inputs.input_ids
    data_to_process["attention_mask"] = model_inputs.attention_mask
    data_to_process["labels"] = model_targets.input_ids

    return data_to_process

    
datapath ='/daatapath
dataprefix ='pop'
typeKG = 'Instances_KG'
model_checkpoint="facebook/bart-base"
experiment_name = 'exp'
learning_rate =1e-4
batch_size = 1
epochs =3
save_model = False
max_target = 512
max_input = 512


train_file = datapath +'/' + dataprefix + '_train' + '.json'
dev_file = datapath +'/'+ dataprefix + '_dev' + '.json'
test_file = datapath +'/' + dataprefix + '_test'+ '.json'


print("Loading dataset from ",datapath)
dataset = load_dataset('json', data_files={'train': train_file, 'valid': dev_file, 'test': test_file})

todrop=list(set(dataset['test'].column_names)-set([typeKG,'story'])) #This line returns a list of all the columns to drop (all columns minus the ones we need (input typeKG and story))


print("Loading tokenizer")
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint,add_eos_token=True)

print("\nProcessing Dataset")
#the processing of the data is done batches for make it faster,number of processes 4
tokenized_dataset = dataset.map(lambda example: process_data_BART(example, tokenizer,max_input,max_target,typeKG), batched=True, num_proc=4,remove_columns=todrop)

print("\nLoading MODEL")
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
#model.to(device)

print("Collator for batches")
collator = DataCollatorForSeq2Seq(tokenizer, model=model) #this is necessary for diving in batch for training

print('Loading rouge')
rouge = evaluate.load('rouge')


def compute_rouge(pred): 
    predictions, labels = pred
    #decode the predictions
    decode_predictions = tokenizer.batch_decode(predictions, skip_special_tokens=True)
    #decode labels
    decode_labels = tokenizer.batch_decode(labels, skip_special_tokens=True,clean_up_tokenization_spaces=True)

    #compute results
    res = rouge.compute(predictions=decode_predictions, references=decode_labels, use_stemmer=True)
    #get %
    return res

print("\nPREPARING FOR TRAINING...")

#defining training arogouments
args = Seq2SeqTrainingArguments(
    experiment_name,
    evaluation_strategy='epoch',
    learning_rate=learning_rate, 
    per_device_train_batch_size= batch_size,
    per_device_eval_batch_size= batch_size,
    gradient_accumulation_steps=3, #compute gradient on n examples KG story 
    weight_decay=0.01, #regularization
    save_total_limit=1, #this is the max amount of checkpoint saved, after which previous checpoints are removed
    num_train_epochs=epochs, #number of epochs
    predict_with_generate=True, 
    generation_max_length = 512, #max number of tokens per generation 
    generation_num_beams=5, #decoding strategy! greedy search, beam search 
    eval_accumulation_steps=1, #backprop  
    fp16=True, #memory management
    disable_tqdm=True)
#only CUDA available -> fp16=True


### almost training time
trainer = Seq2SeqTrainer(
    model, 
    args,
    train_dataset=tokenized_dataset['train'],
    eval_dataset=tokenized_dataset['valid'],
    data_collator=collator,
    tokenizer=tokenizer,
    compute_metrics=compute_rouge
)


trainer.train()


if save_model:
    print("Saving model")
    trainer.save_model(experiment_name+"/saved_model")


print("\nPREDICTING..")
preds, labels, metrics = trainer.predict(tokenized_dataset['test'], num_beams=5, min_length=50, max_length=512, no_repeat_ngram_size=2, early_stopping=True)

predicted_text,golden_labels=tokenize_for_evaluation(tokenizer,preds,labels)

#here is already past the error 
print("\nRESULT SCORES:")

scores = metrics.items()
print(f'Results: {scores}')
```
The data looks the following, to substitute folde in data/path
```
    {
        "story": "Baymax is a character from the film Big Hero 6 starring Scott Adsit. He was created by Steven T Seagle and the American, Duncan Rouleau.",
        "Types_KG": "[CORE] Baymax is a character from the film Big Hero 6 [TRIPLES] Duncan Rouleau - nationality - Americans | Baymax - creators - Duncan Rouleau | Baymax - creator - Steven T. Seagle | Baymax - series - Big Hero 6 (film) | Big Hero 6 (film) - starring - Scott Adsit | Baymax - creator - Duncan Rouleau | Duncan Rouleau - nationality - Americans | Baymax - creators - Steven T. Seagle | Baymax - series - Big Hero 6 (film) | Big Hero 6 (film) - starring - Scott Adsit | Scott Adsit - type - person | Americans - type - ethnic group | Steven T. Seagle - type - person | Duncan Rouleau - type - person | Big Hero 6 (film) - type - person",
        "Instances_KG": "[CORE] Baymax is a character from the film Big Hero 6 [TRIPLES] Duncan Rouleau - nationality - Americans | Baymax - creators - Duncan Rouleau | Baymax - creator - Steven T. Seagle | Baymax - series - Big Hero 6 (film) | Big Hero 6 (film) - starring - Scott Adsit | Baymax - creator - Duncan Rouleau | Duncan Rouleau - nationality - Americans | Baymax - creators - Steven T. Seagle | Baymax - series - Big Hero 6 (film) | Big Hero 6 (film) - starring - Scott Adsit",
        "
```<|||||>@GabHoo I'm afraid you'll have you will have to share complete data example or another script, the current instructions fail at data loading time if I create a file as specified. (`ArrowInvalid: JSON parse error: Missing a name for object member. in row 0`)<|||||>@GabHoo Hello, I had same problem and I think problem in DataCollatorForSeq2Seq, 
more specifically in label_pad_token_id. 
Collator using label_pad_token_id = -100, but your tokenizer using a different (tokenizer.pad_token_id = 1).
Can you try?
`
collator = DataCollatorForSeq2Seq(tokenizer, model=model, label_pad_token_id=tokenizer.pad_token_id)
`<|||||>Hey @gante, I think [behavior of DataCollatorForSeq2Seq](https://github.com/huggingface/transformers/blob/fe861e578f50dc9c06de33cd361d2f625017e624/src/transformers/data/data_collator.py#L576) is really unexpected. Why it requires label_pad_token_id, if it can use tokenizer.pad_token_id as with [padding_side](https://github.com/huggingface/transformers/blob/fe861e578f50dc9c06de33cd361d2f625017e624/src/transformers/data/data_collator.py#LL574C13-L574C26)?<|||||>Hey @Pavloveuge -- the label padding triggers a different behavior at train time (if my memory does not fail me, the loss is ignored for that token)<|||||>Oh, yeah, you right, but this behavior still results in an error. And it doesn't matter which version of the tokenizer I use(Fast or not). 
In case use_fast=False:
`
TypeError: sequence item 9: expected str instance, NoneType found
`
in case use_fast=True:
`
OverflowError: out of range integral type conversion attempted.
`
<|||||>@Pavloveuge that sounds like a bug indeed :) Would you be able to share a short stand-alone script to reproduce the issue?<|||||>@gante Should I open new issue or reopen this?<|||||>@Pavloveuge A new issue would be preferable 👍 