Getting Error from Default Data Collator while training Bert on SQUAD 2.0
Hello, I'm a fresher playing with the transformers. I suppose to train the BERT model `bert-base-cased` on SQUAD 2.0, but having an error in `data_collator`: It shows an error of `TypeError: an integer is required`

Here is the detail:
```
Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]
Iteration:   0%|                                       | 0/4135 [00:00<?, ?it/s]
Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 138, in <module>
    trainer.train()
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 119, in train
    self._trainer.train()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/trainer.py", line 456, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/tqdm/std.py", line 1130, in __iter__
    for obj in iterable:
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/data/data_collator.py", line 62, in default_data_collator
    batch[k] = torch.tensor([f[k] for f in features], dtype=torch.long)
TypeError: an integer is required (got type dict)
```



Here is my trainer configuration:
```
training_args = TrainingArguments(
            output_dir=self.output_dir,
            overwrite_output_dir=True,
            num_train_epochs=2,
            per_gpu_train_batch_size=32,
            # per_device_eval_batch_size=64,
            warmup_steps=500,
            weight_decay=0.01,
            # evaluate_during_training=True,
            save_steps=10_000,
            logging_dir='./logs',
        )
        self._trainer = Trainer(
            model=self._model,
            args=training_args,
            compute_metrics=self.compute_metrics,
            train_dataset=self.train_dataset,
            eval_dataset=self.test_dataset
        )
```

Does anyone know why it happens? And how can I fix this error? I don't know if this is one mistake in my code or from the transformers :-( Thanks for all help. 

Best
First things first, please use ``` when copy-pasting stack traces or code (I've edited your post to use that) otherwise it's not really readable.

It's hard to know what's going on without knowing how you built your `train_dataset`. The data collator seems to have problems with it. The items should be dictionaries of list of ints/tensors. It seems there is some nested dictionary here.<|||||>Sorry about that I will remember to use it :-) And regarding the `train_dataset` I used `SquadV2Processor` to get examples for `train` and `eval`, and then convert them using `squad_convert_examples_to_features`: 

```
_processor = SquadV2Processor()
train_examples = _processor.get_train_examples(squad_dir, filename='SQuAD-v2.0-train.json')
train_dataset = squad_convert_examples_to_features(train_examples, self._tokenizer, max_seq_length=384, doc_stride=128, threads=2,max_query_length=64, is_training=True)
```


Is this correct? <|||||>Could you print the result of `self.train_dataset[0]`? It would be helpful to see what the items look like.<|||||>i can't find any clues from the `self.train_dataset[0]`. It is a `SquadFeature` object, like: 

```
2020-08-12 20:48:57,663 -- [__main__:57][INFO]: train_dataset[0] input_ids: [101, 1706, 2292, 1225, 1103, 6567, 2090, 9273, 2845, 1107, 8109, 1107, 10111, 20500, 1699, 136, 102, 22182, 1193, 117, 1103, 1278, 1144, 170, 2336, 1959, 119, 1335, 4184, 1103, 4304, 4334, 112, 188, 2284, 10945, 1110, 170, 5404, 5921, 1104, 1103, 6567, 2090, 119, 13301, 1107, 1524, 1104, 1103, 4304, 4334, 1105, 4749, 1122, 117, 1110, 170, 7335, 5921, 1104, 4028, 1114, 1739, 1146, 14089, 5591, 1114, 1103, 7051, 107, 159, 21462, 1566, 24930, 2508, 152, 1306, 3965, 107, 119, 5893, 1106, 1103, 4304, 4334, 1110, 1103, 19349, 1104, 1103, 11373, 4641, 119, 13301, 1481, 1103, 171, 17506, 9538, 1110, 1103, 144, 10595, 2430, 117, 170, 14789, 1282, 1104, 8070, 1105, 9284, 119, 1135, 1110, 170, 16498, 1104, 1103, 176, 10595, 2430, 1120, 10111, 20500, 117, 1699, 1187, 1103, 6567, 2090, 25153, 1193, 1691, 1106, 2216, 17666, 6397, 3786, 1573, 25422, 13149, 1107, 8109, 119, 1335, 1103, 1322, 1104, 1103, 1514, 2797, 113, 1105, 1107, 170, 2904, 1413, 1115, 8200, 1194, 124, 11739, 1105, 1103, 3487, 17917, 114, 117, 1110, 170, 3014, 117, 2030, 2576, 5921, 1104, 2090, 119, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2020-08-12 20:48:57,663 -- [__main__:58][INFO]: train_dataset[0] attention_mask: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2020-08-12 20:48:57,663 -- [__main__:59][INFO]: train_dataset[0] token_type_ids: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2020-08-12 20:48:57,663 -- [__main__:60][INFO]: train_dataset[0] cls_index: 0
2020-08-12 20:48:57,663 -- [__main__:61][INFO]: train_dataset[0] p_mask: [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
2020-08-12 20:48:57,663 -- [__main__:62][INFO]: train_dataset[0] example_index: 0
2020-08-12 20:48:57,663 -- [__main__:63][INFO]: train_dataset[0] unique_id: 1000000000
2020-08-12 20:48:57,663 -- [__main__:64][INFO]: train_dataset[0] paragraph_len: 163
2020-08-12 20:48:57,663 -- [__main__:65][INFO]: train_dataset[0] token_is_max_context: {17: True, 18: True, 19: True, 20: True, 21: True, 22: True, 23: True, 24: True, 25: True, 26: True, 27: True, 28: True, 29: True, 30: True, 31: True, 32: True, 33: True, 34: True, 35: True, 36: True, 37: True, 38: True, 39: True, 40: True, 41: True, 42: True, 43: True, 44: True, 45: True, 46: True, 47: True, 48: True, 49: True, 50: True, 51: True, 52: True, 53: True, 54: True, 55: True, 56: True, 57: True, 58: True, 59: True, 60: True, 61: True, 62: True, 63: True, 64: True, 65: True, 66: True, 67: True, 68: True, 69: True, 70: True, 71: True, 72: True, 73: True, 74: True, 75: True, 76: True, 77: True, 78: True, 79: True, 80: True, 81: True, 82: True, 83: True, 84: True, 85: True, 86: True, 87: True, 88: True, 89: True, 90: True, 91: True, 92: True, 93: True, 94: True, 95: True, 96: True, 97: True, 98: True, 99: True, 100: True, 101: True, 102: True, 103: True, 104: True, 105: True, 106: True, 107: True, 108: True, 109: True, 110: True, 111: True, 112: True, 113: True, 114: True, 115: True, 116: True, 117: True, 118: True, 119: True, 120: True, 121: True, 122: True, 123: True, 124: True, 125: True, 126: True, 127: True, 128: True, 129: True, 130: True, 131: True, 132: True, 133: True, 134: True, 135: True, 136: True, 137: True, 138: True, 139: True, 140: True, 141: True, 142: True, 143: True, 144: True, 145: True, 146: True, 147: True, 148: True, 149: True, 150: True, 151: True, 152: True, 153: True, 154: True, 155: True, 156: True, 157: True, 158: True, 159: True, 160: True, 161: True, 162: True, 163: True, 164: True, 165: True, 166: True, 167: True, 168: True, 169: True, 170: True, 171: True, 172: True, 173: True, 174: True, 175: True, 176: True, 177: True, 178: True, 179: True}
2020-08-12 20:48:57,663 -- [__main__:66][INFO]: train_dataset[0] tokens: ['[CLS]', 'To', 'whom', 'did', 'the', 'Virgin', 'Mary', 'allegedly', 'appear', 'in', '1858', 'in', 'Lou', '##rdes', 'France', '?', '[SEP]', 'Architectural', '##ly', ',', 'the', 'school', 'has', 'a', 'Catholic', 'character', '.', 'At', '##op', 'the', 'Main', 'Building', "'", 's', 'gold', 'dome', 'is', 'a', 'golden', 'statue', 'of', 'the', 'Virgin', 'Mary', '.', 'Immediately', 'in', 'front', 'of', 'the', 'Main', 'Building', 'and', 'facing', 'it', ',', 'is', 'a', 'copper', 'statue', 'of', 'Christ', 'with', 'arms', 'up', '##rai', '##sed', 'with', 'the', 'legend', '"', 'V', '##eni', '##te', 'Ad', 'Me', 'O', '##m', '##nes', '"', '.', 'Next', 'to', 'the', 'Main', 'Building', 'is', 'the', 'Basilica', 'of', 'the', 'Sacred', 'Heart', '.', 'Immediately', 'behind', 'the', 'b', '##asi', '##lica', 'is', 'the', 'G', '##rot', '##to', ',', 'a', 'Marian', 'place', 'of', 'prayer', 'and', 'reflection', '.', 'It', 'is', 'a', 'replica', 'of', 'the', 'g', '##rot', '##to', 'at', 'Lou', '##rdes', ',', 'France', 'where', 'the', 'Virgin', 'Mary', 'reputed', '##ly', 'appeared', 'to', 'Saint', 'Bern', '##ade', '##tte', 'So', '##ubi', '##rous', 'in', '1858', '.', 'At', 'the', 'end', 'of', 'the', 'main', 'drive', '(', 'and', 'in', 'a', 'direct', 'line', 'that', 'connects', 'through', '3', 'statues', 'and', 'the', 'Gold', 'Dome', ')', ',', 'is', 'a', 'simple', ',', 'modern', 'stone', 'statue', 'of', 'Mary', '.', '[SEP]']
2020-08-12 20:48:57,663 -- [__main__:67][INFO]: train_dataset[0] token_to_orig_map: {17: 0, 18: 0, 19: 0, 20: 1, 21: 2, 22: 3, 23: 4, 24: 5, 25: 6, 26: 6, 27: 7, 28: 7, 29: 8, 30: 9, 31: 10, 32: 10, 33: 10, 34: 11, 35: 12, 36: 13, 37: 14, 38: 15, 39: 16, 40: 17, 41: 18, 42: 19, 43: 20, 44: 20, 45: 21, 46: 22, 47: 23, 48: 24, 49: 25, 50: 26, 51: 27, 52: 28, 53: 29, 54: 30, 55: 30, 56: 31, 57: 32, 58: 33, 59: 34, 60: 35, 61: 36, 62: 37, 63: 38, 64: 39, 65: 39, 66: 39, 67: 40, 68: 41, 69: 42, 70: 43, 71: 43, 72: 43, 73: 43, 74: 44, 75: 45, 76: 46, 77: 46, 78: 46, 79: 46, 80: 46, 81: 47, 82: 48, 83: 49, 84: 50, 85: 51, 86: 52, 87: 53, 88: 54, 89: 55, 90: 56, 91: 57, 92: 58, 93: 58, 94: 59, 95: 60, 96: 61, 97: 62, 98: 62, 99: 62, 100: 63, 101: 64, 102: 65, 103: 65, 104: 65, 105: 65, 106: 66, 107: 67, 108: 68, 109: 69, 110: 70, 111: 71, 112: 72, 113: 72, 114: 73, 115: 74, 116: 75, 117: 76, 118: 77, 119: 78, 120: 79, 121: 79, 122: 79, 123: 80, 124: 81, 125: 81, 126: 81, 127: 82, 128: 83, 129: 84, 130: 85, 131: 86, 132: 87, 133: 87, 134: 88, 135: 89, 136: 90, 137: 91, 138: 91, 139: 91, 140: 92, 141: 92, 142: 92, 143: 93, 144: 94, 145: 94, 146: 95, 147: 96, 148: 97, 149: 98, 150: 99, 151: 100, 152: 101, 153: 102, 154: 102, 155: 103, 156: 104, 157: 105, 158: 106, 159: 107, 160: 108, 161: 109, 162: 110, 163: 111, 164: 112, 165: 113, 166: 114, 167: 115, 168: 115, 169: 115, 170: 116, 171: 117, 172: 118, 173: 118, 174: 119, 175: 120, 176: 121, 177: 122, 178: 123, 179: 123}
```
<|||||>Ok. You need to remove some keys from it as it has way too many attributes (the failure comes from the fact the data collator is trying to build a tensor from the `token_is_max_context` fields).

The easiest way is probably to use the `SquadDataset` in `data.datasets.squad`, or you can just copy its `__getitem__` method and put it on your own dataset class.<|||||>Thanks Sylvain. But I found that my transformers library doesn't contain `data.datasets.squad` at all. I'm using `transformers==3.0.2`. It only contains `glue` and `language_model` two classes. <|||||>Would you mind to explain how the error comes from the data collator? The `token_is_max_context` only contains a list of `True` flags. Is there a certain order (keys) of these features?<|||||>The data collator should receive a dictionary string to list of ints/tensors. The value associated to `token_is_max_context` is neither a list of int or a tensor, hence the error.
Note that you dataset should have items that are dictionaries with keys that are argument names your model will accept, which another reason why `token_is_max_context` needs to be removed.<|||||>Okay, it seems to make sense. There would be another question: does `transformers` process the original version of SQUAD dataset? I download the data from the SQUAD website, which shouldn't have any errors if it is the exact same as the one `transformers` used. Can I simply remove `token_is_max_context` from the `SquadFeatures` to solve this error? 

Also which version of `transformers` contains data.datasets.squad? I can't find it in 3.0.0, 2.9.0 and even 2.5.0. <|||||>You need to only extract : input_ids, attention_mask and token_type_ids as the rest is probably not helpful to your model. AFAICT the file [data.datasets.squad](https://github.com/huggingface/transformers/blob/master/src/transformers/data/datasets/squad.py) has been there for a while, so you should have it in those versions.<|||||>Thanks for help, Sylvain. Let me give a try today. Hopefully, it works fine.  <|||||>Hi Sylvain,

Sorry for the interruption again. I've created a new class that only contains `input_ids`, `attention_mask` and `token_type_ids`, then the system gives an error: 

```
Traceback (most recent call last):
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 190, in <module>
    trainer.train()
  File "/scratch/yyu/codebase/odqa/odqa/reader/reader_trainer.py", line 168, in train
    self._trainer.train()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/trainer.py", line 375, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/tqdm/std.py", line 1130, in __iter__
    for obj in iterable:
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/data/data_collator.py", line 91, in collate_batch
    batch = self._tensorize_batch(examples)
  File "/scratch/yyu/codebase/odqa/ENV/lib/python3.6/site-packages/transformers/data/data_collator.py", line 99, in _tensorize_batch
    length_of_first = examples[0].size(0)
AttributeError: 'SimpleSquadFeature' object has no attribute 'size'
```

But I didn't find the `size()` function in the original class (SquadFeature) either. Do you know why it happens like that? 

And also I've installed transformers from 2.1.0 to 3.0.2 but can't import a class called `SquadDataset`, there is not a path [data.datasets.squad](https://github.com/huggingface/transformers/blob/master/src/transformers/data/datasets/squad.py), but `data.processor.squad`. When did transformers delete the `SquadDataset`? 

Thanks for help. 
Yanchao<|||||>Don't create a special class, just return a dictionary with those fields and it should work.<|||||>Hi,

I've had nearly the same problem as yy147 (the TypeError related to the default_data_collator), even though my train_dataset had items that were precisely a dictionary mapping the keys: 'input_ids', 'attention_mask', 'token_type_ids', to lists of ints, and 'label' to a torch.LongTensor.

Strangely I managed to solve the problem by copying the transformers.data.data_collator.default_data_collator into my own code and letting the Trainer use that.

Python version 3.6.8
torch version 1.6.0
transformers version 3.0.2

Hope it helps,
Gijs
<|||||>> Don't create a special class, just return a dictionary with those fields and it should work.

Thanks Sylvain. I've found the `SquadDateset`, which is surprisingly not included in any transformers versions if you run `pip install transfermers==3.0.2`. I can only find it if I install `transformers` from the source. It seems something needs to be fixed.

Best,
Yanchao <|||||>Thanks for help, Gijs. I will try to copy and paste it later. It is a wired situation. <|||||>Hi @yy147 , I am also getting a similar error:
```    
length_of_first = examples[0].size(0)
AttributeError: 'dict' object has no attribute 'size
```

Have you managed to fix your error?<|||||>Hi @gungor2, I found another source code extended from transformers example `https://github.com/kamalkraj/BERT-SQuAD/blob/master/utils.py`. It gives a great example to solve the problem of `squad_examples_to_features`. It works well for me. Hope it is helpful for you. <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hi, I have a related issue I am trying to train the [**TrOCR**](https://huggingface.co/microsoft/trocr-large-handwritten) model on my own data what I tried: 
```
# the class for loading data has function :
def __getitem__(self, idx):
        file_name = self.df['file_name'][idx]
        text = self.df['text'][idx]
        # prepare image (i.e. resize + normalize)
        image = Image.open(self.root_dir + file_name).convert("RGB")
        pixel_values = self.processor(image, return_tensors="pt").pixel_values
        labels = self.processor.tokenizer(text, padding="max_length",
                                                 max_length=self.max_target_length).input_ids    
        labels = [label if label != self.processor.tokenizer.pad_token_id else -100 for label in
                  labels]

        encoding = {"pixel_values": pixel_values.squeeze(), "labels": torch.tensor(labels)}
        return encoding
training_args = Seq2SeqTrainingArguments(
        num_train_epochs=25, 
        learning_rate=5e-5,
        predict_with_generate=True,
        evaluation_strategy="steps",
        per_device_train_batch_size=64,
        per_device_eval_batch_size=64,
        fp16=True,    
        output_dir="/1/large/",    
        logging_steps=100,
        save_steps=2000,
        eval_steps=5000,
        )

    trainer = Seq2SeqTrainer( model=model, tokenizer=processor.feature_extractor,
        args=training_args, compute_metrics=compute_metrics, train_dataset=train_dataset,
        eval_dataset=eval_dataset,  data_collator=default_data_collator, )
```
the feed input image to the model  has a `height of 64 `fixed for all and different `width`  
The issue I see is: where the training stops after a few hours
```
Traceback (most recent call last):
  File "train.py", line 191, in <module>
    main()
  File "train.py", line 173, in main
    trainer.train()
  File "/home/user/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1521, in train
    return inner_training_loop(
  File "/home/user/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1737, in _inner_training_loop
    for step, inputs in enumerate(epoch_iterator):
  File "/home/user/venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/home/user/venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/user/venv/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "/home/user/venv/lib/python3.8/site-packages/transformers/trainer_utils.py", line 696, in __call__
    return self.data_collator(features)
  File "/home/user/venv/lib/python3.8/site-packages/transformers/data/data_collator.py", line 67, in default_data_collator
    return torch_default_data_collator(features)
  File "/home/user/venv/lib/python3.8/site-packages/transformers/data/data_collator.py", line 129, in torch_default_data_collator
    batch[k] = torch.stack([f[k] for f in features])
RuntimeError: stack expects each tensor to be equal size, but got [128] at entry 0 and [139] at entry 19
  1%|▊                                                                                                         | 1356/166025 [40:59<82:58:15,  1.81s/it]
```
Transformer version: 4.22.2
@sgugger  @NielsRogge  @

<|||||>Hi,

It looks like your target texts aren't having the same length. You need to not only pad but also set `truncation=True` to make sure all texts have 128 tokens.