TypeError: forward() got an unexpected keyword argument 'labels'
## Environment info

- `transformers` version: 3.0.2
- Platform: Linux-5.3.0-53-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: Distributed

Hey there,

I've run into this issue and not sure how to fix it:

    TypeError: forward() got an unexpected keyword argument 'labels'

I'm running transformers v3.0.2 installed via pip

Please see my code below. There is nothing fancy going on, I'm just trying to train RobertaMLM for a few more epochs on a different dataset. 

```python
import os
import argparse
import datetime
from torch.utils.tensorboard import SummaryWriter
from transformers import RobertaModel, RobertaConfig, RobertaTokenizerFast, LineByLineTextDataset, DataCollatorForLanguageModeling, Trainer, TrainingArguments
from configs import model_directory, tensorboard_directory
from logger import get_logger

log = get_logger(__name__)


args = argparse.Namespace(
	seed=42,
	model_id="Roberta2",
	pretrained_model_name_or_path="roberta-base",
	vocab_file="/data/nlp/roberta_vocabulary/roberta-base-vocab.json",
	merges_file="/data/nlp/roberta_vocabulary/roberta-base-merges.txt",
	filename="/data/nlp/trc2.txt",
	block_size=2**7,
	epochs=25,
)

output_directory = os.path.join(model_directory, args.model_id)
os.makedirs(output_directory, exist_ok=True)
os.environ["TOKENIZERS_PARALLELISM"] = "false"


def build_model():
	tokenizer = RobertaTokenizerFast.from_pretrained(pretrained_model_name_or_path=args.pretrained_model_name_or_path, lowercase=True, add_prefix_space=True, max_len=512)

	config = RobertaConfig.from_pretrained(args.pretrained_model_name_or_path)
	config.output_hidden_states = False

	model = RobertaModel.from_pretrained(pretrained_model_name_or_path=args.pretrained_model_name_or_path, config=config, cache_dir=output_directory)

	dataset = LineByLineTextDataset(
		tokenizer=tokenizer,
		file_path=args.filename,
		block_size=args.block_size,
	)

	data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

	training_args = TrainingArguments(
		seed=args.seed,
		output_dir=output_directory,
		overwrite_output_dir=True,
		num_train_epochs=args.epochs,
		per_device_train_batch_size=128,
		save_steps=10_000,
		# save_total_limit=2,
		fp16=True,
		fp16_opt_level="O1"
	)

	trainer = Trainer(
		model=model,
		args=training_args,
		data_collator=data_collator,
		train_dataset=dataset,
		prediction_loss_only=True,
	)

	trainer.train()
	trainer.save_model(output_directory)

```


tag: @sgugger 
Please copy-paste the entire stack trace, just the error message is not enough to know what's going on :-)<|||||>Hi @sgugger thanks for the reply! 

Please see below for the full stack-trace

```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-2942c2ba4004> in <module>
     42 )
     43
---> 44 trainer.train()
     45 trainer.save_model(output_directory)

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in train(self, model_path)
    497                     continue
    498
--> 499                 tr_loss += self._training_step(model, inputs, optimizer)
    500
    501                 if (step + 1) % self.args.gradient_accumulation_steps == 0 or (

/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _training_step(self, model, inputs, optimizer)
    620             inputs["mems"] = self._past
    621
--> 622         outputs = model(**inputs)
    623         loss = outputs[0]  # model outputs are always tuple in transformers (see doc)
    624

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py in forward(self, *inputs, **kwargs)
    153             return self.module(*inputs[0], **kwargs[0])
    154         replicas = self.replicate(self.module, self.device_ids[:len(inputs)])
--> 155         outputs = self.parallel_apply(replicas, inputs, kwargs)
    156         return self.gather(outputs, self.output_device)
    157

/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py in parallel_apply(self, replicas, inputs, kwargs)
    163
    164     def parallel_apply(self, replicas, inputs, kwargs):
--> 165         return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
    166
    167     def gather(self, outputs, output_device):

/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py in parallel_apply(modules, inputs, kwargs_tup, devices)
     83         output = results[i]
     84         if isinstance(output, ExceptionWrapper):
---> 85             output.reraise()
     86         outputs.append(output)
     87     return outputs

/opt/conda/lib/python3.7/site-packages/torch/_utils.py in reraise(self)
    393             # (https://bugs.python.org/issue2651), so we work around it.
    394             msg = KeyErrorMessage(msg)
--> 395         raise self.exc_type(msg)

TypeError: Caught TypeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker
    output = module(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'labels'

```<|||||>Oh, I misread. `RobertaModel` is not something you can use directly with `Trainer` as it doesn't have any objective (it's the base model without head). You should pick a model with head relevant to your task.<|||||>haha, i feel stupid now :) Thanks!<|||||>


> Oh, I misread. `RobertaModel` is not something you can use directly with `Trainer` as it doesn't have any objective (it's the base model without head). You should pick a model with head relevant to your task.

@sgugger can we finetune models with specific task only (like RobertaForMasekdLm etc) ? is there a way we can pre-train RobertaModel on our data then go for specific tasks?<|||||>I have similar a question as @sanjay23singh. I want to train a no-head RobertaModel on my corpus, then fine tuned using RobertaForSentenceClassification? (as below)

```
model = RobertaModel(config=config)
training_args = ..
trainer =...
trainer.train()
trainer.save_model('myRoberta')

#fine tune
sentimodel = RobertaForSequenceClassification.from_pretrained("./myRoberta")
```

My ultimate goal is training my own corpus with masked only, then do a classification. <|||||>> Oh, I misread. `RobertaModel` is not something you can use directly with `Trainer` as it doesn't have any objective (it's the base model without head). You should pick a model with head relevant to your task.

Sorry, could you explain this in more details here. Thank you.<|||||>I'm absolutely confused by this too, it looks to me like there's a big missing piece in the doc.<|||||>its easy，
training_args = TrainingArguments(
    output_dir="./My_train_BERT",
    overwrite_output_dir=True,
    num_train_epochs=5,
    per_gpu_train_batch_size=64,
    save_steps=10_000,
    save_total_limit=2,
    prediction_loss_only=True,
    **label_smoothing_factor=0.1**   ##add it is ok
)
