RuntimeError: expected scalar type Half but found Float
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....):
bert-large-uncased

Language I am using the model on (English, Chinese....):
English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: (give details)

I am using a modified version of the run_lm_finetuning.py and amp at optimization level o1.  Level O2 runs without issue.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)

I am finetuning on a dataset of sentences, with a min length of 10 tokens, using padding and the mask_tokens function given by the repo. Input and labels are padded per specs and of type=LongTensor (torch.Size([4, 200]) torch.Size([4, 200])) with batch size of 4.

## To Reproduce

Steps to reproduce the behavior:

1. When I run without amp, training works as intended.  If I train at amp level O2, training runs as intended.
2. Running with amp level O1 leads to an error: RuntimeError: expected scalar type Half but found Float
3. I have verified that there is no model.eval() and that scaled_loss and clip_grad_norm_ calls are the same as prescribed in the example.
3. Examples of model initialization, train loop, and error are below.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior
Training to complete without error.

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS:  Google Colab
* Python version:  3.6
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): 1.2.0
* Using GPU ? Y
* Distributed of parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

**Model/Optimizer code:**
```
    model = model_class.from_pretrained(args.model_name_or_path, from_tf=bool('.ckpt' in args.model_name_or_path), config=config)
    model.to(args.device)

    # Prepare optimizer and schedule (linear warmup and decay)
    #no_decay = ['bias', 'LayerNorm.weight']
    no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
    optimizer_grouped_parameters = [
        {'params': [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)], 'weight_decay': args.weight_decay},
        {'params': [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
        ]
    optimizer = AdamW(optimizer_grouped_parameters, lr=args.learning_rate, weight_decay=args.weight_decay)
    
    scheduler = WarmupLinearSchedule(optimizer, warmup_steps=args.warmup_steps, t_total=t_total)
    
    if args.fp16:
        model, optimizer = amp.initialize(model, optimizer, opt_level=args.fp16_opt_level)
```

**Import Part of Train Loop:**
```
        for step, batch in enumerate(epoch_iterator):
            inputs, labels = mask_tokens(batch, tokenizer, args) if args.mlm else mask_labels(batch, tokenizer, args)

            inputs = inputs.to(args.device)
            labels = labels.to(args.device)
            
            model.train()

            loss, _ = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)

            if args.n_gpu > 1:
                loss = loss.mean()  # mean() to average on multi-gpu parallel training
            if args.gradient_accumulation_steps > 1:
                loss = loss / args.gradient_accumulation_steps

            if args.fp16:
                with amp.scale_loss(loss, optimizer) as scaled_loss:
                    scaled_loss.backward()
            else:
                loss.backward()

            tr_loss += loss.item()
            
            if (step + 1) % args.gradient_accumulation_steps == 0:
                print('Clipping Grad Norm. fp16:', args.fp16)
                if args.fp16:
                  torch.nn.utils.clip_grad_norm_(amp.master_params(optimizer), args.max_grad_norm)
                else:
                  torch.nn.utils.clip_grad_norm_(model.parameters(), args.max_grad_norm)
                scheduler.step()  # Update learning rate schedule
                optimizer.step()
                optimizer.zero_grad()
                global_step += 1
```



**Error:**
```
RuntimeError                              Traceback (most recent call last)

<ipython-input-7-627529edcd83> in <module>()
     10 ft_data.fp16_opt_level = 'O1'
     11 
---> 12 run_finetune(ft_data)
     13 
     14 print('Finetuned BERT model loaded.')

12 frames

<ipython-input-6-39df7e7a4180> in run_finetune(args)
    337         torch.distributed.barrier()
    338 
--> 339     global_step, tr_loss = train(args, train_dataset, model, tokenizer)
    340     logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)
    341 

<ipython-input-6-39df7e7a4180> in train(args, train_dataset, model, tokenizer)
    193             model.train()
    194 
--> 195             loss, _ = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)
    196 
    197             if args.n_gpu > 1:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/pytorch_transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, masked_lm_labels)
    767 
    768         sequence_output = outputs[0]
--> 769         prediction_scores = self.cls(sequence_output)
    770 
    771         outputs = (prediction_scores,) + outputs[2:]  # Add hidden states and attention if they are here

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/pytorch_transformers/modeling_bert.py in forward(self, sequence_output)
    417 
    418     def forward(self, sequence_output):
--> 419         prediction_scores = self.predictions(sequence_output)
    420         return prediction_scores
    421 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/pytorch_transformers/modeling_bert.py in forward(self, hidden_states)
    406 
    407     def forward(self, hidden_states):
--> 408         hidden_states = self.transform(hidden_states)
    409         hidden_states = self.decoder(hidden_states) + self.bias
    410         return hidden_states

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/pytorch_transformers/modeling_bert.py in forward(self, hidden_states)
    388         hidden_states = self.dense(hidden_states)
    389         hidden_states = self.transform_act_fn(hidden_states)
--> 390         hidden_states = self.LayerNorm(hidden_states)
    391         return hidden_states
    392 

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    545             result = self._slow_forward(*input, **kwargs)
    546         else:
--> 547             result = self.forward(*input, **kwargs)
    548         for hook in self._forward_hooks.values():
    549             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/dist-packages/apex/normalization/fused_layer_norm.py in forward(self, input)
    157         if self.elementwise_affine:
    158           return FusedLayerNormAffineFunction.apply(
--> 159               input, self.weight, self.bias, self.normalized_shape,self.eps)
    160         else:
    161           return FusedLayerNormFunction.apply(input, self.normalized_shape, self.eps)

/usr/local/lib/python3.6/dist-packages/apex/normalization/fused_layer_norm.py in forward(ctx, input, weight, bias, normalized_shape, eps)
     23     bias_ = bias.contiguous()
     24     output, mean, invvar = fused_layer_norm_cuda.forward_affine(
---> 25         input_, ctx.normalized_shape, weight_, bias_, ctx.eps)
     26     ctx.save_for_backward(input_, weight_, bias_, mean, invvar)
     27     return output

RuntimeError: expected scalar type Half but found Float (data<c10::Half> at /usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorMethods.h:1821)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f0840be9273 in /usr/local/lib/python3.6/dist-packages/torch/lib/libc10.so)
frame #1: c10::Half* at::Tensor::data<c10::Half>() const + 0x3ee (0x7f08298ccf8e in /usr/local/lib/python3.6/dist-packages/fused_layer_norm_cuda.cpython-36m-x86_64-linux-gnu.so)
frame #2: cuda_layer_norm(at::Tensor*, at::Tensor*, at::Tensor*, at::Tensor*, int, int, c10::ArrayRef<long>, at::Tensor*, at::Tensor*, double) + 0x4c5 (0x7f08298ca745 in /usr/local/lib/python3.6/dist-packages/fused_layer_norm_cuda.cpython-36m-x86_64-linux-gnu.so)
```
There are a total of 63 frames which output in the error that have been truncated here.

I've encountered this problem as well<|||||>Seems like an apex error (apex should be converting the tensors to half).
Maybe try to update or reinstall apex following carefully the required step for installation? <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
