Training loss is not logged correctly when doing evaluation with Trainer
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.4.2
- Platform: Linux-5.4.0-62-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@sgugger 

## Information

Model I am using (Bert, XLNet ...): Reformer

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I'm doing a text generation with Reformer using my own dataset.

## To reproduce

Steps to reproduce the behavior:

1. Set `logging_steps=10` and `evaluation_strategy="steps`, `eval_steps=20` in `TrainingArguments`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

I'm doing training in Jupyter notebook. With the settings above, the logging output shows training loss and validation loss every 20 steps (which is the `eval_steps`). However, I want the training loss to be logged at higher frequency than the validation loss, for example at 10 steps like above. This is because running evaluation will take some time for large validation set, while I still want to monitor mini-batch training loss.

When inspecting logs with Tensor Board, no training loss is logged at all (even the values every 20 steps).

If I disable evaluation (`evaluation_strategy="no"`), the training loss is logged every 10 steps as expected.

## Expected behavior

When enabling evaluation in Trainer, training loss should be logged every `logging_steps`, while validation loss is logged every `eval_steps`

I am not able to reproduce, on my side I do see logs every logging_steps for the training loss and learning rate, and every eval_steps for the validation loss and metrics, both in the console and TensorBoard. Could you try again with a source install?<|||||>After I restarted TensorBoard, training loss showed up correctly again. Maybe something went wrong with TensorBoard. Thank you for your prompt response! FYI it worked correctly with my current version `transformers=4.4.2`. It was a silly mistake from my side.