Fail to run text classification example with run_tf_text_classification
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: Ubuntu 18.04
- Python version: 3.6.9
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 2.3.1
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Distributed

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

Text classification with own dataset

## To reproduce

Steps to reproduce the behavior:

1. 
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

I use my own datasets {train/dev/test}/.csv and run `run_tf_text_classification.py`, the training seems OK, while error occurs while evaluation as below:

> 2020-10-10 07:33:15.368292: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at resource_variable_ops.cc:537 : Not found: Resource localhost/_AnonymousVar110/N10tensorflow3VarE does not exist.
Traceback (most recent call last):
  File "run_tf_text_classification.py", line 292, in <module>
    main()
  File "run_tf_text_classification.py", line 267, in main
    trainer.train()
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer_tf.py", line 592, in train
    self.evaluate()
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer_tf.py", line 438, in evaluate
    output = self.prediction_loop(eval_ds, steps, num_examples, description="Evaluation")
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer_tf.py", line 327, in prediction_loop
    logits = self.distributed_prediction_steps(batch)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 814, in _call
    results = self._stateful_fn(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.NotFoundError:  Resource localhost/_AnonymousVar110/N10tensorflow3VarE does not exist.
	 [[node AssignAddVariableOp (defined at /usr/local/lib/python3.6/dist-packages/transformers/trainer_tf.py:457) ]] [Op:__inference_distributed_prediction_steps_11885]

Function call stack:
distributed_prediction_steps

Might be of interest to @jplu <|||||>Hello!

Can you give more detail on how to reproduce your issue, otherwise we cannot help you.<|||||>> Hello!
> 
> Can you give more detail on how to reproduce your issue, otherwise we cannot help you.

Thanks for your replay. I followed the instruction [here](https://github.com/huggingface/transformers/blob/master/examples/text-classification/README.md) with my own datasets (I can not provide my dataset due to confidentiality.). My script is below:

> python3 run_tf_text_classification.py \
--train_file $data_dir/train.csv \
--dev_file $data_dir/dev.csv \
--test_file $data_dir/test.csv \
--label_column_id 0 \
--model_name_or_path distilbert-base-uncased \
--cache_dir $cache_dir \
--output_dir $output_dir \
--num_train_epochs 4 \
--per_device_train_batch_size 12 \
--per_device_eval_batch_size 12 \
--do_train \
--do_eval \
--do_predict \
--logging_steps 10 \
--evaluate_during_training \
--save_steps 10 \
--overwrite_output_dir \
--max_seq_length 128

I can see the training started and run for a while, as there were checkpoints saved in the `output_dir`.<|||||>Sorry, I tried with one of my dataset with the exact same command line on 4/2/1 GPUs and CPU and cannot reproduce your error. The only one thing I can tell you to do is to be sure to use the master version of the script. Otherwise without more information I cannot really help you more sorry :(<|||||>> Sorry, I tried with one of my dataset with the exact same command line on 4/2/1 GPUs and CPU and cannot reproduce your error. The only one thing I can tell you to do is to be sure to use the master version of the script. Otherwise without more information I cannot really help you more sorry :(

Thanks for your help. It could be related to tensorflow or transformers versions. I will try a few of them and see how it would solve my problem.<|||||>@lkluo I have the exact same problem. Did you solve this?<|||||>> @lkluo I have the exact same problem. Did you solve this?

I have tried many ways without any luck, so I gave up.
You may open a new issue and seek help from @jplu.<|||||>@lkluo Thanks for your response. I think I got it fixed. I was saving the model after each epoch with `tf.saved_model.save(self.model, self.args.output_dir)`. However, when using the model for evaluation after saving it once with this method, I got the error you described. I changed it to using `self.model.ckpt_manager.save()` which is a bit inconvenient since I want .pb files, but at least the code runs fine now. If your error is also related to storing the model, this might help you.
<|||||>> @lkluo Thanks for your response. I think I got it fixed. I was saving the model after each epoch with `tf.saved_model.save(self.model, self.args.output_dir)`. However, when using the model for evaluation after saving it once with this method, I got the error you described. I changed it to using `self.model.ckpt_manager.save()` which is a bit inconvenient since I want .pb files, but at least the code runs fine now. If your error is also related to storing the model, this might help you.

Good to know, thanks for letting me know. I will definitely give a try with your method.<|||||>## Environment info
Platform: Jupyter notebook on Ubuntu 2004
TF version: 2.3.1
Transformers version: 3.5.0
Python version: 3.6.9
Single GPU: RTX2080TI

## Issue
I am encountering the same error during evaluation using TFTrainer.train(). It is not reproducible, and it seems to happen randomly. I installed the latest docker image from the Tensorflow website (docker pull tensorflow/tensorflow:latest-gpu-jupyter). It seems everyone is downgrading Tensorflow to avoid this issue? What is the lowest possible version of Tensorflow for using Transformers 3.5.0?<|||||>I met the same problem. It caused by TFTrainer.train() when excute to line 573 'self.distributed_training_steps(batch)' in trainer_tf.py. And it throws 
```
2020-11-21 19:34:31.165454: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:172] Filling up shuffle buffer (this may take a while): 9713 of 16392
2020-11-21 19:34:38.101580: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:221] Shuffle buffer filled.
```
I tried with colab gpu, it is not work. And I searched the same issue "Shuffle buffer filled." in tensorflow, it is still not solved.<|||||>I reopen this issue because many others encountered the same problem.<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.