Error In Running Predictions for run_text_classification.py
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0
- Platform: Windows-10-10.0.18362-SP0
- Python version: 3.8.0
- PyTorch version (GPU?): 1.7.1+cpu (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help
@Rocketknight1



## Information

I ran training successfully with below command

run_text_classification.py \
--model_name_or_path microsoft/xtremedistil-l6-h384-uncased \
--output_dir classificationoutput \
--do_train \
--train_file PreparedData.csv \
--do_eval \
--validation_file PreparedData.csv \
--num_train_epochs 100 \
--test_file PreparedData.csv

train file format

label,data
l1, my sentence 1
l1, my sentence 2
l2, my sentence 3
l2, my sentence 4
. . . 


## To reproduce

Now after training i want to do some predictions so created PredictionData.csv with single column as below

data
my sentence 1
my sentence 2
my sentence 3
. . . 

Then ran the prediction as below using the model and config from the output of training


%run run_text_classification.py \
--model_name_or_path C:\Users\xxxxxxxxx\classificationoutput\tf_model.h5 \
--config_name C:\Users\xxxxxxxxx\classificationoutput\config.json\
--output_dir classificationoutput \
--do_predict \
--test_file PredictionData.csv

## Got Error as below

INFO:__main__:Checkpoint detected, resuming training from checkpoint in classificationoutput. To avoid this behavior, change the `--output_dir` or add `--overwrite_output_dir` to train from scratch.
INFO:__main__:Training/evaluation parameters TrainingArguments(output_dir=classificationoutput, overwrite_output_dir=False, do_train=False, do_eval=None, do_predict=True, evaluation_strategy=IntervalStrategy.NO, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, lr_scheduler_type=SchedulerType.LINEAR, warmup_ratio=0.0, warmup_steps=0, logging_dir=runs\Apr27_15-55-43_GC8SQLQ2E, logging_strategy=IntervalStrategy.STEPS, logging_first_step=False, logging_steps=500, save_strategy=IntervalStrategy.STEPS, save_steps=500, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level=O1, fp16_backend=auto, fp16_full_eval=False, local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name=classificationoutput, disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, sharded_ddp=[], deepspeed=None, label_smoothing_factor=0.0, adafactor=False, group_by_length=False, length_column_name=length, report_to=['tensorboard'], ddp_find_unused_parameters=None, dataloader_pin_memory=True, skip_memory_metrics=False, _n_gpu=0, mp_parameters=)
INFO:__main__:Loading a local file for test: PredictionData.csv
WARNING:datasets.builder:Using custom data configuration default-5a3e83535773f703
Downloading and preparing dataset csv/default (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to C:\Users\xxxxxxxxx\.cache\huggingface\datasets\csv\default-5a3e83535773f703\0.0.0\2dc6629a9ff6b5697d82c25b73731dd440507a69cbce8b425db50b751e8fcfd0...
Dataset csv downloaded and prepared to C:\Users\xxxxxxxxx\.cache\huggingface\datasets\csv\default-5a3e83535773f703\0.0.0\2dc6629a9ff6b5697d82c25b73731dd440507a69cbce8b425db50b751e8fcfd0. Subsequent calls will reuse this data.
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
~\run_text_classification.py in <module>
    535 
    536 if __name__ == "__main__":
--> 537     main()

~\run_text_classification.py in main()
    350             use_auth_token=True if model_args.use_auth_token else None,
    351         )
--> 352     tokenizer = AutoTokenizer.from_pretrained(
    353         model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path,
    354         cache_dir=model_args.cache_dir,

c:\python38\lib\site-packages\transformers\models\auto\tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    388         kwargs["_from_auto"] = True
    389         if not isinstance(config, PretrainedConfig):
--> 390             config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
    391 
    392         use_fast = kwargs.pop("use_fast", True)

c:\python38\lib\site-packages\transformers\models\auto\configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    396         """
    397         kwargs["_from_auto"] = True
--> 398         config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
    399         if "model_type" in config_dict:
    400             config_class = CONFIG_MAPPING[config_dict["model_type"]]

c:\python38\lib\site-packages\transformers\configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
    466             )
    467             # Load config dict
--> 468             config_dict = cls._dict_from_json_file(resolved_config_file)
    469 
    470         except EnvironmentError as err:

c:\python38\lib\site-packages\transformers\configuration_utils.py in _dict_from_json_file(cls, json_file)
    549     def _dict_from_json_file(cls, json_file: Union[str, os.PathLike]):
    550         with open(json_file, "r", encoding="utf-8") as reader:
--> 551             text = reader.read()
    552         return json.loads(text)
    553 

c:\python38\lib\codecs.py in decode(self, input, final)
    320         # decode input (taking the buffer into account)
    321         data = self.buffer + input
--> 322         (result, consumed) = self._buffer_decode(data, self.errors, final)
    323         # keep undecoded input until the next call
    324         self.buffer = data[consumed:]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

Hi, thank you for the report! I'm examining it now and I'm hoping to push some updates to that script today.<|||||>I haven't been able to reproduce this issue, but I have a PR open to make modifications to the script. I'll let you know as soon as the updated version is available - would you be willing to check if the issue is still there once it is?<|||||>The script has been updated! Please let me know if you encounter the same problems.<|||||>@Rocketknight1 Thank you i am able to run predictions and it gives correct prediction for trained data.. 

Here is my Wish List if you can provide:
I am trying to integrate run_text_classification.py to my program where i will provide it a sentence and it gives the prediction as per labelling. and my program uses that label for something useful.

1. It writes to a file the prediction result, will that be possible if I import run_text_classification.py in my program and you exposed a function which takes input list of strings for sentences to classify with all other parameters required to run. And  returns list of strings with predictions in same order. That way i do not have to read a file always for the result.

2. Default fallback label: Meaning if i passed a sentence for classification, if model is not able to classify as per trained labelled data, it  returns the Default fallback label set by user. Tat way i can know for which sentences i have to retrain the model

@Rocketknight1 thanks in advance

Rajesh Dhiman








<|||||>Hm! Number 2 in particular is a fairly advanced ML topic - getting models to know which inputs they can and can't classify accurately is surprisingly hard. This is a fairly fundamental problem that people are still writing papers about, and not one we can really tackle well in an introductory example.

Your suggestion for 1) is certainly possible, though, and we'll think about it! Our intention isn't to support every possible use case with the examples, though! We really just want to show one working example that shows off a lot of the features of the library, and we expect that users will have to modify the code themselves in a lot of cases.<|||||>@Rocketknight1 
Hi 
How I can get the Confidence score in the prediction results.. i need to have that, Is there any option i can set in settings <|||||>@Rocketknight1
i got it.. Sorry it was a dumb Question.. I am a nerd..<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.