Thanks.
1] What do you mean “.binary”?
2] Before applying your comment,
torch.__config__.parallel_info()
ATen/Parallel:
    at::get_num_threads() : 1
    at::get_num_interop_threads() : 2
OpenMP not found
Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
    mkl_get_max_threads() : 4
Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
std::thread::hardware_concurrency() : 4
Environment variables:
    OMP_NUM_THREADS : [not set]
    MKL_NUM_THREADS : [not set]
ATen parallel backend: OpenMP

Size (MB): 438.017609
Size (MB): 181.499089
Evaluating: 100%|███████████████████████████████████████████████████████████████████████| 51/51 [01:52<00:00,  2.21s/it]
{'acc': 0.8602941176470589, 'f1': 0.9018932874354562, 'acc_and_f1': 0.8810937025412575}
Evaluate total time (seconds): 112.9
Evaluating: 100%|███████████████████████████████████████████████████████████████████████| 51/51 [01:51<00:00,  2.18s/it]
{'acc': 0.8578431372549019, 'f1': 0.8993055555555555, 'acc_and_f1': 0.8785743464052287}
Evaluate total time (seconds): 111.2

3] After,
ATen/Parallel:
    at::get_num_threads() : 1
    at::get_num_interop_threads() : 2
OpenMP not found
Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
    mkl_get_max_threads() : 1
Intel(R) MKL-DNN v1.5.0 (Git Hash e2ac1fac44c5078ca927cb9b90e1b3066a0b2ed0)
std::thread::hardware_concurrency() : 4
Environment variables:
    OMP_NUM_THREADS : 1
    MKL_NUM_THREADS : 1
ATen parallel backend: OpenMP

Size (MB): 438.017609
Size (MB): 181.499089
Evaluating: 100%|███████████████████████████████████████████████████████████████████████| 51/51 [01:49<00:00,  2.15s/it]
{'acc': 0.8602941176470589, 'f1': 0.9018932874354562, 'acc_and_f1': 0.8810937025412575}
Evaluate total time (seconds): 110.0
Evaluating: 100%|███████████████████████████████████████████████████████████████████████| 51/51 [01:47<00:00,  2.12s/it]
{'acc': 0.8578431372549019, 'f1': 0.8993055555555555, 'acc_and_f1': 0.8785743464052287}
Evaluate total time (seconds): 108.0


I couldn’t get a meaningful speed-up.
p.s) Interface of function  convert_examples_to_features  from transformers was changed.
So I modified run.py.