I’ve tried as follows,
export MKL_NUM_THREADS = 1
export OMP_NUM_THREADS = 1
python Bert_quantze_tutorial.py

touch.config.parallel_info()
1.8.0.dev20201014+cu101
ATen/Parallel:
        at::get_num_threads() : 1
        at::get_num_interop_threads() : 16
OpenMP 201511 (a.k.a. OpenMP 4.5)
        omp_get_max_threads() : 1
Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
        mkl_get_max_threads() : 1
Intel(R) MKL-DNN v1.6.0 (Git Hash 5ef631a030a6f73131c77892041042805a06064f)
std::thread::hardware_concurrency() : 32
Environment variables:
        OMP_NUM_THREADS : 1
        MKL_NUM_THREADS : 1

But can’t get any speed up with this setting;
Result
Size (MB): 438.021641
Size (MB): 181.502781
Evaluating: 100%|████████████████████████████████████████████████████████████████████████████████| 51/51 [02:00<00:00,  2.37s/it]
{'acc': 0.8602941176470589, 'f1': 0.9018932874354562, 'acc_and_f1': 0.8810937025412575}
Evaluate total time (seconds): 120.8
Evaluating: 100%|████████████████████████████████████████████████████████████████████████████████| 51/51 [02:01<00:00,  2.39s/it]
{'acc': 0.8578431372549019, 'f1': 0.8999999999999999, 'acc_and_f1': 0.878921568627451}
Evaluate total time (seconds): 122.0