Finally, I found that maybe it is caused by set_num_interop_threads.
For earlier libtorch (e.g. libtorch 1.0.0, libtorch 1.1.0), there are no functions like at::set_num_interop_threads or torch::set_num_interop_threads, and only one thread works when calling libtorch functions in OpenMP.
From libtorch 1.2.0, we can find at::set_num_interop_threads in <ATen/Parallel.h> or  torch::set_num_interop_threads in <torch/utils.h> in later versions, and the final results are correct.