It is totally my bad, thank you @mrshenli. I learned how to create and use the tensor at module’s forward in DataParallel from your advices.
os.environ['CUDA_LAUNCH_BLOCKING'] = '1'   is the fault…
Someone who use this cuda option for checking the logs, because it stops DataParallel process, it is recommend to comment it…