Thanks very much!
but this code with another issue:
I cannot run the code with  --fp16 --fp16_opt_level=O1, it meets OOM, but I can run it without it. It seems so interesting and confusing.
BTW, a similar issue is reported at https://github.com/huggingface/transformers/issues/8403 4
Really look forward to your reply
thanks again