I have figured out my problem, it has nothing to do with the way of using convert_sync_bn. The solution is that if I use apex, I should use convert_sync_bn before initializing the amp. There will be problem if I use convert_sync_bn after it.