Thanks! Actually, I found that tf.sess.run is always synchronized. So tensorflow doesn’t need CUDA synchronization.
I was running multiple processes using CUDA MPS, so it was essential to ensure every process is synchronized.
Thank you for your reply