Oach, sorry. I just check the gpu state. It’s great. I just stupidly thought the Device should show cuda:3/4/5/6 (it shouldn’t of course since only 4 gpus are visible).
Thank you so much for your quick reply. Your help really save me since it’s my first time to use accelerate package.