It reports the same error, even if I try the code snippet you provided. Thanks anyway.
I am suspecting the reason is I am using  for loop  in the code. My input format is [batch, time, height, width, channel]. My code do for loop on time axis.
FYI, code runs fine on CPU.
Best