Really appreciate your reply and the code!  I changed float to double(because float failed like you stated) and ran the code on GPU.
I got the right answer.However, the max_mem and time consumed between bmm and element-wise are of great difference:

image.png1792×356 70.6 KB

similar issue occured when running on CPU.
And now I am really confused. I assumed there is some optimization specifically for bmm, maybe parallelism?