Solved by mmy in post #9 
              
              
                Thank you for your patient answers so many times , I finally find the reason which is my CUDA code in backward pass has some bug.Change the int i=0 in the first for loop to int i=index, then it pass the gradcheck with epsilon=1e-3 and 1e-4, but fail in 1e-5 or less, which i think is the problem of p…