Solved by ptrblck in post #2 
              
              
                Did you compare the forward and backward implementation to the PyTorch implementations and did you see any issues? 
The forward could be checked by calculating the abs().max() error of the outputs, while torch.autograd.gradcheck might be helpful to check the backward implementation.