I’m getting the row and col for the identity matrix by rows, cols = AAXW0W1.size() so I thought it’s matching the shape of vector I’m backpropagating. Am I wrong?
Also I was comparing the result with this function
def torch_autograd(A, W0, W1, X):
    # Forward
    W0W1 = torch.mm(W0, W1)
    AX = torch.mm(A, X)
    AXW0W1 = torch.mm(AX, W0W1)
    AAXW0W1 = torch.mm(A, AXW0W1)
    XW0W1 = torch.mm(X, W0W1)
    
    # Backward / Gradients
    rows, cols = AAXW0W1.size()
    AAXW0W1.backward(torch.eye(rows, cols))
    return A.grad