Solved by googlebot in post #4 
              
              
                if you have memory problem with this: 

vals = (A[rows]*B[cols]).sum(1) 

maybe make it a function and wrap it with utils.checkpoint