Based on the error message it looks like the error is raised from a mismatch between dim1 of mat1 and dim0 of mat2, while I assume dim0 in mat1 is representing the batch size.
Are you initializing these matrices manually based on the batch size?
If not, could you post a minimal code snippet to reproduce this error?