If a and b are of the shape  (k,n,m) then the shape of your output is (k, n, n, m). This means that, if you are using int32, you’ll need to allocate at least 4*k*n*n*m bytes. In your case, it counts up to 4 * 4 * 6000 * 6000 * 1000 bytes, or 536 Gb, it seems a little too much for me. So maybe you should implement some lazy calculation of your output in order to fit it in any modern GPU.