Thanks @tom for you reply. Is there be any other solution to this problem? I have shown a simple example here. My actual implementation is very much similar, except it requires out of place implementation (C=A * B[:, :, None, None]). As I have seen that the out place implementation of the solution given by @tom, in my case is highly memory inefficient for a batch size of 64. The error messege is:
RuntimeError: CUDA out of memory. Tried to allocate 256.00 GiB (GPU 0; 23.65 GiB total capacity; 2.12 GiB already allocated; 20.16 GiB free; 2.20 GiB reserved in total by PyTorch)