Hi,
The simplest think to do would be to make your nn.Module support batch dimension (using bmm and matmul) to replace your current version.
If you are using nightly builds, you might be able to get that automatically by using the new vmap Module. @richard will be able to help you with that if you want to go down that path