Hi,
The input tensor, once the batch dimension is added will be 192 x 4096 x 4096 that adds up to ~12GB of memory.
If you want to handle the batch dimension in a less memory hangry manner, I would suggest: w = torch.bmm(A.unsqueeze(0).expand_as(v), v).