The cudnn error could still mask the OOM issue.
You could try to disable cudnn via torch.backends.cudnn.enabled = False and check the lower batch size again and also use the benchmark mode via torch.backends.cudnn.benchmark = True or deterministic via torch.backends.cudnn.deterministic = True.
If the code is running fine without cudnn and just raises the NOT_SUPPORTED issue, could you post your model all necessary input shapes, so that we could try to debug this issue?