Yes, unfortunately, the cudnn ops always do the reduction.
You can check libraries like https://github.com/cybertronai/autograd-hacks 2 that implement custom backwards to replace these ops and give per-sample gradients efficiently.