Why does the grads.shape above contains batch dimension twice? Ideally, I’d like to get something like [5, 10, 20] where 5 is the batch dimension.
I understand it may make sense when the outputs depend on other elements of the batch (e.g. with a batchnorm), but when I don’t need this dependence, is there a way to break it? Should I sum over one of these batch dimensions?
@albanD