So that you know which Tensors to call “expand” on 
In other words, this doesn’t need to change the autograd engine, just the layer implementations.
Changing all layers to do this by default would break things because of extra memory usage. But you could create custom layers that extend existing layer with “per-example-gradient-capturing” behavior using the expand trick.
Adding for Conv2d weight batching seems like a generally useful feature to have.  “groups” feature already implements something similar because each group has its own set of filters.