josevalim:

The custom broadcasting rules come in hand on some of the autograd operations. We can mimic the behaviour using broadcast+transpose but I wonder if there is a preferred way to achieve this (and if we could be missing some optimizations by going the broadcast+transpose route).


The only cost I can think of has to do with the overhead of the extra operations (e.g. transpose [or unsqueeze], plus normal broadcast, plus transpose back)