you’re misunderstanding, these are “vector” ops, elementwise ops are “scalar” ones with one-to-one input to output element dependencies. With this trivial dependency structure, fusion implementation is relatively simple and generic. And what is talked about is inter-op fusion, so it works with two or more operations (e.g. log(a*b+c)), not e.g. nn.LayerNorm by itself.
ps IIRC there are some JIT optimizations for some layer combinations, that’s like another form of fusion too