There was a PR in pytorch doing that (running stats can also be absorbed in the previous layer). https://github.com/pytorch/pytorch/pull/901 316. It can be done once the network is trained and parameters no longer change. Other than for performance, it does not matter whether batch norm is absorbed or run as a separate layer, results are the same.