I think it depends on your use case which approach to take.
I.e. in case you are rewriting the model architecture and would like to constantly return intermediates, I would probably stick to overriding the forward method, since it would also allow you to script the model (forward hooks are unsupported right now in torch.jit). It could also make the overall model architecture clearer, as new users wouldn’t need to check if and which layers are using hooks.
On the other hand, I’m using hooks most of the time as I’m usually more interested in a quick way to debug issues etc.
In any case, I haven’t seen a slowdown yet (but also didn’t profile it) so could you post a minimal code snippet showing your usage and the slowdown, please?