Hello.
While implementing SSD series, I found that their extracting features is quite tricky. Most of implementations just modifiy forward method, but in my opinion, it would be better using forward hook.
So, with simple experiment on CPU, I found it does not affect on parameters and gradients of model.
But there was about 20 seconds delay compared to ‘forward method way’, which took 350 seconds.
(Considering 60000 inputs of MNIST, 20 / 60000 s is very short time, but anyway there was a delay)
I registered forward hook only once at Model class, and the only function applied to register_forward_hook was python-list-append for 3 tensors.
But it’s just an experimental conclusion, and I’m still not clear whether to use hook or not for running model.
If anyone knows if the register forward hook affects on speed of model running and why, or other reason of not recommending using hook, please help me.