Thank you for answering my question.
Here is my code.
class FeatureExtractor(nn.Module):
    def __init__(self, model: nn.Module, hook_layers: List[str]):
        super(FeatureExtractor, self).__init__()
        self.model = model
        self.hook_layers = hook_layers

        for name, module in self.model.named_modules():
            if name in self.hook_layers:
                module.register_forward_hook(self.extract())

        self.features = []

    def extract(self):
        def _extract(module, f_in, f_out):
            self.features.append(f_out)
        return _extract

    def forward(self, input):
        _ = self.model(input)
        return self.features


And the simple experiment was like below

The backbone of model is very lightly modified MobileNet V1, and three features from it are feed to only the classifier of RetinaNet.
The loss functiion is CrossEntropyLoss of nn, and the optimizer is Adam.
Training was done only 1 epoch on MNIST dataset with 32 batches.

As a result, training with modifiying forward method took about 350 seconds on average, and the ‘hook-way’ took 370 seconds on average.
But since these are run on the laptop’s CPU, it couldn’t be stable, I think.