OK! I made a stupid mistake.
I just figured out what had happened.
A simple test is below:
import torchvision
import torch

model = torchvision.models.detection.keypointrcnn_resnet50_fpn()
model.eval()

all_features = []

class test_dataset(object):
    def __init__(self):
        pass
    def __getitem__(self, idx):
        return torch.rand((3,224,224)), torch.rand((1))
    def __len__(self):
        return 5

dataset = test_dataset()

for image,target in dataset:
    image = torch.unsqueeze(image,0) # use one image due to the limit of memeory in pratical use
    all_features = []
    def get_layer(module, x):
        all_features.append(x)
    model.rpn.head.register_forward_pre_hook(get_layer) # repeatly register!
    with torch.no_grad():
        out = model(image)    
    print(len(all_features))

And it will get:
1 2 3 4 5 6…
It’s because:
model.rpn.head.register_forward_pre_hook(get_layer) # repeatly register!

This function is used repeatedly, so the “forward()” function will get another hook every time, resulting in the linearly increasing of the features.
Thank u for your coding advice!