You could do this: Add a the model in init like this
self.main_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)

and then in the forward pass you could do this
x = self.main_model(x)
x = self.l1(x)