I think it is mostly correct, but I think you need to zero the bias of the fc layer.
Another line of code using
nn.init.zeros_(resnet50_feature_extractor.fc.bias)

I usually use
resnet50_feature_extractor = nn.Sequential(*list(models.resnet50(pretrained = True).children())[:-1])

but in this way you need to reshape the output by squeezing the spatial dimensions.