Since your feature extractors share the same model, the gradients will be accumulated in the layers.
This could make the training a bit complicated, because the gradients in the first layers could be larger than in the last layers.
I’ve created a small code snippet showing that the gradients from the base model are the same as the sub-models:
vgg = models.vgg19(pretrained=False)
local_feature_extractor1 = nn.Sequential(*list(vgg.features)[:15])
local_feature_extractor2 = nn.Sequential(*list(vgg.features)[:-13])
local_feature_extractor3 = nn.Sequential(*list(vgg.features)[:-6])

x = torch.randn(1, 3, 224, 224)

output = vgg(x)
output.mean().backward()
print(vgg.features[0].weight.grad.abs().mean())

output = local_feature_extractor1(x)
output.mean().backward()
print(local_feature_extractor1[0].weight.grad.abs().mean())
print(vgg.features[0].weight.grad.abs().mean())

output = local_feature_extractor2(x)
output.mean().backward()
print(local_feature_extractor2[0].weight.grad.abs().mean())
print(vgg.features[0].weight.grad.abs().mean())

output = local_feature_extractor3(x)
output.mean().backward()
print(local_feature_extractor3[0].weight.grad.abs().mean())
print(vgg.features[0].weight.grad.abs().mean())