If you would like to use a ResNet as the base model, just assign it as such and change the number of output features:
self.base = models.resnet50(pretrained=True)
self.base.fc = nn.Linear(self.base.fc.in_features, 64)
self.fc_vowels = nn.Linear(64, ...)
...