You are extracting features from the pre-trained VGG and training your own classification layer, I am right to assume that? Because you are extracting features then flattening them. If you do not remove the last layer of the VGG, your output would be of the shape-[batch_size, num_classes], already giving you the class probabilities.
Though you can always add extra linear layers on top of the VGG network.