Thanks for the quick reply @colesbury!
I swapped multi-class and multi-label, sorry for the confusion, I indeed meant multi-class as the main case to consider, and multi-label as an extended problem.
If I consider ImageNet1k (multi-class problem), I can use the following 2 models to achieve a good classifier:
A) model based on triplet loss + KNN or 1-layer network to determine class
B) model with softmax + cross entropy
Additionally, I could even take the before-last layer for model B to get a feature vector instead of a class to do some ranking (images with similar features ending up in the same cluster).
So now I can conclude these two models can do the same two things: classifying and ranking. I wonder how they compare:

If it is not an open-ended problem, is it still a good idea to use triplets for learning features instead of the simpler model A?
Can I expect better ranking of the feature vector by the triplet model A instead of the vector I can take from the before-last (or so) layer from model B?