Hey Andrew,
thanks for the quick response!
Regarding your aspects to consider:


The loss should be calculated as usual, e.g. cross entropy between true and predicted label.


I mean the output after global average pooling, i.e. the input for the fully connected layer. So it should be just a normally trained ResNet that is cut open before the FCE and then presents a 256-dimensional embedding.


Is your code equivalent to the following?


resnet18 = torchvision.models.resnet18(pretrained=False)
resnet18.fc.out_features = 256

If I understand your code correctly, this would create a 256-dimensional embedding, but not be trainable to be class-discriminative (unless adding another FC-layer, which I do not want to do). In contrast, I desire the embedding coming directly from the convolutional operations (and downsampled by GAP).
Best,
always