If you need embeddings after training is complete just do another loop with model.eval() and either use forward hook of modify alexnet to get 4096 FC layer output in eval mode or if you pass additional parameter to forward method. You don’t need to split samples per class because you can always get current class from the target labels.