Hi @ptrblck
Let me try and answer with a list:

Data-set is AffectNet: http://mohammadmahoor.com/affectnet 58

I’ve removed certain labels as they seem to confuse the networks (uncertain, non-face, etc)
My top-1 with 7 labels is always below 30%, if I use only valence score (2 output classes) it goes up to 62%
I’ve calculted the mean and std of the entire data-set and I’m using it for normalisation. Using Imagenet mean and std was 1% to 2% worse only.
Input is RGB 224x224
The data seems to be from internet sources, I have no opinion on how “clean” or accurate it is, but the paper says that it has been human-annotated/labeled for the part I am using. I am open to using other datasets if you know a better one to suggest

I have 3 python scripts really, one which loads the custom dataset (I’ve written unit tests) and basically does the following:

load image
normalise and tensorify

A script of models which wraps around the output layers of AlexNet, SqueezeNet, VGG11, VGG19 (Only experimenting with AlexNet so far)
And the train and evaluate script with all the hyperparameters.
I have tuned it down to using:

0.001 learning rate
0.9 momentum and SGD optimiser in combination with CE loss
0.0005 L2
100 epochs
64 or 128 batch size
4000 images for training (out of about half a million) which are randomly picked. I am filtering uncertain, unknown and non-faces
I am using a MultuStepLR which seems to slightly improve accuracy

I can post the code if needed, but my most striking observation is that changing the output from 7 labels to 2 labels makes a huge impact as already mentioned. Trying with the original 11 labels would produce at best 20% top-1 accuracy.
I’ve been logging all training attempts in a mongo DB since this morning as I am trying to take a systematic and methodological approach. One observation I’ve made is that CE was exploding to a NaN when I used half instead of float and that with larger training data-sets it would go beyond 1.0 easily.
Overfitting seems to happen with the smaller training set (I got CE down to 0.000X) but it is always a bit noisy.
I’ll commit the code and then copy-paste the training and model scripts.