Hello,
I’ve been trying for a couple of months now to train a network to recognise facial expressions.
When I discovered PyTorch I spent time reading all the tutorials, blogs on CNN, and so on.
I’ve spent a good 5-6 years on ML and yet I’m still dumbfounded by how complex training a CNN appears to be.
I’ve got a very large (half a million) dataset of facial expressions, and a Titan Xp to my disposal.
At first I tried copying simple CNN architectures from various papers. Tried small input (64x64) tried single channel grayscale, etc.
I then tried using existing models (VGG11, VGG19) on grayscale, and recently I tried with RGB.
All my attempts have failed so far. Most won’t even converge, and the few that do (using face valence and not labels/classes) when evaluating are very very poor (less than 35% accuracy).
I want to ask:

can I fine tune an existing pre-trained network such as ResNet150 or inception for my task? Do I simply freeze the feature extractors, change the last layer/classifier and re-train? I’ve searched current answers, posts and github repositories but it’s not very clear.
does normalisation play an important role? I can calculate means and standard deviation for the dataset
should I even bother training a large network from scratch? E.g, VGG19, ResNet18, etc?
what kind of accuracy during evaluation should I realistically expect?
I’ve tried using data augmentation, but it appears to have made matters worse (e.g., CE never drops below a certain value)
I can probably throw a second Titan Xp to the task, but this is becoming very stressing. Any help, advise, feedback or criticism is more than welcomed!