I think it is a bit different,
What I am saying, is a whole neural network at the bottom, which only knows how to predict lips, similarly whole neural network for tongue.
And then these combine together to form next hierarchy.
But what is combining two neural networks in PyTorch, so if one of them is able to predict lips, other nose, then combination would be able to predict (nose + lips)?