I’m curious about the theory behind having two linear final layers, though. Is it just to add another non-linearity (because of the activation function) at the end to train? Has there been any research that confirms that this yields better results than a single final linear layer?