Yeap you are right, to make things more clear I guess I need to post the whole code. However, as i run the code above, I think the network trains as I want. Compared to training the whole net, it performance worse on the aux output, yet better than random.