This is really really interesting. So is this problem similar to the credit assignment problem with BP/BPTT, or just completely different? Are Tree-LSTMs a special case of this? Could you name a concrete example of when you would want to change the shape of the hidden layers besides when using neuroevolution or neural architecture search (although if I remember correctly, they just completely retrain from scratch each time they grow the network)? Is this problem also somehow related to transfer learning, like would changing the hidden layer repeatedly be kinda like doing transfer learning repeatedly during training? Also if the hidden layers change, is the new hidden layer initialized with the parameters from the hidden layer from the previous iteration and the additional weights - if the hidden layer grew - randomly initialized and if the hidden layer shrunk - some of the parameters removed, or is the new hidden layer randomly initialized?