Thanks! @ptrblck this does seem like a feasible solution. Is there a way to partially freeze the layers as well? As in freeze the older learned weights and train only the new portion initialized at random? Like freeze only half the layers?