Solved by inkplay in post #3 
              
              
                Hi ptrblck I actually posted this question to stackoverflow and some one helped me solve it. I neglected an important concept in progressive networks and that is fading in the channels. Eg toRGB and FromRGB in the progan paper. Here is the answer that helped me build more intuitions. 

Second, this …