It seems that it doesn’t work well with feed forward. I don’t see a speedup nor reduced memory. However, when using other networks such as conv networks (as suggested) then there is indeed a significant speedup and memory reduction.