Thanks a bunch for this explanation!! 
A couple of things I would ask…
Correct me if I’m wrong here. So, putting it in mathematical terms. If I want this difference to be large,
KKMN - (KKM + MN)  = KKMN - M(KK + N) = M(KKN - KK - N)
I should have a large M, and should try other values for K and N. This is the correct approach, right?
And second, although doing this might give me a large difference, but possibly will take me away from my main objective of having least number of params to begin with. If that is correct, can you shed some light on how can I tweak PyTorch model architecture to have a model with fewer params (because I had a model on Keras with same structure and had about 3000 paramters)? Or is having fewer parameters even a good metric for a model to utilize less resources during inference?