Will the gradient computed in this case the same as when I move the modification (code posted) into the network? It seems to me that there will be difference. I have not done enough experiments to conclude though.