Problem also exists in a much simpler situation.  If I just use Mod1 and add 0.0001 to the output same thing happens.  Conv2d->sigmoid makes grad_in be inverse sigmoid of grad_out.  Conv2d->sigmoid->add 0.0001 makes grad_in = grad_out