What do you use to do this expand? Because the expand and expand_as functions can only expand dimensions that are originally of size 1. In that case, the gradients along all the expanded dimensions are accumulated onto the 1 dimension from the original Variable.