Im not sure if thats right, I think one one need another convolution over the hidden units (not just adding stuff, that would not enforce weight sharing) because its the same function being applied at each “time” dimension.