I found the solution to this and would like to share here for anyone reading in the future:
nn.Sequential() is not suitable in this case. Instead, you can use nn.ModuleList and manually create a module that performs the addition.