I have a model that looks like this:
RegressionRNN(
  (ms_group): MorphosynGroup(
    (rlayer): GRU(147, 256, batch_first=True, bidirectional=True)
  )
  (ft_group): FastTextGroup(
    (rlayer): GRU(147, 256, batch_first=True, bidirectional=True)
  )
  (concat): Concatenator()
  (linear_layer): Linear(in_features=512, out_features=1, bias=True)
)

This is a model with two different kind of inputs. One goes into ms_group, the other goes into ft_group. The results of these groups are then concatenated, and on the concatenation the linear layer is then executed. I feel, though, that the model as represented above is not semantically clear about what is being concatenated. The concatenator class looks like this
class Concatenator(nn.Module):
    def __init__(self):
        super(Concatenator, self).__init__()

    def forward(self, inputs):
        if len(inputs) > 1:
            catted = torch.cat(inputs, dim=1)
        else:
            catted = inputs[0]

        return catted 

It is simple enough, but I wanted to put this in its separate group so that you can see from the model representation that somewhere a concatenation is taking place. I feel, though, that improvements are possible but I’m not sure how. Should I place the ms_group, ft_group, and concat in their own group (e.g. ‘encoder’), and the linear layer as the decoder layer?
The code is written in a highly modular way so that you can choose to only use ms_group, or only ft_group, and based on the used number of groups the results are concatenated or not, and so on. So I wish to make it more visibly clear what is going on but I’m not sure what the conventions in PyTorch are to do this.