You can use https://pytorch.org/docs/stable/torch.html#torch.split 305


Split xinput into xinput1 and xinput2 using torch.split
Apply different non-linearity on xinput1 and 2 respectively
Use torch.cat to concatenate