I am trying to implement Split brain AutoEncoder 12 paper for understanding and it needs merging. I  am not sure I misinterpret the paper, but as per knowledge the author is trying to implement as follows:
L_channel (Input) => Network 1 => ab_channel (output)
ab_channel (Input) => Network 2 => L_channel (output)

finally merge both networks along channels for each layers to test different Classification/Detection network comparisons. e.g. Classification network
RGB => (Network 1 + Network 2) => fc5 features => classifier
Now I am not sure how to merge them, if I try to use ‘group’ parameter in convolution  to split the network then how do we split 3 channel input so that 1 part of network looks “L” channel and other “ab” channel.
It might need some time to read the paper t answer my query. This is explained in section  3.1/3.2 of paper.
Thanks in advance.