Hi! Thank you for your answer. Eventually, I’ve used torch.cat.
I use group=1 in conv operation (both outputs depend on both inputs), therefore the property has to be rewritten in the following way:
conv(cat(A,B), cat(W1,W2)) = cat(conv(A,cat(W1,W2)), conv(B,cat(W1,W2))
And the implementation of this technique seems to be even less effective than a single torch.cat.