So you would also do something like my SampleConv approach from my first post i.e. selecting certain kernels? Would it make sense to also select only output channels?
And yes, at second glance I think you are totally right. Addition makes no sense at all so concatenation might be the way to go.
Unfortunately I still can’t wrap my head around their claimed 75% decrease in memory consumption.