Sorry, I don’t know how tensorflow deal with this case. But it seems a technique report Memory-Efficient Implementation of DenseNets 136 has already implemented this feature in pytorch.
I will concatenate only in dimension 0 and 1. Will try this trick. Thanks for your help.