Unfortunately, it ends up with the same error even after concatenation
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[1, 1, 384, 384] to have 3 channels, but got 1 channels instead