Your current implementation misses the relu as well as adaptive pooling layer from the DenseNet implementation as used here 5, which would create a tensor of [batch_size, 1024, 7, 7] for the output of self.densenet.features.
Also, you might want to concatenate the activation tensors in dim1 (feature dimension).
These changes should work:
...
    y = self.densenet.features(y)
    y = F.relu(y)
    y = F.adaptive_avg_pool2d(y, (1, 1))
    y = y.view(y.size(0), -1)
    # y = F.relu(self.dfc1(y)) remove this here, if you are using the first relu

    x = torch.cat((x,y), 1)
...