You can save outputs and concatenate them to a bigger tensor. In the example below I take the first layer (pink layer) and concatenate it to the second (green) layer. This is input to the red layer. Note that it’s taking the whole layer, not just the pink nodes at the end as in your drawing. Hope you get the picture and can go from there 
Edit: You might have to change the shapes to make them fit. Use the torch.view() function. Also, you might find some inspirational code from a model class named Densenet, which basically does this but for CNNs
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.layer1 = # Init layers

    def forward(self, x):
        x1 = self.layer1(x) # Pink ones
        x2 = self.layer2(x1) # Green ones

        pink_and_green = torch.cat((x1, x2), dim=0)
        x3 = self.layer3(pink_and_green) # Red ones
        return x3