Whis is supported but you have to write your own module for it. A simple example is this:
class CustomModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = torch.nn.Conv2d(1,3,3)
        self.relu = torch.nn.ReLU()
        self.conv2 = torch.nn.Conv2d(3,1,3)

    def forward(self, input_batch):
        conv1 = self.conv1(input_batch)
        relu = self.relu(conv1)
        conv2 = self.conv2(relu) 
        return conv1, relu, conv2

If you do the whole loss calculation afterwards (which is what I would always recommend) you have access to all intermediate values. Alternatively you could use forward hooks 4 to obtain the intermediate values you need.