It’s sometimes easier to replace the last layer with an nn.Identity module instead of trying to remove it in order to get the penultimate activations. I don’t know which layer you would like to remove, but this workflow might also work for you.