Solved by ptrblck in post #2 
              
              
                For a general PyTorch model you could use forward hooks to get the intermediate activations as described here.  Since NeMo seems to be using a PyTorch model internally, you would have to access its layers to register the hooks.