Thank you very much for your replies!
Actually I think I fixed my original question: self.rnn was a nn.Module and now I also made in ScriptModule, but now I have a new problem. Looks like I cannot loop over nn.ModuleList. I tried to index it in the loop but did not work as well. Is it even possible to use jit for nn.ModuleList?
I omitted some parts for brevity:
class PyramidalRNNENcoder(ScriptModule):

__constants__ = ['num_layers']

def __init__(self, num_mels, encoder_size, num_layers, downsampling=None, dropout=0.0):
    super(PyramidalRNNENcoder, self).__init__()
    ...
    self.rnns =nn.ModuleList()
    for i in range(num_layers):
        input_size = num_mels*2 if i == 0 else encoder_size*2
        lstm_i = nn.LSTM(input_size,
                                 hidden_size=encoder_size, bidirectional=True)
        initialize_lstm(lstm_i)           
        self.rnns.append(lstm_i)
    self.num_layers = num_layers
    ...
    
@torch.jit.script_method
def forward(self, x, x_length):
    batch_size = x.size(0)
    ...
    idx = 0
    
    for rnn in self.rnns:
      ~~~~~~~~~~~~~~~~~~  RuntimeError:  python value of type 'ModuleList' cannot be used as a tuple:
      rnn_result = rnn(data)