I hear you and agree. This si what’s causing me troubles:
  model1(
    (embedding): Embedding(43860, 620, padding_idx=0)
    (rnn): GRU(620, 2400, batch_first=True, dropout=0.25)
  )
  model2(
    (embedding): Embedding(43860, 620, padding_idx=0)
    (rnn): GRU(620, 1200, batch_first=True, dropout=0.25, bidirectional=True)
  )

models 1& 2 are pre-trained, and even though I set  grad = False the moment I add them - nothing works. I can’t quite reduce anything on those two - it’s a black box pretty much. Any advice for a creative workaround?