Hi Reuben.
While I can’t share the exact code because I don’t own it, here’s the gist of what I did:
model = T5ForConditionalGeneration(...)
encoder_outputs = model.encoder(encoder_input_ids, return_dict=True, output_hidden_states=True)
decoder_input_ids = torch.tensor([t5config.starter_token_id])
generated = model.greedy_search(decoder_input_ids, encoder_outputs=encoder_outputs)