You can print out any PyTorch model to see its architecture, or you can simply look at the source code. On the highest level you’ll find shared embeddings shared, an encoder encoder, a deocder decoder and a LM head lm_head. You can use any of those. So for instance, unfreezing the decoder transformer layers:
for i, m in enumerate(model.decoder.block):        
    #Only un-freeze the last n transformer blocks in the decoder
    if i+1 > 12 - config['UNFREEZE_LAST_N']:
        for parameter in m.parameters():
            parameter.requires_grad = True 

It is block because the decoder is a T5Stack which has its transformer layers as a ModuleList called block.

  

      github.com
  

  
    huggingface/transformers/blob/8bbb53e20b7873ba7f63be70d4d798e0c3568bfa/src/transformers/models/t5/modeling_t5.py#L811 3



      
      
                  return shifted_input_ids
          
          

          
class T5Stack(T5PreTrainedModel):
              def __init__(self, config, embed_tokens=None):
                  super().__init__(config)
          
          
        self.embed_tokens = embed_tokens
                  self.is_decoder = config.is_decoder
          
          
        self.block = nn.ModuleList(
                      [T5Block(config, has_relative_attention_bias=bool(i == 0)) for i in range(config.num_layers)]
                  )
                  self.final_layer_norm = T5LayerNorm(config.d_model, eps=config.layer_norm_epsilon)
                  self.dropout = nn.Dropout(config.dropout_rate)
          
          
        self.init_weights()
                  # Model parallel
                  self.model_parallel = False
                  self.device_map = None
                  self.gradient_checkpointing = False
      
    
  


  

  
    
    
  

  


So it isn’t hard at all, you just have to go look for the right name in the code.