Hello, I would like to use the  prefix_allowed_tokens_fn as an input to the model.generate() function, in order to perform constrained text generation with BART.

  

      github.com
  

  
    huggingface/transformers/blob/master/src/transformers/generation_utils.py#L670 16



      
      
              length_penalty: Optional[float] = None,
              no_repeat_ngram_size: Optional[int] = None,
              encoder_no_repeat_ngram_size: Optional[int] = None,
              num_return_sequences: Optional[int] = None,
              max_time: Optional[float] = None,
              max_new_tokens: Optional[int] = None,
              decoder_start_token_id: Optional[int] = None,
              use_cache: Optional[bool] = None,
              num_beam_groups: Optional[int] = None,
              diversity_penalty: Optional[float] = None,
              prefix_allowed_tokens_fn: Optional[Callable[[int, torch.Tensor], List[int]]] = None,
              output_attentions: Optional[bool] = None,
              output_hidden_states: Optional[bool] = None,
              output_scores: Optional[bool] = None,
              return_dict_in_generate: Optional[bool] = None,
              forced_bos_token_id: Optional[int] = None,
              forced_eos_token_id: Optional[int] = None,
              remove_invalid_values: Optional[bool] = None,
              synced_gpus: Optional[bool] = None,
              **model_kwargs,
          ) -> Union[GreedySearchOutput, SampleOutput, BeamSearchOutput, BeamSampleOutput, torch.LongTensor]:
      
    
  


  

  
    
    
  

  


I tried to adapt the function in the original repository here 66, but it doesn’t seem to be working. Can you please tell me if there are any examples of the kinds of functions that can be given as input to this parameter? Thank you!