Hi everyone,
I’m trying to use the feature “hyperparameter_search” but cannot get the best model per the trial. Each trial always get the last value.
I see the code line here calling the Ray tune function to get the best model without the “scope” parameter passed. Default value of that function is “last”.

  

      github.com
  

  
    huggingface/transformers/blob/123cce6ffcafe6ecece7eb9914395d2ebcc98a48/src/transformers/integrations.py#L294



      
      
              # special attr set by tune.with_parameters
              if hasattr(trainable, "__mixins__"):
                  dynamic_modules_import_trainable.__mixins__ = trainable.__mixins__
          
          
    analysis = ray.tune.run(
                  dynamic_modules_import_trainable,
                  config=trainer.hp_space(None),
                  num_samples=n_trials,
                  **kwargs,
              )
              best_trial = analysis.get_best_trial(metric="objective", mode=direction[:3])
              best_run = BestRun(best_trial.trial_id, best_trial.last_result["objective"], best_trial.config)
              if _tb_writer is not None:
                  trainer.add_callback(_tb_writer)
              return best_run
          
          

          
def run_hp_search_sigopt(trainer, n_trials: int, direction: str, **kwargs) -> BestRun:
          
          
    from sigopt import Connection
          
      

    
  


  

  
    
    
  

  


Is there any way to workaround this?
Thanks!