title: How do I change - using for loops to call multiple functions - into - using a pipeline to call a class?
body: <p>So the basic requirement is that, I get a dictionary of models from user, and a dictionary of their hyper parameters and give a report. Currently goal is for binary classification, but this can be extended later.</p>
  
  <p>This is what I am currently doing:</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  # import pandas_profiling as pp
  import matplotlib.pyplot as plt
  %matplotlib inline
  import seaborn as sns
  from sklearn.model_selection import train_test_split, cross_val_score, RandomizedSearchCV
  from sklearn.ensemble import RandomForestClassifier, VotingClassifier
  from sklearn.linear_model import LogisticRegression
  from sklearn.svm import SVC
  from sklearn.ensemble import GradientBoostingClassifier
  from sklearn.preprocessing import StandardScaler
  from sklearn.metrics import f1_score, roc_auc_score, recall_score, precision_score, make_scorer
  from sklearn import datasets
  # import joblib
  import warnings
  warnings.filterwarnings('ignore')
  
  cancer = datasets.load_breast_cancer()
  df = pd.DataFrame(cancer.data, columns=cancer.feature_names)
  df['target'] = cancer.target
  target = df['target']
  X_train, X_test, y_train, y_test = train_test_split(df.drop(columns='target', axis=1), target, test_size=0.4, random_state=13, stratify=target)
  
  def build_model(model_name, model_class, params=None):
      """
      return model instance
      """
      if 'Ridge' in model_name:
          model = model_class(penalty='l2')
      elif 'Lasso' in model_name:
          model = model_class(penalty='l1')
      elif 'Ensemble' in model_name:
          model = model_class(estimators=[('rf', RandomForestClassifier()), ('gbm', GradientBoostingClassifier())], voting='hard')
      else:
          model = model_class()
  
      if params is not None:
          print('Custom Model Parameters provided. Implementing Randomized Search for {} model'.format(model_name))
          rscv = RandomizedSearchCV(estimator=model, param_distributions=params[model_name],
                                    random_state=22, n_iter=10, cv=5, verbose=1, n_jobs=-1,
                                   scoring=make_scorer(f1_score), error_score=0.0)
          return rscv
  
      print('No model parameters provided. Using sklearn default values for {} model'.format(model_name))
      return model
  
  def fit_model(model_name, model_instance, xTrain, yTrain):
      """
      fit model
      """
      if model_name == 'SVM':
          scaler = StandardScaler()
          model = model_instance.fit(scaler.fit_transform(xTrain), yTrain)
      else:
          model = model_instance.fit(xTrain, yTrain)
  
      return model
  
  def predict_vals(fitted_model, xTest):
      """
      predict and return vals
      """
      if model_name == 'SVM':
          scaler = StandardScaler()
          y_prediction = fitted_model.predict(scaler.fit_transform(xTest))
      else:
          y_prediction = fitted_model.predict(xTest)
  
      return y_prediction
  
  def get_metrics(yTest, y_prediction):
      """
      get metrics after getting prediction
      """
      return [recall_score(yTest, y_prediction),
              precision_score(yTest, y_prediction), 
              f1_score(yTest, y_prediction),
             roc_auc_score(yTest, y_prediction)]
  
  def model_report(list_of_metrics):
      """
      add metrics to df, return df
      """
      df = pd.DataFrame(list_of_metrics, columns=['Model', 'Recall', 'Precision', 'f1', 'roc_auc'])
      df = df.round(3)
      return df
  
  models = {
      'Logistic Regression Ridge': LogisticRegression,
      'Logistic Regression Lasso': LogisticRegression,
      'Random Forest': RandomForestClassifier,
      'SVM': SVC,
      'GBM': GradientBoostingClassifier,
      'EnsembleRFGBM': VotingClassifier
  }
  
  model_parameters = {
      'SVM': {
          'C': np.random.uniform(50, 1, [25]),#[1, 10, 100, 1000],
          'class_weight': ['balanced'],
          'gamma': [0.0001, 0.001],
          'kernel': ['linear']
      },
      'Random Forest': {
          'n_estimators': [5, 10, 50, 100, 200],
          'max_depth': [3, 5, 10, 20, 40],
          'criterion': ['gini', 'entropy'],
          'bootstrap': [True, False],
          'min_samples_leaf': [np.random.randint(1,10)]
      },
      'Logistic Regression Ridge': {
          'C': np.random.rand(25),
          'class_weight': ['balanced']
      },
      'Logistic Regression Lasso': {
          'C': np.random.rand(25),
          'class_weight': ['balanced']
      },
      'GBM': {
          'n_estimators': [10, 50, 100, 200, 500],
          'max_depth': [3, 5, 10, None],
          'min_samples_leaf': [np.random.randint(1,10)]
      },
      'EnsembleRFGBM': {
          'rf__n_estimators': [5, 10, 50, 100, 200],
          'rf__max_depth': [3, 5, 10, 20, 40],
          'rf__min_samples_leaf': [np.random.randint(1,10)],
          'gbm__n_estimators': [10, 50, 100, 200, 500],
          'gbm__max_depth': [3, 5, 10, None],
          'gbm__min_samples_leaf': [np.random.randint(1,10)]
      }
  }
  </code></pre>
  
  <p>Without parameters I get the following report.</p>
  
  <pre><code># without parameters
  lst = []
  for model_name, model_class in models.items():
      model_instance = build_model(model_name, model_class)
      fitted_model = fit_model(model_name, model_instance, X_train, y_train)
      y_predicted = predict_vals(fitted_model, X_test)
      metrics = get_metrics(y_test, y_predicted)
  
      lst.append([model_name] + metrics)
  
  model_report(lst)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/qAdwW.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qAdwW.png" alt="enter image description here"></a></p>
  
  <p>With parameters given as input</p>
  
  <pre><code># with parameters
  lst = []
  for model_name, model_class in models.items():
      model_instance = build_model(model_name, model_class, model_parameters)
      fitted_model = fit_model(model_name, model_instance, X_train, y_train)
      y_predicted = predict_vals(fitted_model, X_test)
      metrics = get_metrics(y_test, y_predicted)
  
      lst.append([model_name] + metrics)
  
  model_report(lst)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/7Cj4y.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7Cj4y.png" alt="enter image description here"></a></p>
  
  <p>The task given to me right now is as follows.</p>
  
  <ol>
  <li>Take from user, a dictionary of models, and their parameters. If parameters are not provided, then use defaults of the models.</li>
  <li>Give as output the report (as seen in images)</li>
  </ol>
  
  <p>I was told that I should change the functions to classes. And avoid for loops if possible.</p>
  
  <p>My challenges:</p>
  
  <ol>
  <li>How do I change all the functions into a class and methods? Basically my senior wants something like</li>
  </ol>
  
  <p><code>report.getReport # gives the dataFrame of the report</code></p>
  
  <p>But the above sounds to me like it can be done in a function as follows (I don't understand why/how a class would be beneficial)</p>
  
  <pre><code>customReport(whatever inputs I'd like to give) # gives df of report
  </code></pre>
  
  <ol start="2">
  <li>How do I avoid <code>for loops</code> to get through the user inputs for various models? What I thought was that I could use <a href="https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html" rel="nofollow noreferrer">sklearn pipeline</a>, since according to my understanding, pipeline is a series of steps, so from user take the params and models, and execute them as a series of steps. This avoids the for loops.</li>
  </ol>
  
  <p>Something like this</p>
  
  <pre><code>customPipeline = Pipeline([ ('rf', RandomForestClassifier(with relevant params from params dict),
                               'SVC', SVC(with relevant params from params dict)) ] )
  </code></pre>
  
  <p><strong>Similar solution</strong> I found is <a href="http://www.davidsbatista.net/blog/2018/02/23/model_optimization/" rel="nofollow noreferrer">here</a> but I would like to avoid <code>for loops</code> as such.</p>
  
  <p><strong>Another related solution</strong>  <a href="https://stackoverflow.com/questions/50285973/pipeline-multiple-classifiers">here</a> is using a class which can switch between different models. But here I would require that the user be able to give option whether he wants to do Gridsearch/RandomizedSearch/CV/None. My thinking is that I use this class, then inherit this to another class which the user can give input to choose Gridsearch/RandomizedSearch/CV/None etc. I'm not sure if I'm thinking in the right direction.</p>
  
  <hr>
  
  <p><strong>NOTE</strong> A full working solution is desirable (would love it) but not mandatory. It is ok if your answer has a skeleton which can give me a direction how to proceed. I am ok with exploring and learning from it.</p>