title: Root mean log squared error issue with scitkit-learn.ensemble.GradientBoostingRegressor
body: <p>I am working on a Kaggle competition <a href="https://www.kaggle.com/c/nyc-taxi-trip-duration/data" rel="nofollow noreferrer">(data here)</a>, and I am having trouble using scikit-learn's GradientBoostingRegressor. The competition is using the root mean log squared error (RMLSE) to evaluate predictions.</p>
  
  <hr>
  
  <p>For the sake of an MWE, here is the code I used to clean the <code>train.csv</code> at the link above:</p>
  
  <pre><code>import datetime
  import pandas as pd
  
  train = pd.read_csv("train.csv", index_col=0)
  
  train.pickup_datetime = pd.to_datetime(train.pickup_datetime)
  train["pickup_month"] = train.pickup_datetime.apply(lambda x: x.month)
  train["pickup_day"] = train.pickup_datetime.apply(lambda x: x.day)
  train["pickup_hour"] = train.pickup_datetime.apply(lambda x: x.hour)
  train["pickup_minute"] = train.pickup_datetime.apply(lambda x: x.minute)
  train["pickup_weekday"] = train.pickup_datetime.apply(lambda x: x.weekday())
  train = train.drop(["pickup_datetime", "dropoff_datetime"], axis=1)
  train["store_and_fwd_flag"] = pd.get_dummies(train.store_and_fwd_flag, drop_first=True)
  
  X_train = train.drop("trip_duration", axis=1)
  y_train = train.trip_duration
  </code></pre>
  
  <hr>
  
  <p>To illustrate something that <em>works</em>, if I use a random forest, then the RMSLE is computed just fine:</p>
  
  <pre><code>import numpy as np
  from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
  from sklearn.metrics import make_scorer
  from sklearn.model_selection import cross_val_score
  
  
  def rmsle(predicted, real):
      sum=0.0
      for x in range(len(predicted)):
          p = np.log(predicted[x]+1)
          r = np.log(real[x]+1)
          sum = sum + (p - r)**2
      return (sum/len(predicted))**0.5
  
  rmsle_score = make_scorer(rmsle, greater_is_better=False)
  
  rf = RandomForestRegressor(random_state=1839, n_jobs=-1, verbose=2)
  rf_scores = cross_val_score(rf, X_train, y_train, cv=3, scoring=rmsle_score)
  print(np.mean(rf_scores))
  </code></pre>
  
  <p>This runs just fine. <strong>However</strong>, the gradient boosting regressor throws <code>RuntimeWarning: invalid value encountered in log</code>, and I get a <code>nan</code> from the <code>print</code> statement. Looking at the array of three RMSLE scores, they are all <code>nan</code>.</p>
  
  <pre><code>gb = GradientBoostingRegressor(verbose=2)
  gbr_scores = cross_val_score(gb, X_train, y_train, cv=3, scoring=rmsle_score)
  print(np.mean(gbr_scores))
  </code></pre>
  
  <p>I assume this is because I'm getting a negative value at some place where I shouldn't be. Kaggle told me it was encountering zero or non-negative RMSLE, as well, when I uploaded my predictions there to see if it was something about my code. Is there a reason why gradient boosting cannot be used for this problem? If I use <code>mean_squared_error</code> as the scorer (<code>mse_score = make_scorer(mean_squared_error, greater_is_better=False)</code>), it returns that just fine.</p>
  
  <p>I'm sure I'm missing something simple about gradient boosting; why is this scoring method <em>not</em> working for the gradient boosting regressor?</p>