title: Strange sampling results from statsmodels.api.GLM (Generalised linear model)
body: <p>I encounter a problem in using python tool &quot;statsmodels.api.GLM&quot;, which I cannot understand. I come here asking for helps.</p>
  <p>I'm working on an example of (see the Section of) &quot;Cubic and Natual Cubic Splines&quot;
  on this page <a href="https://www.analyticsvidhya.com/blog/2018/03/introduction-regression-splines-python-codes/" rel="nofollow noreferrer">https://www.analyticsvidhya.com/blog/2018/03/introduction-regression-splines-python-codes/</a>  (data link is included in the page or <a href="https://drive.google.com/file/d/1QIHCTvHQIBpilzbNxGmbdEBEbmEkMd_K/view" rel="nofollow noreferrer">here</a>)</p>
  <p>The problem is that.  After fitting the data, I try to predict values at given places of x (eg. the xp00 and xp01 in the following code). Then I find that, once the requested positions having different min and max (i.e., the xp01) from the training x-set (i.e., the xp), the result becomes something else, not at all my transitional expectation that, at the same position, the prediction should be exactly the same value, whatever how you made the request because the fit to the data is done and fixed.   I'm expecting the pred01 is overlapped with pred00, but just shorter a the left end.</p>
  <pre><code># import modules
  import pandas as pd
  import numpy as np
  import statsmodels.api as sm
  import matplotlib.pyplot as plt 
  %matplotlib inline
  
  # read data_set
  data = pd.read_csv(&quot;Wage.csv&quot;)
  
  data.head()
  
  data_x = data['age']
  data_y = data['wage']
  
  # Dividing data into train and validation datasets
  from sklearn.model_selection import train_test_split
  train_x, valid_x, train_y, valid_y = train_test_split(data_x, data_y, test_size=0.33, random_state = 1)
  
  
  
  from patsy import dmatrix
  import statsmodels.api as sm
  import statsmodels.formula.api as smf
  from sklearn.metrics import mean_squared_error
  from math import sqrt
  
  
  # Generating cubic spline with 3 knots at 25, 40 and 60
  transformed_x = dmatrix(&quot;bs(train, knots=(25,40,60), degree=3, include_intercept=False)&quot;, {&quot;train&quot;: train_x},return_type='dataframe')
  
  # Fitting Generalised linear model on transformed dataset
  fit1 = sm.GLM(train_y, transformed_x).fit()
  
  # Prediction on splines
  pred1 = fit1.predict(dmatrix(&quot;bs(valid, knots=(25,40,60), include_intercept=False)&quot;, {&quot;valid&quot;: valid_x}, return_type='dataframe'))
  
  # Calculating RMSE values
  rms1 = sqrt(mean_squared_error(valid_y, pred1))
  print(rms1)
  #-&gt; 39.4
  
  # We will plot the graph for 70 observations only
  xp = np.linspace(valid_x.min(),valid_x.max(),70)
  xp00 = np.linspace(valid_x.min(),valid_x.max(),170)
  xp01 = np.linspace(valid_x.min()+4,valid_x.max(),170)   # just shift the lower bound a bit
  
  
  
  
  # Make some predictions
  pred1 = fit1.predict(dmatrix(&quot;bs(xp, knots=(25,40,60), include_intercept=False)&quot;, {&quot;xp&quot;: xp}, return_type='dataframe'))
  pred00 = fit1.predict(dmatrix(&quot;bs(xp, knots=(25,40,60), include_intercept=False)&quot;, {&quot;xp&quot;: xp00}, return_type='dataframe'))
  pred01 = fit1.predict(dmatrix(&quot;bs(xp, knots=(25,40,60), include_intercept=False)&quot;, {&quot;xp&quot;: xp01}, return_type='dataframe'))
  
  
  
  
  SMALL_SIZE = 4
  gamma=0.4
  plt.rc('font', size=SMALL_SIZE)
  plt.rc('axes', titlesize=SMALL_SIZE)
  plt.figure(figsize=(5,2),dpi=300)
  
  # Plot the splines and error bands
  plt.scatter(data.age, data.wage, facecolor='None', edgecolor='k', alpha=0.1)
  #plt.plot(xp, pred1, label='Specifying degree =3 with 3 knots')
  plt.plot(xp, pred1, color='r', label='Specifying degree =3 with 4 knots     xp')
  plt.plot(xp00, pred00, color='g', label='Specifying degree =3 with 4 knots  xp00')
  plt.plot(xp01, pred01, color='b', label='Specifying degree =3 with 4 knots  xp00')
  plt.legend()
  plt.xlim(15,85)
  plt.ylim(0,350)
  plt.xlabel('age')
  plt.ylabel('wage')
  plt.show()
  </code></pre>
  <p>Well, I have no right to enclose the figure in the post! so please click the link below and check the strange results.  Perhaps not strange just myself don't know how to use it. I'm ready to see.</p>
  <p><a href="https://i.stack.imgur.com/uFkGH.jpg" rel="nofollow noreferrer">the strange reuslts</a> (URL :<a href="https://i.stack.imgur.com/uFkGH.jpg" rel="nofollow noreferrer">https://i.stack.imgur.com/uFkGH.jpg</a>)</p>
  <p>Thanks!!</p>
  <p>Yanbin</p>