title: Spark ML Linear Regression - What Hyper-parameters to Tune
body: <p>I'm using the <a href="http://spark.apache.org/docs/latest/ml-classification-regression.html#regression" rel="nofollow noreferrer">LinearRegression</a> model in the Spark ML for predicting price. It is a single variate regression <code>(x=time, y=price)</code>.</p>
  
  <p>Assume my data is clean, what are the usual steps to take to improve this model?</p>
  
  <p>So far, I tried tuning <code>regularization parameter</code> using <a href="http://spark.apache.org/docs/latest/ml-tuning.html#cross-validation" rel="nofollow noreferrer">cross-validation</a>, and got <code>rmse=15</code> given <code>stdev=30</code>.</p>
  
  <p>Are there any other significant hyper-parameters I should care about? It seems Spark ML is not well documented for hyper-parameter tuning...</p>
  
  <hr>
  
  <p><strong>Update</strong></p>
  
  <p>I was able to tune up parameters using <code>ParamGrid</code> and <code>Cross-Validation</code>. However, is there any way to see how the fitted line looks like after correctly training a linear regression model? How can I know if the line is <code>quadric</code> or <code>cubic</code> etc? It would be great if there is a way to visualize the fitted line with all training data points.</p>