Answers.Body: <p>Looking at your data, I see that it's only 126 samples long. For datasets of that size, classical statistical methods outperform all ML methods. Look at <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889" rel="nofollow noreferrer">this paper</a>, where they use the 1045 M3 univariate time series to show that old school statistical methods always outperform machine learning methods for short sequences.</p>
  <p>Sure, your data is not univariate, but the argument still stands; Your dataset is nowhere large enough to justify using ML.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-10T03:59:51.413Z
Answers.Text: That reference paper gave a valid point, I'll work on applying statistical methods and then can compare with the ML models too. Could a larger dataset hopefully give better result with the above code?  Can I use statistical methods/above code to predict multiple outputs? example: `dataset = pd.read_csv('Data.csv')
  X = dataset.iloc[:, 0:3].values
  y = dataset.iloc[:, 3:9].values` If you could do so, please provide me with more references that might help with my work.
Answers.UserDisplayName: Taqi Ahmed
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-10T09:47:31.787Z
Answers.Text: @TaqiAhmed Multiple outputs can easily be dealt with by creating multiple models; That's how many SK-learn models actually do it under the hood.
  Multiple inputs is much more of a challenge; Look at something like ARIMAX and Vector Autoregressive models.
Answers.UserDisplayName: Avatrin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-09T21:04:38.250Z
Answers.LastActivityDate: 2021-11-09T21:04:38.250Z
Answers.UserDisplayName: Avatrin
Body: <p>I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results?</p>
  <p>I get similar performance for ANNs, decision tree, and random forest (slightly better) and even negative for polynomial regression.
  The graphs for test data performance and training data also get a DataConversionWarning</p>
  <p>You can find the data I used <a href="https://drive.google.com/file/d/1QSpscwi-00T-mNP3xrtYdwZCud9dv691/view" rel="nofollow noreferrer">here</a></p>
  <p>The plots I obtained look like this:</p>
  <p><a href="https://i.sstatic.net/x4GJD.png" rel="nofollow noreferrer">actual vs predicted for test data</a>
  <a href="https://i.sstatic.net/w2zrv.png" rel="nofollow noreferrer">actual vs predicted for training data</a></p>
  <pre><code>import numpy as np
  import matplotlib.pyplot as plt
  import pandas as pd
  from sklearn.model_selection import train_test_split
  from sklearn.svm import SVR
  from sklearn.metrics import r2_score
  #
  #
  dataset = pd.read_csv('Data.csv')
  X = dataset.iloc[:, :-1].values
  y = dataset.iloc[:, -1].values 
  #
  #
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 1)
  #
  #
  regressor = SVR(kernel = 'linear', gamma='auto')
  regressor.fit(X_train, y_train.ravel())
  y_predict = regressor.predict(X_test)
  np.set_printoptions(precision=2)
  print(np.concatenate((y_predict.reshape(len(y_predict),1), y_test.reshape(len(y_test),1 )), 1))
  #
  #
  r2_score(y_test, y_predict)
  #
  #
  #model performance
  fig, ax = plt.subplots()
  ax.scatter(y_test, y_predict)
  ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)
  ax.set_xlabel('Actual')
  ax.set_ylabel('Predicted')
  #regression line
  y_test, y_predict = y_test.reshape(-1,1), y_predict.reshape(-1,1)
  ax.plot(y_test, regressor.fit(y_test, y_predict).predict(y_test))
  ax.set_title('Final Prediction-R2: ' + str(r2_score(y_test, y_predict)))
  plt.show()
  #
  #
  #training data performance
  fig, ax = plt.subplots()
  ax.scatter(y_train, y_train_predict)
  ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)
  ax.set_xlabel('Actual')
  ax.set_ylabel('Predicted')
  #regression line
  y_train, y_train_predict = y_train.reshape(-1,1), y_train_predict.reshape(-1,1)
  ax.plot(y_train, regressor.fit(y_train, y_train_predict).predict(y_train))
  ax.set_title('Final Prediction-R2: ' + str(r2_score(y_train, y_train_predict)))
  plt.show()
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-09T16:48:34.913Z
Comments.Text: You are using a linear kernel. Why would you expect anything good out of data you know are non-linear? Try different kernels first. Those are what makes support vector machines/regressors non-linear.
Comments.UserDisplayName: Avatrin
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-09T20:07:40.820Z
Comments.Text: I already did try rbf, poly and linear kernels and got the best score of 0.42 with rbf kernel.  `regressor_linear = SVR(kernel = 'linear',C=100, gamma='auto')
  regressor_rbf = SVR(kernel = 'rbf',C=100,gamma=0.1, epsilon=0.1)
  regressor_poly = SVR(kernel = 'poly', gamma='auto', degree=3,epsilon=0.1, coef0=1)
  regressor_linear.fit(X_train, y_train.ravel())`
Comments.UserDisplayName: Taqi Ahmed
Tags: <regression><performance><support-vector-machine><scikit-learn>
Title: Do I need to tune the hyper-parameters or more data if SVR model performs poorly?
UserDisplayName: Taqi Ahmed
ThreadText: # Do I need to tune the hyper-parameters or more data if SVR model performs poorly?
  
  (Asked by: username_0 on 2021-11-07)
  
  I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results?
  I get similar performance for ANNs, decision tree, and random forest (slightly better) and even negative for polynomial regression.
  The graphs for test data performance and training data also get a DataConversionWarning
  You can find the data I used here
  The plots I obtained look like this:
  actual vs predicted for test data
  actual vs predicted for training data
  import numpy as np
  import matplotlib.pyplot as plt
  import pandas as pd
  from sklearn.model_selection import train_test_split
  from sklearn.svm import SVR
  from sklearn.metrics import r2_score
  #
  #
  dataset = pd.read_csv('Data.csv')
  X = dataset.iloc[:, :-1].values
  y = dataset.iloc[:, -1].values 
  #
  #
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 1)
  #
  #
  regressor = SVR(kernel = 'linear', gamma='auto')
  regressor.fit(X_train, y_train.ravel())
  y_predict = regressor.predict(X_test)
  np.set_printoptions(precision=2)
  print(np.concatenate((y_predict.reshape(len(y_predict),1), y_test.reshape(len(y_test),1 )), 1))
  #
  #
  r2_score(y_test, y_predict)
  #
  #
  #model performance
  fig, ax = plt.subplots()
  ax.scatter(y_test, y_predict)
  ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)
  ax.set_xlabel('Actual')
  ax.set_ylabel('Predicted')
  #regression line
  y_test, y_predict = y_test.reshape(-1,1), y_predict.reshape(-1,1)
  ax.plot(y_test, regressor.fit(y_test, y_predict).predict(y_test))
  ax.set_title('Final Prediction-R2: ' + str(r2_score(y_test, y_predict)))
  plt.show()
  #
  #
  #training data performance
  fig, ax = plt.subplots()
  ax.scatter(y_train, y_train_predict)
  ax.plot([y.min(), y.max()], [y.min(), y.max()], 'k--', lw=4)
  ax.set_xlabel('Actual')
  ax.set_ylabel('Predicted')
  #regression line
  y_train, y_train_predict = y_train.reshape(-1,1), y_train_predict.reshape(-1,1)
  ax.plot(y_train, regressor.fit(y_train, y_train_predict).predict(y_train))
  ax.set_title('Final Prediction-R2: ' + str(r2_score(y_train, y_train_predict)))
  plt.show()
  
  **Tags:** regression, performance, support-vector-machine, scikit-learn
  > **Comments:**
  > **username_1**: You are using a linear kernel. Why would you expect anything good out of data you know are non-linear? Try different kernels first. Those are what makes support vector machines/regressors non-linear.  
  > **username_0**: I already did try rbf, poly and linear kernels and got the best score of 0.42 with rbf kernel.  `regressor_linear = SVR(kernel = 'linear',C=100, gamma='auto') regressor_rbf = SVR(kernel = 'rbf',C=100,gamma=0.1, epsilon=0.1) regressor_poly = SVR(kernel = 'poly', gamma='auto', degree=3,epsilon=0.1, coef0=1) regressor_linear.fit(X_train, y_train.ravel())`  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-11-09. Score: 0 | Accepted answer
  
  Looking at your data, I see that it's only 126 samples long. For datasets of that size, classical statistical methods outperform all ML methods. Look at [this paper](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889), where they use the 1045 M3 univariate time series to show that old school statistical methods always outperform machine learning methods for short sequences.
  
  Sure, your data is not univariate, but the argument still stands; Your dataset is nowhere large enough to justify using ML.
  
  > **Comments:**
  > **username_0**: That reference paper gave a valid point, I'll work on applying statistical methods and then can compare with the ML models too. Could a larger dataset hopefully give better result with the above code?  Can I use statistical methods/above code to predict multiple outputs? example: `dataset = pd.read_csv('Data.csv') X = dataset.iloc[:, 0:3].values y = dataset.iloc[:, 3:9].values` If you could do so, please provide me with more references that might help with my work.  
  > **username_1**: @TaqiAhmed Multiple outputs can easily be dealt with by creating multiple models; That's how many SK-learn models actually do it under the hood. Multiple inputs is much more of a challenge; Look at something like ARIMAX and Vector Autoregressive models.  
  
  ---
  
  