Answers.Body: <blockquote>
    <p>Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?</p>
  </blockquote>
  
  <p>I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.</p>
  
  <p>I would try with a smaller network, so you "force" it to learn something, instead of memorizing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-31T15:10:39.877Z
Answers.Text: So maybe one hidden layer with 20 to 30 neurons?
Answers.UserDisplayName: alim1990
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-31T13:27:44.003Z
Answers.LastActivityDate: 2020-01-31T13:27:44.003Z
Answers.UserDisplayName: Vildemort
Body: <p>I am trying to solve the kaggle's house prices using neural network. I've already made it with ensembling several models (XGBoost, GradientBooster and Ridge) and I've got a great score ranking me between the top 25%.</p>
  
  <p>I imagined that by adding a new model to the ensembled models like ANN would increase prediction accuracy, so I did the following:</p>
  
  <pre><code>import keras
  
  model = keras.models.Sequential()
  
  model.add(keras.layers.Dense(235, activation='relu', input_shape=(235,)))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(1))
  
  model.compile(optimizer='adam', loss='mean_squared_error')
  model.summary()
  model.fit(dataset_ann, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=3)])
  y_pred = model.predict(X_test_ann)
  </code></pre>
  
  <p>I choosed 235 neurons for each layer, as the training set has 235 features.</p>
  
  <p>For model ensembling:</p>
  
  <pre><code>y_p = (0.1*model.predict(X_test_ann)+0.2*gbr.predict(testset)+0.3*xgb.predict(testset)+0.1*regressor.predict(testset)+0.1*elastic.predict(testset)+0.2*ridge.predict(testset))
  </code></pre>
  
  <p>The shape of <code>y_p</code> is <code>(1459, 1459)</code> instead of <code>(1459, )</code> where columns are all having the same values, so taking <code>y_p[0]</code> would be more than enough.</p>
  
  <p>I submitted the result into kaggle and went from top 25% into bottom 60%.</p>
  
  <p>Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?</p>
  
  <p>I tried with epoch = 30, 100, 1000 and got nearly the same bad ranking. </p>
  
Tags: <neural-networks><python><artificial-neuron>
Title: Applying Artificial neural network into kaggle's house prices data set gave bad predicted values
UserDisplayName: alim1990
ThreadText: # Applying Artificial neural network into kaggle's house prices data set gave bad predicted values
  
  (Asked by: username_0 on 2020-01-31)
  
  I am trying to solve the kaggle's house prices using neural network. I've already made it with ensembling several models (XGBoost, GradientBooster and Ridge) and I've got a great score ranking me between the top 25%.
  
  I imagined that by adding a new model to the ensembled models like ANN would increase prediction accuracy, so I did the following:
  
  ```
  import keras
  
  model = keras.models.Sequential()
  
  model.add(keras.layers.Dense(235, activation='relu', input_shape=(235,)))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(235, activation='relu'))
  model.add(keras.layers.Dense(1))
  
  model.compile(optimizer='adam', loss='mean_squared_error')
  model.summary()
  model.fit(dataset_ann, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=3)])
  y_pred = model.predict(X_test_ann)
  
  ```
  
  I choosed 235 neurons for each layer, as the training set has 235 features.
  
  For model ensembling:
  
  ```
  y_p = (0.1*model.predict(X_test_ann)+0.2*gbr.predict(testset)+0.3*xgb.predict(testset)+0.1*regressor.predict(testset)+0.1*elastic.predict(testset)+0.2*ridge.predict(testset))
  
  ```
  
  The shape of `y_p` is `(1459, 1459)` instead of `(1459, )` where columns are all having the same values, so taking `y_p[0]` would be more than enough.
  
  I submitted the result into kaggle and went from top 25% into bottom 60%.
  
  Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?
  
  I tried with epoch = 30, 100, 1000 and got nearly the same bad ranking.
  
  **Tags:** neural-networks, python, artificial-neuron
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-31. Score: 2
  
  > Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?
  
  I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.
  
  I would try with a smaller network, so you "force" it to learn something, instead of memorizing.
  
  > **Comments:**
  > **username_0**: So maybe one hidden layer with 20 to 30 neurons?  
  
  ---
  
  