Body: <p>I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the <a href="http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/welcome_e.html" rel="nofollow noreferrer">Java Neural Network Simulator</a>, JavaNNS, program.</p>
  
  <p>In one practice I have to build a neural network to use backpropagation on it.</p>
  
  <p>I have created a neural network with one input layer with 12 nodes, one hidden layer with 6 nodes and one output layer with 1 node.</p>
  
  <p>I'm using Kaggle's titanic competition data with this format following Dataquest course <a href="https://www.dataquest.io/course/kaggle-fundamentals" rel="nofollow noreferrer">Getting Started with Kaggle for Titanic competition</a>:</p>
  
  <pre><code>Pclass_1,Pclass_2,Pclass_3,Sex_female,Sex_male,Age_categories_Missing,Age_categories_Infant,Age_categories_Child,Age_categories_Teenager,Age_categories_Young Adult,Age_categories_Adult,Age_categories_Senior,Survived
  0,0,1,0,1,0,0,0,0,1,0,0,0
  1,0,0,1,0,0,0,0,0,0,1,0,1
  0,0,1,1,0,0,0,0,0,1,0,0,1
  1,0,0,1,0,0,0,0,0,1,0,0,1
  0,0,1,0,1,0,0,0,0,1,0,0,0
  0,0,1,0,1,1,0,0,0,0,0,0,0
  1,0,0,0,1,0,0,0,0,0,1,0,0
  0,0,1,0,1,0,1,0,0,0,0,0,0
  </code></pre>
  
  <p>If you want see the same data better in an Spreadsheet:
  <a href="https://i.sstatic.net/Ab2aX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Ab2aX.png" alt="enter image description here"></a></p>
  
  <p>But they preprocess the data to use it with linear regression and I don't know if I can use these data with backpropagation</p>
  
  <p>I think something is wrong because when I run backpropagation in JavaNNS I get these data:</p>
  
  <pre><code>opened at: Sat Feb 17 17:29:40 CET 2018
  Step 200 MSE:   0.5381023044692738  validation: 0.11675894327003862
  Step 400 MSE:   0.5372328944712378  validation: 0.11700781497209432
  Step 600 MSE:   0.5370386219557437  validation: 0.11691717861750939
  Step 800 MSE:   0.5370348711919518  validation: 0.11696104763606407
  Step 1000 MSE:  0.5369724294992798  validation: 0.11697568840154722
  Step 1200 MSE:  0.5369697016710676  validation: 0.11665485957481342
  Step 1400 MSE:  0.5370053339270906  validation: 0.11684215268609244
  Step 1600 MSE:  0.5370121961199371  validation: 0.11670833992558485
  Step 1800 MSE:  0.5370200812483633  validation: 0.11673550099633925
  Step 2000 MSE:  0.5367923502149529  validation: 0.11675956129361797
  </code></pre>
  
  <p>Nothing changes, it is like it doesn't learn anything.</p>
  
  <p>How many hidden layers does the network have with how many nodes on each hidden layer?</p>
  
  <p>Maybe the problem is that the data have been prepared to be used in Linear regression and I using it with Backpropagation.</p>
  
  <p>I have only created the neural network, I haven't implemented the backpropagation algorithm because it is already implemented in JavaNNS.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-19T15:21:15.967Z
Comments.Text: Can you please tell us which gradient descent algorithm you use and which learning rate?
Comments.UserDisplayName: Molnár István
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-19T15:58:54.320Z
Comments.Text: @MolnárIstván The learning rate is 2.0 but I don't know which gradient descent algorithm is JavaNNS is using.
Comments.UserDisplayName: VansFannel
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-19T16:19:20.927Z
Comments.Text: The data has been prepared in Dataquest tutorial to be used with Linear Regression. Can I use it with backpropagation?
Comments.UserDisplayName: VansFannel
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-19T18:34:01.973Z
Comments.Text: Well, I pretty sure that 2.0 for learning rate is way too high. Try 1e-2, 1e-3 or 1e-4 (it differs which gradient descent algorithm you use) . Besides, I think you have some misunderstanding about backpropagation. It is for adjusting the weights in your network through the loss. Well of course the prepocessing can be bad in a lot of cases, but here I think it should work.
Comments.UserDisplayName: Molnár István
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-21T09:46:40.970Z
Comments.Text: @MolnárIstván I have talke with my professor and he said me that there isn't enough data to train the network. With 12 inputs and only 718 cases, it only covers the 17% of the combinations. I don't understand why I have to use JavaNNS. The train data is part of a Kaggle competition and it works with linear regression.
Comments.UserDisplayName: VansFannel
Tags: <neural-networks><backpropagation><hidden-layers>
Title: Data prepared to linear regression. Can I use it with backpropagation?
UserDisplayName: VansFannel
ThreadText: # Data prepared to linear regression. Can I use it with backpropagation?
  
  (Asked by: username_0 on 2018-02-17)
  
  I'm studying a Master's Degree in Artificial Intelligence and I need to learn how to use the [Java Neural Network Simulator](http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/welcome_e.html), JavaNNS, program.
  
  In one practice I have to build a neural network to use backpropagation on it.
  
  I have created a neural network with one input layer with 12 nodes, one hidden layer with 6 nodes and one output layer with 1 node.
  
  I'm using Kaggle's titanic competition data with this format following Dataquest course [Getting Started with Kaggle for Titanic competition](https://www.dataquest.io/course/kaggle-fundamentals):
  
  ```
  Pclass_1,Pclass_2,Pclass_3,Sex_female,Sex_male,Age_categories_Missing,Age_categories_Infant,Age_categories_Child,Age_categories_Teenager,Age_categories_Young Adult,Age_categories_Adult,Age_categories_Senior,Survived
  0,0,1,0,1,0,0,0,0,1,0,0,0
  1,0,0,1,0,0,0,0,0,0,1,0,1
  0,0,1,1,0,0,0,0,0,1,0,0,1
  1,0,0,1,0,0,0,0,0,1,0,0,1
  0,0,1,0,1,0,0,0,0,1,0,0,0
  0,0,1,0,1,1,0,0,0,0,0,0,0
  1,0,0,0,1,0,0,0,0,0,1,0,0
  0,0,1,0,1,0,1,0,0,0,0,0,0
  
  ```
  
  If you want see the same data better in an Spreadsheet:
  [![enter image description here](https://i.sstatic.net/Ab2aX.png)](https://i.sstatic.net/Ab2aX.png)
  
  But they preprocess the data to use it with linear regression and I don't know if I can use these data with backpropagation
  
  I think something is wrong because when I run backpropagation in JavaNNS I get these data:
  
  ```
  opened at: Sat Feb 17 17:29:40 CET 2018
  Step 200 MSE:   0.5381023044692738  validation: 0.11675894327003862
  Step 400 MSE:   0.5372328944712378  validation: 0.11700781497209432
  Step 600 MSE:   0.5370386219557437  validation: 0.11691717861750939
  Step 800 MSE:   0.5370348711919518  validation: 0.11696104763606407
  Step 1000 MSE:  0.5369724294992798  validation: 0.11697568840154722
  Step 1200 MSE:  0.5369697016710676  validation: 0.11665485957481342
  Step 1400 MSE:  0.5370053339270906  validation: 0.11684215268609244
  Step 1600 MSE:  0.5370121961199371  validation: 0.11670833992558485
  Step 1800 MSE:  0.5370200812483633  validation: 0.11673550099633925
  Step 2000 MSE:  0.5367923502149529  validation: 0.11675956129361797
  
  ```
  
  Nothing changes, it is like it doesn't learn anything.
  
  How many hidden layers does the network have with how many nodes on each hidden layer?
  
  Maybe the problem is that the data have been prepared to be used in Linear regression and I using it with Backpropagation.
  
  I have only created the neural network, I haven't implemented the backpropagation algorithm because it is already implemented in JavaNNS.
  
  **Tags:** neural-networks, backpropagation, hidden-layers
  
  > **Comments:**
  > **username_1**: Can you please tell us which gradient descent algorithm you use and which learning rate?  
  > **username_0**: @username_1 The learning rate is 2.0 but I don't know which gradient descent algorithm is JavaNNS is using.  
  > **username_0**: The data has been prepared in Dataquest tutorial to be used with Linear Regression. Can I use it with backpropagation?  
  > **username_1**: Well, I pretty sure that 2.0 for learning rate is way too high. Try 1e-2, 1e-3 or 1e-4 (it differs which gradient descent algorithm you use) . Besides, I think you have some misunderstanding about backpropagation. It is for adjusting the weights in your network through the loss. Well of course the prepocessing can be bad in a lot of cases, but here I think it should work.  
  > **username_0**: @username_1 I have talke with my professor and he said me that there isn't enough data to train the network. With 12 inputs and only 718 cases, it only covers the 17% of the combinations. I don't understand why I have to use JavaNNS. The train data is part of a Kaggle competition and it works with linear regression.  
  
  ---
  
  ## 0 answers
  
  