Answers.Body: <p>The labels are not unique for the input domain <code>[0,20]</code>. Think about <code>sin(x)=0</code>, <code>x</code> could be <code>0</code>, <code>pi</code>, <code>2*pi</code>, <code>3*pi</code>, ..., <code>n*pi</code>, all are correct from a mathematical point of view, but this is not reflected in your MSE loss. At this point your NN has to guess the correct label from your input data. Predicting the mean of your input data is the safest bet for the network.</p>
  <p>In essence you're trying to build a arcsin function with a NN. If only consider x values in <code>[-0.5*pi,0.5*pi]</code>, the labels are unique and your network should work.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-14T10:10:23.320Z
Answers.LastActivityDate: 2020-12-14T10:10:23.320Z
Answers.UserDisplayName: Tinu
Body: <p>I made a simple feedforward neural network (FFNN) to predict <span class="math-container">$x$</span> from <span class="math-container">$\sin(x)$</span>. It failed. Does it mean the model has overfitted? Why doesn't it work?</p>
  <p><a href="https://i.sstatic.net/XKBbB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XKBbB.png" alt="enter image description here" /></a></p>
  <pre><code>set.seed(1234567890)
  Var3 &lt;- runif(500, 0, 20)
  mydata3 &lt;- data.frame(Sin=sin(Var3),Var=Var3)
  set.seed(1234567890)
  winit &lt;- runif(5500, -1, 1)
  #hidUnit &lt;- c(9,1)
  set.seed(1234567890)
  nn3 &lt;-neuralnet(formula = Var~Sin,data = mydata3,
                  hidden =c(4,2,1),startweights =winit,
                learningrate = 0.01,act.fct = &quot;tanh&quot;)
  
  plot(mydata3, cex=2,main='Predicting x from Sin(x)',
       pch = 21,bg=&quot;darkgrey&quot;,
       ylab=&quot;X&quot;,xlab=&quot;Sin(X)&quot;)
  points(mydata3[,1],predict(nn3,mydata3), col=&quot;darkred&quot;, 
         cex=1,pch=21,bg=&quot;red&quot;)
  
  legend(&quot;bottomleft&quot;, legend=c(&quot;true&quot;,&quot;predicted&quot;), pch=c(21,21),
         col = c(&quot;darkgrey&quot;,&quot;red&quot;),cex = 0.65,bty = &quot;n&quot;)
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-13T16:01:46.300Z
Comments.Text: Im not familiar with r code. you are just feeding the x value into the network and adding tanh function to the hidden layers in the network am i right ? Christopher Bishop's machine learning and pattern recognition textbook has an example of sin(x) function I believe. However, he tries to fit sin(x) with a polynomial function.
Comments.UserDisplayName: calveeen
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-13T23:49:27.290Z
Comments.Text: it's the other way around! feeding the network sin x and try to predict x.
Comments.UserDisplayName: homa taha
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-14T00:54:03.553Z
Comments.Text: Any loss function metrics that you can report ? Eg. training loss
Comments.UserDisplayName: calveeen
Tags: <r>
Title: Why does my neural network to predict $x$ given $\sin(x)$ not generalize?
UserDisplayName: homa taha
ThreadText: # Why does my neural network to predict $x$ given $\sin(x)$ not generalize?
  
  (Asked by: username_0 on 2020-12-13)
  
  I made a simple feedforward neural network (FFNN) to predict $x$ from $\sin(x)$. It failed. Does it mean the model has overfitted? Why doesn't it work?
  
  set.seed(1234567890)
  Var3 <- runif(500, 0, 20)
  mydata3 <- data.frame(Sin=sin(Var3),Var=Var3)
  set.seed(1234567890)
  winit <- runif(5500, -1, 1)
  #hidUnit <- c(9,1)
  set.seed(1234567890)
  nn3 <-neuralnet(formula = Var~Sin,data = mydata3,
                  hidden =c(4,2,1),startweights =winit,
                learningrate = 0.01,act.fct = "tanh")
  
  plot(mydata3, cex=2,main='Predicting x from Sin(x)',
       pch = 21,bg="darkgrey",
       ylab="X",xlab="Sin(X)")
  points(mydata3[,1],predict(nn3,mydata3), col="darkred", 
         cex=1,pch=21,bg="red")
  
  legend("bottomleft", legend=c("true","predicted"), pch=c(21,21),
         col = c("darkgrey","red"),cex = 0.65,bty = "n")
  
  **Tags:** r
  > **Comments:**
  > **username_1**: Im not familiar with r code. you are just feeding the x value into the network and adding tanh function to the hidden layers in the network am i right ? Christopher Bishop's machine learning and pattern recognition textbook has an example of sin(x) function I believe. However, he tries to fit sin(x) with a polynomial function.  
  > **username_0**: it's the other way around! feeding the network sin x and try to predict x.  
  > **username_1**: Any loss function metrics that you can report ? Eg. training loss  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-12-14. Score: 1 | Accepted answer
  
  The labels are not unique for the input domain `[0,20]`. Think about `sin(x)=0`, `x` could be `0`, `pi`, `2*pi`, `3*pi`, ..., `n*pi`, all are correct from a mathematical point of view, but this is not reflected in your MSE loss. At this point your NN has to guess the correct label from your input data. Predicting the mean of your input data is the safest bet for the network.
  
  In essence you're trying to build a arcsin function with a NN. If only consider x values in `[-0.5*pi,0.5*pi]`, the labels are unique and your network should work.
  
  
  ---
  
  