Answers.Body: <p>You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables <span class="math-container">$(y_i, x_i)$</span> where <span class="math-container">$y_i$</span> is <span class="math-container">$i$</span>-th output, and <span class="math-container">$x_i$</span> is <span class="math-container">$i$</span>-th input then you can transform your input variable into vector</p>
  <p><span class="math-container">\begin{equation}
  \phi = [1, \sin(x), \cos(x), \sin(2x), \cos(2x), \ldots, \sin(nx), \cos(nx)]
  \end{equation}</span>
  where <span class="math-container">$n$</span> is some arbitrary number.</p>
  <p>Then you will have a standard linear regression model
  <span class="math-container">\begin{equation}
  Y = A\Phi
  \end{equation}</span>
  where <span class="math-container">$Y = [y_1, \ldots, y_n]$</span>, <span class="math-container">$\Phi = [ \phi_1, \ldots, \phi_n]$</span> and <span class="math-container">$A$</span> is matrix of unknown parameters that you need to learn.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-07T10:59:13.203Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/120529/discussion-on-answer-by-brale-which-approach-should-i-use-to-classify-points-abo).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-06T11:06:58.377Z
Answers.LastActivityDate: 2021-03-06T11:06:58.377Z
Answers.UserDisplayName: Brale
Body: <p>In a linear regression problem, a line can divide a data set into two categories. So, basically, points above the line belong to category <code>1</code>, and points below the line belong to category <code>-1</code>.</p>
  <p>However, my professor has asked me to write a C++ program in which the program will classify whether the data points lie above or below a <strong>sine</strong> function.</p>
  <p>Let me explain a bit more. So, first, we will generate a data set <span class="math-container">$$D = \{(x_i, y_i) \} \label{0}\tag{0} $$</span> with random <span class="math-container">$x$</span> and <span class="math-container">$y$</span> coordinates, for example, according to this equation</p>
  <p><span class="math-container">$$y(x) = A + B \sin(Cx)\label{1}\tag{1},$$</span></p>
  <p>where <span class="math-container">$A$</span>, <span class="math-container">$B$</span>, and <span class="math-container">$C$</span> are known.</p>
  <p>The data points above the sine function will have a label <code>1</code> on them, and the points below the function will have <code>-1</code>.</p>
  <p>Now, this data set <span class="math-container">$D$</span> in \ref{0} has to be fed to a C++ program. This C++ program has to somehow learn the curve separating the two data point categories. After training, the program will then classify some new query data points.</p>
  <p>The key difficulty is that the program does <strong>not</strong> know in advance that the points were scattered around a <strong>sine</strong> curve. It does <strong>not</strong> know the values of <span class="math-container">$A$</span>, <span class="math-container">$B$</span>, or <span class="math-container">$C$</span> in equation \ref{1}. It also does <strong>not</strong> know that the curve is a <strong>sine</strong> curve.</p>
  <p>Now, this is where I am stuck. I do not know if I need to use a neural network to solve this problem. If a neural network is to be used, then I presume that backpropagation will have to be used in some way. I can generate the data set and I can feed the data into the program.</p>
  <p><strong>Which approach (algorithm and model) should I use to solve this problem?</strong></p>
  <p>I have studied linear classification with the perceptron learning algorithm, but this sine-classifier stuff is a huge step-up for me. Another important thing is that I am <strong>not</strong> allowed to use any ready-made C++ libraries for Machine Learning. If a neural network solution is needed, then I will have to design the neural network from scratch. Note that I don't need any C++ code, but I am just looking for some guidance on how to approach this problem.</p>
  
Tags: <neural-networks><machine-learning><binary-classification><c++><model-request>
Title: Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?
UserDisplayName: Quadro
ThreadText: # Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?
  
  (Asked by: username_0 on 2021-03-06)
  
  In a linear regression problem, a line can divide a data set into two categories. So, basically, points above the line belong to category `1`, and points below the line belong to category `-1`.
  
  However, my professor has asked me to write a C++ program in which the program will classify whether the data points lie above or below a **sine** function.
  
  Let me explain a bit more. So, first, we will generate a data set $$D = \{(x\_i, y\_i) \} \label{0}\tag{0} $$ with random $x$ and $y$ coordinates, for example, according to this equation
  
  $$y(x) = A + B \sin(Cx)\label{1}\tag{1},$$
  
  where $A$, $B$, and $C$ are known.
  
  The data points above the sine function will have a label `1` on them, and the points below the function will have `-1`.
  
  Now, this data set $D$ in \ref{0} has to be fed to a C++ program. This C++ program has to somehow learn the curve separating the two data point categories. After training, the program will then classify some new query data points.
  
  The key difficulty is that the program does **not** know in advance that the points were scattered around a **sine** curve. It does **not** know the values of $A$, $B$, or $C$ in equation \ref{1}. It also does **not** know that the curve is a **sine** curve.
  
  Now, this is where I am stuck. I do not know if I need to use a neural network to solve this problem. If a neural network is to be used, then I presume that backpropagation will have to be used in some way. I can generate the data set and I can feed the data into the program.
  
  **Which approach (algorithm and model) should I use to solve this problem?**
  
  I have studied linear classification with the perceptron learning algorithm, but this sine-classifier stuff is a huge step-up for me. Another important thing is that I am **not** allowed to use any ready-made C++ libraries for Machine Learning. If a neural network solution is needed, then I will have to design the neural network from scratch. Note that I don't need any C++ code, but I am just looking for some guidance on how to approach this problem.
  
  **Tags:** neural-networks, machine-learning, binary-classification, c++, model-request
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-06. Score: 1
  
  You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y\_i, x\_i)$ where $y\_i$ is $i$-th output, and $x\_i$ is $i$-th input then you can transform your input variable into vector
  
  \begin{equation}
  \phi = [1, \sin(x), \cos(x), \sin(2x), \cos(2x), \ldots, \sin(nx), \cos(nx)]
  \end{equation}
  where $n$ is some arbitrary number.
  
  Then you will have a standard linear regression model
  \begin{equation}
  Y = A\Phi
  \end{equation}
  where $Y = [y\_1, \ldots, y\_n]$, $\Phi = [ \phi\_1, \ldots, \phi\_n]$ and $A$ is matrix of unknown parameters that you need to learn.
  
  > **Comments:**
  > **username_2**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/120529/discussion-on-answer-by-username_1-which-approach-should-i-use-to-classify-points-abo).  
  
  ---
  
  