Answers.Body: <p>You can try it with <a href="https://www.tandfonline.com/doi/abs/10.1080/01621459.1985.10478157" rel="nofollow noreferrer">ACE</a> (alternating conditional expectations) - an algorithm that searches for transformations</p>
  
  <p><span class="math-container">$$\theta(y) = f_1(x_1)+f_2(x_2)+...+f_n(x_n)$$</span>
  The functions <span class="math-container">$\theta$</span>, and <span class="math-container">$f_i$</span> are estimated from data. I'll give an example in R here. There is also a <a href="https://github.com/partofthething/ace" rel="nofollow noreferrer">package in Python</a> that does ACE.</p>
  
  <p>Let's generate some data</p>
  
  <pre><code>np &lt;- 100 # number of points
  R &lt;- runif(np, min = 0.9, max = 1.1)
  alpha &lt;- runif(np, min = 0.0, max = 2*pi)
  x &lt;- R*cos(alpha)
  y &lt;- R*sin(alpha)
  par(mfrow=c(1,1))
  plot(x,y)
  </code></pre>
  
  <p><a href="https://i.sstatic.net/4Lz7o.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4Lz7o.png" alt="enter image description here"></a></p>
  
  <p>ACE to estimate <span class="math-container">$\theta(y) = f(x)$</span>:</p>
  
  <pre><code>library(acepack)
  a &lt;- ace(x,y, delrsq = 0.0001)
  </code></pre>
  
  <p>See the transforms, <span class="math-container">$\theta$</span>, and <span class="math-container">$f$</span></p>
  
  <pre><code>par(mfrow=c(1,2))
  plot(a<span class="math-container">$x,a$</span>tx)
  plot(a<span class="math-container">$y,a$</span>ty)
  </code></pre>
  
  <p><a href="https://i.sstatic.net/8bDc3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8bDc3.png" alt="enter image description here"></a></p>
  
  <p>They look like parabolas, so let's fit them.</p>
  
  <pre><code>xx&lt;-drop(a<span class="math-container">$x)
  yy&lt;-drop(a$</span>tx)
  
  plot(xx,yy)
  m.x &lt;- lm(yy ~ xx+I(xx^2))
  xnew=sort(xx)
  lines(xnew, predict(m.x, list(xx=xnew)),col="red",lwd=2)
  
  xx&lt;-drop(a<span class="math-container">$y)
  yy&lt;-drop(a$</span>ty)
  
  plot(xx,yy)
  m.y &lt;- lm(yy ~ xx+I(xx^2))
  m.y$coefficients
  xnew=sort(xx)
  lines(xnew, predict(m.y, list(xx=xnew)),col="red",lwd=2)
  </code></pre>
  
  <p><a href="https://i.sstatic.net/ecDXM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ecDXM.png" alt="enter image description here"></a></p>
  
  <p>The red parabolas don't go through the transforms exactly but we don't need it. We use them only as hints to find the exact relations. We can tweak the parameters later. The parameters of our approximate fits are </p>
  
  <pre><code>m.x$coefficients
  
  (Intercept)          xx     I(xx^2) 
   1.31459869  0.07527254 -2.55098259
  </code></pre>
  
  <p>which means <span class="math-container">$f(x) \approx  1.3-2.56 x^2$</span></p>
  
  <pre><code>m.y$coefficients
   (Intercept)           xx      I(xx^2) 
  -1.342572495  0.001216412  2.791683219
  </code></pre>
  
  <p>which means <span class="math-container">$\theta(y) \approx -1.3 + 2.8 y^2$</span></p>
  
  <p>So, we have 
  <span class="math-container">$$-1.3 + 2.8 y^2 \approx 1.3-2.56 x^2$$</span>
  or
  <span class="math-container">$$2.56 x^2 + 2.8 y^2 \approx 2.6 $$</span></p>
  
  <p>From here you can recover <span class="math-container">$x^2+y^2 = 1$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-20T12:53:16.067Z
Answers.LastActivityDate: 2019-01-20T12:53:16.067Z
Answers.UserDisplayName: Vladislav Gladkikh
Body: <p>I would appreciate your help with this (naive) question of mine.</p>
  
  <p>Given the set of points located on a circle, <span class="math-container">$x_{i}, y_{i}$</span> as the input data, Can a deep/machine learning algorithm infer that radius of the circle is constant ? In other words, given the data <span class="math-container">$x_{i}, y_{i}$</span> is there way that algorithm discovers the constraint: <span class="math-container">$x_{i}^2 + y_{i}^2 = \text{constant}$</span> ? </p>
  
  <p>I would also appreciate any related reference on the subject.    </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T09:40:21.367Z
Comments.Text: No, I am not aiming to discover attractors in phase space, but in general I would like to discover algebraic constraints hidden in the data. The circle example is the simplest one I could think of.
Comments.UserDisplayName: user91411
Tags: <machine-learning><deep-learning><training>
Title: Extracting algebraic constraints from the input data
UserDisplayName: user91411
ThreadText: # Extracting algebraic constraints from the input data
  
  (Asked by: username_0 on 2019-01-13)
  
  I would appreciate your help with this (naive) question of mine.
  
  Given the set of points located on a circle, $x\_{i}, y\_{i}$ as the input data, Can a deep/machine learning algorithm infer that radius of the circle is constant ? In other words, given the data $x\_{i}, y\_{i}$ is there way that algorithm discovers the constraint: $x\_{i}^2 + y\_{i}^2 = \text{constant}$ ?
  
  I would also appreciate any related reference on the subject.
  
  **Tags:** machine-learning, deep-learning, training
  
  > **Comments:**
  > **username_0**: No, I am not aiming to discover attractors in phase space, but in general I would like to discover algebraic constraints hidden in the data. The circle example is the simplest one I could think of.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-01-20. Score: 1
  
  You can try it with [ACE](https://www.tandfonline.com/doi/abs/10.1080/01621459.1985.10478157) (alternating conditional expectations) - an algorithm that searches for transformations
  
  $$\theta(y) = f\_1(x\_1)+f\_2(x\_2)+...+f\_n(x\_n)$$
  The functions $\theta$, and $f\_i$ are estimated from data. I'll give an example in R here. There is also a [package in Python](https://github.com/partofthething/ace) that does ACE.
  
  Let's generate some data
  
  ```
  np <- 100 # number of points
  R <- runif(np, min = 0.9, max = 1.1)
  alpha <- runif(np, min = 0.0, max = 2*pi)
  x <- R*cos(alpha)
  y <- R*sin(alpha)
  par(mfrow=c(1,1))
  plot(x,y)
  
  ```
  
  [![enter image description here](https://i.sstatic.net/4Lz7o.png)](https://i.sstatic.net/4Lz7o.png)
  
  ACE to estimate $\theta(y) = f(x)$:
  
  ```
  library(acepack)
  a <- ace(x,y, delrsq = 0.0001)
  
  ```
  
  See the transforms, $\theta$, and $f$
  
  ```
  par(mfrow=c(1,2))
  plot(a$x,a$tx)
  plot(a$y,a$ty)
  
  ```
  
  [![enter image description here](https://i.sstatic.net/8bDc3.png)](https://i.sstatic.net/8bDc3.png)
  
  They look like parabolas, so let's fit them.
  
  ```
  xx<-drop(a$x)
  yy<-drop(a$tx)
  
  plot(xx,yy)
  m.x <- lm(yy ~ xx+I(xx^2))
  xnew=sort(xx)
  lines(xnew, predict(m.x, list(xx=xnew)),col="red",lwd=2)
  
  xx<-drop(a$y)
  yy<-drop(a$ty)
  
  plot(xx,yy)
  m.y <- lm(yy ~ xx+I(xx^2))
  m.y$coefficients
  xnew=sort(xx)
  lines(xnew, predict(m.y, list(xx=xnew)),col="red",lwd=2)
  
  ```
  
  [![enter image description here](https://i.sstatic.net/ecDXM.png)](https://i.sstatic.net/ecDXM.png)
  
  The red parabolas don't go through the transforms exactly but we don't need it. We use them only as hints to find the exact relations. We can tweak the parameters later. The parameters of our approximate fits are
  
  ```
  m.x$coefficients
  
  (Intercept)          xx     I(xx^2) 
   1.31459869  0.07527254 -2.55098259
  
  ```
  
  which means $f(x) \approx 1.3-2.56 x^2$
  
  ```
  m.y$coefficients
   (Intercept)           xx      I(xx^2) 
  -1.342572495  0.001216412  2.791683219
  
  ```
  
  which means $\theta(y) \approx -1.3 + 2.8 y^2$
  
  So, we have
  $$-1.3 + 2.8 y^2 \approx 1.3-2.56 x^2$$
  or
  $$2.56 x^2 + 2.8 y^2 \approx 2.6 $$
  
  From here you can recover $x^2+y^2 = 1$
  
  
  ---
  
  