Answers.Body: <p>I'm not completely sure I understand your simulation. What I think you are doing is:</p>
  
  <ol>
  <li>Generate 5 angles specified in radians (Are these always normalized to within (0, <span class="math-container">$2*\pi$</span>)?).</li>
  <li>Interpret each angle as a <strong>unit</strong> vector in a 2d space.</li>
  <li>Add the unit vectors together, yielding a vector that lies somewhere inside a circle with radius 5.</li>
  <li>Ask whether the summed vector is more or less than a distance <span class="math-container">$y$</span> from the origin of the circle.</li>
  </ol>
  
  <p>If you're doing that, your problem looks like trying to learn a separation of two concentric rings, which is a well known benchmarking problem for classification. </p>
  
  <p>I am reasonably certain you can learn this pattern with several layers of ReLU neurons. I'm not certain that convolutional layers will help you much here. The main patterns I'd expect the network to learn are:</p>
  
  <ul>
  <li>perhaps 2-3 layers to learn whether the point lies far away from the origin in each of several different directions.</li>
  <li>1 layer to learn where the decision boundary is in each of 4 directions away from the origin.</li>
  <li>1 layer to inclusive-OR the 4 decision boundaries together.</li>
  </ul>
  
  <p>My guess is that this is fairly easy to learn with 4 layers of 8 ReLU neurons, or something like it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-09T00:47:17.180Z
Answers.Text: You got the question exactly right. I was not thinking about a unit vector but scaling the unit vector for my custom steps instead, anyway the procedure stays the same. It is very inspiring for me to see you actually taking a guess about the meaning of each layer! Also, very interesting idea of ring separation, a well known problem, which is what I hoped I would find!
Answers.UserDisplayName: Genoma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-08T23:17:00.987Z
Answers.LastActivityDate: 2019-10-08T23:17:00.987Z
Answers.UserDisplayName: John Doucette
Body: <p>I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively the probability of failure or success. The NN is trained in a simulation.</p>
  <p>The simulation goes this way: it takes 5 angles in radians and calculates the vector sum of 5 vectors having <span class="math-container">$x$</span> as module and <span class="math-container">$\alpha$</span> as angles (taken from the input). It returns <span class="math-container">$1$</span> if the vector sum has a module greater than <span class="math-container">$y$</span>, or <span class="math-container">$0$</span> if it is less than <span class="math-container">$y$</span>.</p>
  <p>My intention is to be able to tell sequences of radians that will generate vectors with a sum greater than <span class="math-container">$y$</span> in module from the ones which won't.</p>
  <p>Which would be the best configuration to achieve this? Is the configuration I set up (1D convolution layer + 2 linear layers) efficient? If so, would it be easy to find the right size for the convolution? Or should I just remove it?</p>
  <p>I noticed that if I change the order of the input angles the output of the simulation will be the same. Is there a particular configuration you should use when dealing with these cases?</p>
  
Tags: <neural-networks><convolutional-neural-networks><ai-design>
Title: An architecture for classifying distance from origin for a sum of vectors?
UserDisplayName: Genoma
ThreadText: # An architecture for classifying distance from origin for a sum of vectors?
  
  (Asked by: username_0 on 2019-10-08)
  
  I built a three-layer neural network (first is 1D convolutional and the remaining two are linear). It takes an input of 5 angles in radians, and outputs two numbers from 0 to 1, which are respectively the probability of failure or success. The NN is trained in a simulation.
  
  The simulation goes this way: it takes 5 angles in radians and calculates the vector sum of 5 vectors having $x$ as module and $\alpha$ as angles (taken from the input). It returns $1$ if the vector sum has a module greater than $y$, or $0$ if it is less than $y$.
  
  My intention is to be able to tell sequences of radians that will generate vectors with a sum greater than $y$ in module from the ones which won't.
  
  Which would be the best configuration to achieve this? Is the configuration I set up (1D convolution layer + 2 linear layers) efficient? If so, would it be easy to find the right size for the convolution? Or should I just remove it?
  
  I noticed that if I change the order of the input angles the output of the simulation will be the same. Is there a particular configuration you should use when dealing with these cases?
  
  **Tags:** neural-networks, convolutional-neural-networks, ai-design
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-08. Score: 2
  
  I'm not completely sure I understand your simulation. What I think you are doing is:
  
  1. Generate 5 angles specified in radians (Are these always normalized to within (0, $2\*\pi$)?).
  2. Interpret each angle as a **unit** vector in a 2d space.
  3. Add the unit vectors together, yielding a vector that lies somewhere inside a circle with radius 5.
  4. Ask whether the summed vector is more or less than a distance $y$ from the origin of the circle.
  
  If you're doing that, your problem looks like trying to learn a separation of two concentric rings, which is a well known benchmarking problem for classification.
  
  I am reasonably certain you can learn this pattern with several layers of ReLU neurons. I'm not certain that convolutional layers will help you much here. The main patterns I'd expect the network to learn are:
  
  * perhaps 2-3 layers to learn whether the point lies far away from the origin in each of several different directions.
  * 1 layer to learn where the decision boundary is in each of 4 directions away from the origin.
  * 1 layer to inclusive-OR the 4 decision boundaries together.
  
  My guess is that this is fairly easy to learn with 4 layers of 8 ReLU neurons, or something like it.
  
  > **Comments:**
  > **username_0**: You got the question exactly right. I was not thinking about a unit vector but scaling the unit vector for my custom steps instead, anyway the procedure stays the same. It is very inspiring for me to see you actually taking a guess about the meaning of each layer! Also, very interesting idea of ring separation, a well known problem, which is what I hoped I would find!  
  
  ---
  
  