Answers.Body: <p>@BlindKungFuMaster answered my question in comments, but his answer doesn't reflect that information. If he changes his answer to contain the actual answer, I'll remove mine. </p>
  
  <p>The answer to the question "How can object types be differentiated in the input of a neural network?" is:</p>
  
  <p><strong>Use different sensors (nerves) to detect different types of input.</strong> </p>
  
  <p>So for example:</p>
  
  <ul>
  <li><p>If you want the network to be capable of differentiating two different types of objects in a simple 2D environment, use two different sets of nerves, one to detect one object type, one to detect the other. So if you want to sense 10 points around your "organism", have 20 nerves, 10 neurons in your input layer dealing with one object type, 10 dealing with the other. </p></li>
  <li><p>That's the most simple example, dealing with binary differences (object type 1 or object type 2), but it could be less binary, like this: Let's say there are three object types and think of each object types as 1/3 of a color value. So you have three sets of 10 nerves in total transmitting to your input layer (30 neurons) and each nerve set senses either R, G, or B. If you set up your system so that there are three "objects" to collide with, stacked on top of each other (like a single object), your neural network will be capable of handling objects differently based on their RGB color value, meaning it can now handle nearly infinite "types" of objects. </p></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-25T15:20:31.780Z
Answers.LastActivityDate: 2017-05-25T15:20:31.780Z
Answers.UserDisplayName: J.Todd
Answers.Body: <p>Neural networks learn. That's what they are for. For your task there are two sensible scenarios: </p>
  
  <ol>
  <li><p>You have a fixed reaction for danger and a fixed reaction for food and you only have to learn how to distinguish between them. In that case you basically try to classify the situation to trigger the right fixed response and this classification would be learned by <a href="http://neuralnetworksanddeeplearning.com" rel="nofollow noreferrer">backpropagation</a>.</p></li>
  <li><p>You directly learn to act for a given situation. In that case you can either use a genetic algorithm or you use reinforcement learning with backpropagation.</p></li>
  </ol>
  
  <p>I would recommend using a genetic algorithm, because it is significantly easier and also makes sense in this situation. You would randomly initialise your network, let it run around in the environment and remember how much food it ate and how often or how quickly it died. Then you would randomly change the weights of your network and do the same thing again. If it did better this time around you would proceed to use the new weights otherwise you go back to the old weights and try a different random change. 
  By selecting successful random changes it would over time learn to avoid danger and seek out food.</p>
  
  <p>Edit: To my mind you have a fundamental misunderstanding how perception works. If you see a lion and a cake, do those trigger different kinds of cells on your retina? No! All nerve cells are used to detect all kinds of objects! The classification, i.e. whether you are seeing a lion or a cake happens in the neural network i.e. in the higher regions of your visual cortex, far removed from the initial nerve activation. Your lion might be yellow and your cake might be yellow, only if you analyse the high level structure of your nerve inputs can you decide what you are seeing. That is the task of a neural network. And that high level structure analysis is what a neural network learns. </p>
  
  <p>What seems to confuse you is the example you linked. In that example this very sparse distance measuring is enough to differentiate between walls and boosters in your high level structure analysis, because the different points of the walls that you sample have a certain relative position that you can analyse and conclude that they constitute the wall. </p>
  
  <p>In your scenario very sparse distance measuring will not help you obviously. The distance of an object doesn't tell you whether it's a lion or a cake. Distance and color would be a solution to that. Or, more realistically, you have different shapes and much tighter distance sampling, and high level analysis can work out the shape from a couple of closely measured distances. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-03T17:14:31.493Z
Answers.Text: @J.Todd The same principle applies whether it's a red circle and a blue circle, or a lion and a cake, just more neurons and more training data might be needed.
Answers.UserDisplayName: Criticize SE actions means ban
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T13:13:57.007Z
Answers.Text: I'm using genetic evolution for the learning, but Im having trouble understanding your answer as to how the network actually differentiates the two types of objects. Lets say instead of walls and boosters we just have circles, all the same size. Some circles are good, some are bad. They can be in different positions at any given time, they move randomly (a randomly dynamic environment).
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T13:13:59.327Z
Answers.Text: (cont) I'm struggling to understand how you're suggesting the network would learn to differentiate the two types of objects, without, as I thought you were originally saying, having two separate sets of inputs and nerves each capable of sensing one object type or the other.
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T13:43:04.647Z
Answers.Text: It can only learn the difference if there is a difference in the input to the nerves. If the good circles are green and the bad circles are red and color vision is part of the input, that'll be enough. If the bad circles always move in groups and the good circles are always alone and it input vision is wide enough to see this difference, that would be enough too. This really has nothing to do with having one network or two.
Answers.UserDisplayName: BlindKungFuMaster
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T14:03:33.627Z
Answers.Text: I was never suggesting two networks, I was talking about twice the inputs, one set connected to nerves which collide with good objects, one set which collide with bad. With what you just said "If the good circles are green and the bad circles are red and color vision is part of the input", you surely understand only values between a range of -1 to 1 can be input to each input neuron in a neural network, so how would that "color vision" (the thing Im trying to do basically) work fundamentally? How do you input that to the neural system?
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T14:10:37.000Z
Answers.Text: What Im getting at is that each neuron essentially gets 1 dimensional input (range between two values) but 2D input would be needed to input both collision and color through each input layer neuron.
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-19T08:16:50.977Z
Answers.Text: @BlindKungFuMaster This question should be migrated to chat.It seems the OP is curious!
Answers.UserDisplayName: quintumnia
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-25T11:53:57.967Z
Answers.Text: @Viziionary: Color vision in a simulated environment can be based on RGB values of the pixels on screen, so every pixel gets three nerves. The full range of all nerves will be 2D, every single nerve will receive just one data point. If one set of nerves only touches bad objects and the other set of nerves only touches good objects, than you already have to know which objects are good and which are bad. Then most of what the NN should learn is already done.
Answers.UserDisplayName: BlindKungFuMaster
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-25T14:43:47.903Z
Answers.Text: @BlindKungFuMaster See that's what I thought you were saying (from your comment earlier). But man, what's up with your answer? Your answer is just an explanation of genetic based evolutionary development of a neural network. Your comments answer my question but your answer doesn't seem to contain any of the actual answer to the question, which was how to differentiate sensory input of different objects types (which thanks to your comments, I understand is to have different input neurons sense different types of environmental information).
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-26T12:50:46.393Z
Answers.Text: I see why I'm confused by what you're trying to use as the answer now: You're referencing high level cognitive skills (recognizing the difference between a cake and a lion) which is a much more complex neural system than what I'm talking about. My question was in regards to a very low level, fundamental system. To my knowledge, simulating and developing a neural network capable of learning to know the difference between a cake and a lion in 3D space and react differently to each requires incredibly large amounts of computation. I was asking about a simple 2D fundamental scenario with circles.
Answers.UserDisplayName: J.Todd
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-18T07:00:38.150Z
Answers.LastActivityDate: 2017-05-26T07:09:30.393Z
Answers.LastEditDate: 2017-05-26T07:09:30.393Z
Answers.UserDisplayName: BlindKungFuMaster
Body: <p>In a neural network when inputting nerve input to sense a 2D environment, how do you differentiate two types of objects (with similar shape and size) so the neural network can treat them differently?</p>
  
  <p>Each neuron in the input layer of a neural network essentially gets 1 dimensional input (range between two values) but 2 dimensional input would be needed to send both collision and category/type information through each input layer neuron. How do you get around that?</p>
  
  <p>Note: After having confusion regarding the scenario / situation I'm asking about compared to other more complex scenarios, and the long comment series that ensued, I'm realizing one challenge of this site is that it's much more complicated and diverse subject matter than code, or the various other topics of Stack Exchange where the problems can be very clearly and simply expressed. Here it's more challenging to express your question and scenario clearly to avoid confusion. </p>
  
  <p>Also there's probably a higher skill gap between an AI learner / enthusiast, and an expert AI specialist, compared to other fields, so that could potentially lead to even more difficulty communicating the answer / question in ways everyone can understand without confusion. Challenging SE site to ask good questions on!</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T09:45:46.557Z
Comments.Text: Hello, welcome to AI.SE. Please note that implementation questions are off-topic here.
Comments.UserDisplayName: Mithical
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T09:49:09.893Z
Comments.Text: @Mithrandir this is a fundamental question, if this cant be asked this site is useless to anyone interested in AI.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T09:50:41.073Z
Comments.Text: No, it just has a different scope than you were expecting.
Comments.UserDisplayName: Mithical
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T10:01:02.740Z
Comments.Text: @Mithrandir it's such a narrow scope that you'll never have more than 100 users here. The scope defined here is almost the same as WorldBuilding SE except only AI can be asked about.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T11:28:54.067Z
Comments.Text: Isn't your input different for different objects? Maybe you should give a bit more details about your input und the task you are trying to accomplish.
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T11:42:45.443Z
Comments.Text: @BlindKungFuMaster I started off giving more details but Mith said it was off topic so I over generalized it to stress that its a fundamental question.It's just a basic feed forward neural network with 20 neurons in the input layer each connecting to a nerve, each nerve radiating from the entity being controlled.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T14:01:33.890Z
Comments.Text: And what is supposed to be the output of the neural network?
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T14:49:53.910Z
Comments.Text: @Mithrandir This is an on-topic question since it's not asking about concrete implementations but how to differentiate (in the general case) two objects in a two-dimensional environment by a neural network. If this question is not on-topic, this website is completely useless and if possible I will vote for it to be closed.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T15:00:01.963Z
Comments.Text: @Viziionary I agree this question should be on-topic on this website, on the other hand, even though I'm currently not an expert in neural networks, I'm not sure if there's just one way of doing what you want to do, so it's possible you may require to narrow your domain so that a more concrete and thus useful answer may be provided to you ;)
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T15:28:54.183Z
Comments.Text: @nbro I think BlindKungFuMaster has a point about needing two sets of input neurons to handle different object types - and that may indeed be the concrete way of doing it in general.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-17T15:31:37.317Z
Comments.Text: @BlindKungFuMaster to answer your earlier question, the output is motion on the x and y axis. We're talking about your generic "hello world" feed forward 2D environment, eat food, avoid obstacles neural network.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-18T06:47:57.793Z
Comments.Text: I never said that you need two sets of input neurons. The input will be different for different objects, because they are different objects. If the inputs are the same for different objects you should change what your creature perceives.
Comments.UserDisplayName: BlindKungFuMaster
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-18T06:57:19.743Z
Comments.Text: @BlindKungFuMaster I'm talking about a scenario [like this](https://www.youtube.com/watch?v=5lJuEW-5vr8) where you have X number of "nerves" radiating from your entity being controlled. As it is in the example I just showed, those nerves can't fundamentally know whether they're touching a wall (which the network learns to avoid) or a "booster" for example (which we want the network to learn to be attracted to). So from what you were saying, it seemed to me to do this, we need to create a second set of nerves and a second set of input neurons, and have the second set of nerves only detect good.
Comments.UserDisplayName: J.Todd
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-05-18T07:05:03.963Z
Comments.Text: In your example a single nerve cannot know whether it is touching a booster or a wall, but the full input of all nerves makes it able to derive which nerves are touching a wall and which a booster.
Comments.UserDisplayName: BlindKungFuMaster
Tags: <neural-networks>
Title: How can object types be differentiated in the input of a neural network?
UserDisplayName: J.Todd
ThreadText: # How can object types be differentiated in the input of a neural network?
  
  (Asked by: username_0 on 2017-05-17)
  
  In a neural network when inputting nerve input to sense a 2D environment, how do you differentiate two types of objects (with similar shape and size) so the neural network can treat them differently?
  
  Each neuron in the input layer of a neural network essentially gets 1 dimensional input (range between two values) but 2 dimensional input would be needed to send both collision and category/type information through each input layer neuron. How do you get around that?
  
  Note: After having confusion regarding the scenario / situation I'm asking about compared to other more complex scenarios, and the long comment series that ensued, I'm realizing one challenge of this site is that it's much more complicated and diverse subject matter than code, or the various other topics of Stack Exchange where the problems can be very clearly and simply expressed. Here it's more challenging to express your question and scenario clearly to avoid confusion.
  
  Also there's probably a higher skill gap between an AI learner / enthusiast, and an expert AI specialist, compared to other fields, so that could potentially lead to even more difficulty communicating the answer / question in ways everyone can understand without confusion. Challenging SE site to ask good questions on!
  
  **Tags:** neural-networks
  
  > **Comments:**
  > **username_1**: Hello, welcome to AI.SE. Please note that implementation questions are off-topic here.  
  > **username_0**: @Mithrandir this is a fundamental question, if this cant be asked this site is useless to anyone interested in AI.  
  > **username_1**: No, it just has a different scope than you were expecting.  
  > **username_0**: @Mithrandir it's such a narrow scope that you'll never have more than 100 users here. The scope defined here is almost the same as WorldBuilding SE except only AI can be asked about.  
  > **username_2**: Isn't your input different for different objects? Maybe you should give a bit more details about your input und the task you are trying to accomplish.  
  > **username_0**: @username_2 I started off giving more details but Mith said it was off topic so I over generalized it to stress that its a fundamental question.It's just a basic feed forward neural network with 20 neurons in the input layer each connecting to a nerve, each nerve radiating from the entity being controlled.  
  > **username_2**: And what is supposed to be the output of the neural network?  
  > **username_3**: @Mithrandir This is an on-topic question since it's not asking about concrete implementations but how to differentiate (in the general case) two objects in a two-dimensional environment by a neural network. If this question is not on-topic, this website is completely useless and if possible I will vote for it to be closed.  
  > **username_3**: @Viziionary I agree this question should be on-topic on this website, on the other hand, even though I'm currently not an expert in neural networks, I'm not sure if there's just one way of doing what you want to do, so it's possible you may require to narrow your domain so that a more concrete and thus useful answer may be provided to you ;)  
  > **username_0**: @username_3 I think username_2 has a point about needing two sets of input neurons to handle different object types - and that may indeed be the concrete way of doing it in general.  
  > **username_0**: @username_2 to answer your earlier question, the output is motion on the x and y axis. We're talking about your generic "hello world" feed forward 2D environment, eat food, avoid obstacles neural network.  
  > **username_2**: I never said that you need two sets of input neurons. The input will be different for different objects, because they are different objects. If the inputs are the same for different objects you should change what your creature perceives.  
  > **username_0**: @username_2 I'm talking about a scenario [like this](https://www.youtube.com/watch?v=5lJuEW-5vr8) where you have X number of "nerves" radiating from your entity being controlled. As it is in the example I just showed, those nerves can't fundamentally know whether they're touching a wall (which the network learns to avoid) or a "booster" for example (which we want the network to learn to be attracted to). So from what you were saying, it seemed to me to do this, we need to create a second set of nerves and a second set of input neurons, and have the second set of nerves only detect good.  
  > **username_2**: In your example a single nerve cannot know whether it is touching a booster or a wall, but the full input of all nerves makes it able to derive which nerves are touching a wall and which a booster.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2017-05-25. Score: 0 | Accepted answer
  
  @username_2 answered my question in comments, but his answer doesn't reflect that information. If he changes his answer to contain the actual answer, I'll remove mine.
  
  The answer to the question "How can object types be differentiated in the input of a neural network?" is:
  
  **Use different sensors (nerves) to detect different types of input.**
  
  So for example:
  
  * If you want the network to be capable of differentiating two different types of objects in a simple 2D environment, use two different sets of nerves, one to detect one object type, one to detect the other. So if you want to sense 10 points around your "organism", have 20 nerves, 10 neurons in your input layer dealing with one object type, 10 dealing with the other.
  * That's the most simple example, dealing with binary differences (object type 1 or object type 2), but it could be less binary, like this: Let's say there are three object types and think of each object types as 1/3 of a color value. So you have three sets of 10 nerves in total transmitting to your input layer (30 neurons) and each nerve set senses either R, G, or B. If you set up your system so that there are three "objects" to collide with, stacked on top of each other (like a single object), your neural network will be capable of handling objects differently based on their RGB color value, meaning it can now handle nearly infinite "types" of objects.
  
  
  ---
  
  ### Answer by username_2 on 2017-05-18. Score: 2
  
  Neural networks learn. That's what they are for. For your task there are two sensible scenarios:
  
  1. You have a fixed reaction for danger and a fixed reaction for food and you only have to learn how to distinguish between them. In that case you basically try to classify the situation to trigger the right fixed response and this classification would be learned by [backpropagation](http://neuralnetworksanddeeplearning.com).
  2. You directly learn to act for a given situation. In that case you can either use a genetic algorithm or you use reinforcement learning with backpropagation.
  
  I would recommend using a genetic algorithm, because it is significantly easier and also makes sense in this situation. You would randomly initialise your network, let it run around in the environment and remember how much food it ate and how often or how quickly it died. Then you would randomly change the weights of your network and do the same thing again. If it did better this time around you would proceed to use the new weights otherwise you go back to the old weights and try a different random change.
  By selecting successful random changes it would over time learn to avoid danger and seek out food.
  
  Edit: To my mind you have a fundamental misunderstanding how perception works. If you see a lion and a cake, do those trigger different kinds of cells on your retina? No! All nerve cells are used to detect all kinds of objects! The classification, i.e. whether you are seeing a lion or a cake happens in the neural network i.e. in the higher regions of your visual cortex, far removed from the initial nerve activation. Your lion might be yellow and your cake might be yellow, only if you analyse the high level structure of your nerve inputs can you decide what you are seeing. That is the task of a neural network. And that high level structure analysis is what a neural network learns.
  
  What seems to confuse you is the example you linked. In that example this very sparse distance measuring is enough to differentiate between walls and boosters in your high level structure analysis, because the different points of the walls that you sample have a certain relative position that you can analyse and conclude that they constitute the wall.
  
  In your scenario very sparse distance measuring will not help you obviously. The distance of an object doesn't tell you whether it's a lion or a cake. Distance and color would be a solution to that. Or, more realistically, you have different shapes and much tighter distance sampling, and high level analysis can work out the shape from a couple of closely measured distances.
  
  > **Comments:**
  > **username_0**: I'm using genetic evolution for the learning, but Im having trouble understanding your answer as to how the network actually differentiates the two types of objects. Lets say instead of walls and boosters we just have circles, all the same size. Some circles are good, some are bad. They can be in different positions at any given time, they move randomly (a randomly dynamic environment).  
  > **username_0**: (cont) I'm struggling to understand how you're suggesting the network would learn to differentiate the two types of objects, without, as I thought you were originally saying, having two separate sets of inputs and nerves each capable of sensing one object type or the other.  
  > **username_2**: It can only learn the difference if there is a difference in the input to the nerves. If the good circles are green and the bad circles are red and color vision is part of the input, that'll be enough. If the bad circles always move in groups and the good circles are always alone and it input vision is wide enough to see this difference, that would be enough too. This really has nothing to do with having one network or two.  
  > **username_0**: I was never suggesting two networks, I was talking about twice the inputs, one set connected to nerves which collide with good objects, one set which collide with bad. With what you just said "If the good circles are green and the bad circles are red and color vision is part of the input", you surely understand only values between a range of -1 to 1 can be input to each input neuron in a neural network, so how would that "color vision" (the thing Im trying to do basically) work fundamentally? How do you input that to the neural system?  
  > **username_0**: What Im getting at is that each neuron essentially gets 1 dimensional input (range between two values) but 2D input would be needed to input both collision and color through each input layer neuron.  
  > **username_5**: @username_2 This question should be migrated to chat.It seems the OP is curious!  
  > **username_2**: @Viziionary: Color vision in a simulated environment can be based on RGB values of the pixels on screen, so every pixel gets three nerves. The full range of all nerves will be 2D, every single nerve will receive just one data point. If one set of nerves only touches bad objects and the other set of nerves only touches good objects, than you already have to know which objects are good and which are bad. Then most of what the NN should learn is already done.  
  > **username_0**: @username_2 See that's what I thought you were saying (from your comment earlier). But man, what's up with your answer? Your answer is just an explanation of genetic based evolutionary development of a neural network. Your comments answer my question but your answer doesn't seem to contain any of the actual answer to the question, which was how to differentiate sensory input of different objects types (which thanks to your comments, I understand is to have different input neurons sense different types of environmental information).  
  > **username_0**: I see why I'm confused by what you're trying to use as the answer now: You're referencing high level cognitive skills (recognizing the difference between a cake and a lion) which is a much more complex neural system than what I'm talking about. My question was in regards to a very low level, fundamental system. To my knowledge, simulating and developing a neural network capable of learning to know the difference between a cake and a lion in 3D space and react differently to each requires incredibly large amounts of computation. I was asking about a simple 2D fundamental scenario with circles.  
  > **username_4**: @username_0 The same principle applies whether it's a red circle and a blue circle, or a lion and a cake, just more neurons and more training data might be needed.  
  
  ---
  
  