Body: <p>Hello I want to design a AI.The neural network of my AI will consist of 1 input layer of neurons and 1 output layer.</p>
  <p>What is very unique about the neural network is that the # of input neurons will change while the AI is learning.</p>
  <p>For example suppose we 3 snapshots over a learning epoch:</p>
  <p><a href="https://i.sstatic.net/DdHr6KE4.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DdHr6KE4.png" alt="" /></a></p>
  <p>I need to look for the algorithm which would decide when to add or delete a neuron.Any ideas?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T08:15:15.193Z
Comments.Text: Welcome to AI Stack Exchange. It is worth explaining *why* you want this flexible architecture, and the nature of problems you expect it to solve. There _are_ existing learning methods that allow NN architecture to change, but I have not seen any that would change the input layer, since that is usually defined by the problem being solved - so it appears you want to change the data or problem, perhaps as a form of curriculum learning (where you start with a simple problem and use that to bootstrap solutions to more complex ones)?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T09:21:15.320Z
Comments.Text: @NeilSlater imagine a "dumb" AI first booted taking values through sensors and being asked to answer a question.The AI doesnt know which sensors to pay attention to so to start the process , it takes the values of only 1 sensor then it outputs some value.But if the correct output isnt a function of the values of the 1 selected sensor , then it needs to reevaluate its strategy and take into consideration values coming from a different subset from the values set of all sensors.Until the AI is rewarded ,it has to keep searching you need to be able to add/remove neurons from the input.Thanks.
Comments.UserDisplayName: Root Groves
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T13:54:50.067Z
Comments.Text: So you will know what the ful llist of sensors is at any point, but you want the AI to discover the useful ones during learning? Does this *have* to involve starting with one sensor, or could it start equally by using them all and ruling out the ones that don't help with establishing correct answer?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T17:19:30.777Z
Comments.Text: @NeilSlater Yes I want the AI to discover the useful ones during learning
Comments.UserDisplayName: Root Groves
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T18:02:44.543Z
Comments.Text: Is your requirement for this question to build an NN that performs that useful input discovery, (and the details of how it works secondary), or are you only interested in approaches that start with one sensor connected and build up from there, as per your drawings?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T20:49:29.047Z
Comments.Text: The useful input discovery.
Comments.UserDisplayName: Root Groves
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-27T22:39:01.933Z
Comments.Text: Have you looked at the common L1 weight decay regularization algo for MLPs?
Comments.UserDisplayName: cinch
Tags: <neural-networks><machine-learning><deep-learning><dynamic-programming>
Title: Neural network with a variable # of neurons
UserDisplayName: Root Groves
ThreadText: # Neural network with a variable # of neurons
  
  (Asked by: username_0 on 2024-06-27)
  
  Hello I want to design a AI.The neural network of my AI will consist of 1 input layer of neurons and 1 output layer.
  
  What is very unique about the neural network is that the # of input neurons will change while the AI is learning.
  
  For example suppose we 3 snapshots over a learning epoch:
  
  [![](https://i.sstatic.net/DdHr6KE4.png)](https://i.sstatic.net/DdHr6KE4.png)
  
  I need to look for the algorithm which would decide when to add or delete a neuron.Any ideas?
  
  **Tags:** neural-networks, machine-learning, deep-learning, dynamic-programming
  > **Comments:**
  > **username_1**: Welcome to AI Stack Exchange. It is worth explaining *why* you want this flexible architecture, and the nature of problems you expect it to solve. There _are_ existing learning methods that allow NN architecture to change, but I have not seen any that would change the input layer, since that is usually defined by the problem being solved - so it appears you want to change the data or problem, perhaps as a form of curriculum learning (where you start with a simple problem and use that to bootstrap solutions to more complex ones)?  
  > **username_0**: @NeilSlater imagine a "dumb" AI first booted taking values through sensors and being asked to answer a question.The AI doesnt know which sensors to pay attention to so to start the process , it takes the values of only 1 sensor then it outputs some value.But if the correct output isnt a function of the values of the 1 selected sensor , then it needs to reevaluate its strategy and take into consideration values coming from a different subset from the values set of all sensors.Until the AI is rewarded ,it has to keep searching you need to be able to add/remove neurons from the input.Thanks.  
  > **username_1**: So you will know what the ful llist of sensors is at any point, but you want the AI to discover the useful ones during learning? Does this *have* to involve starting with one sensor, or could it start equally by using them all and ruling out the ones that don't help with establishing correct answer?  
  > **username_0**: @NeilSlater Yes I want the AI to discover the useful ones during learning  
  > **username_1**: Is your requirement for this question to build an NN that performs that useful input discovery, (and the details of how it works secondary), or are you only interested in approaches that start with one sensor connected and build up from there, as per your drawings?  
  > **username_0**: The useful input discovery.  
  > **username_2**: Have you looked at the common L1 weight decay regularization algo for MLPs?  
  
  ---
  
  ## 0 answers
  
  