Answers.Body: <p>Your proposed model type is the same as a neural network where the weights can be either 0 or 1, with nothing else allowed.</p>
  <p>The main problem with learning this model will be that you cannot take a gradient step on the weights based on a simple loss function. So that may rule out gradient-based methods.</p>
  <p>A genetic algorithm may still be able to search for the optimal connections that minimise a loss. Also local searches where you make some small changes and test, e.g. simulated annealing.</p>
  <p>If your aggregation function is fixed, and not learnable, it's not clear how expressive this model could be. However, with some equivalent to bias, and lots of layers, I suspect in some cases you could show that it was actually equivalent to a neural network. The limitations on training approaches that could work probably mean such a model has niche used at best.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-15T13:28:45.540Z
Answers.LastActivityDate: 2023-08-15T13:28:45.540Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am curious if there is an algorithm that has the structure of a neural network, but it
  does not have weights or biases on the edges and some aggregation functions as nodes. The point of learning in this algorithm would be learning which edges are connected to each node
  meaning the first node of the hidden layer is only connected to feature 1, 4 and 6 of the input layer and so on. I am not worried about the aggregation but more about if there is an algorithm that would learn something like setting those edges/unsetting them.</p>
  
Tags: <machine-learning>
Title: Algorithm for selecting edges in a neural network without weights and biases?
UserDisplayName: Korborb
ThreadText: # Algorithm for selecting edges in a neural network without weights and biases?
  
  (Asked by: username_0 on 2023-08-15)
  
  I am curious if there is an algorithm that has the structure of a neural network, but it
  does not have weights or biases on the edges and some aggregation functions as nodes. The point of learning in this algorithm would be learning which edges are connected to each node
  meaning the first node of the hidden layer is only connected to feature 1, 4 and 6 of the input layer and so on. I am not worried about the aggregation but more about if there is an algorithm that would learn something like setting those edges/unsetting them.
  
  **Tags:** machine-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-08-15. Score: 3 | Accepted answer
  
  Your proposed model type is the same as a neural network where the weights can be either 0 or 1, with nothing else allowed.
  
  The main problem with learning this model will be that you cannot take a gradient step on the weights based on a simple loss function. So that may rule out gradient-based methods.
  
  A genetic algorithm may still be able to search for the optimal connections that minimise a loss. Also local searches where you make some small changes and test, e.g. simulated annealing.
  
  If your aggregation function is fixed, and not learnable, it's not clear how expressive this model could be. However, with some equivalent to bias, and lots of layers, I suspect in some cases you could show that it was actually equivalent to a neural network. The limitations on training approaches that could work probably mean such a model has niche used at best.
  
  
  ---
  
  