Answers.Body: <p>Mixture of Experts might be what you are looking for.</p>
  <p>A Mixture of Experts model (MoE), divides a task into subtasks and designs seperate models for each of the tasks (This would be <code>N</code> in your case). It also defines a gating model to decide which expert to use, and during inference it uses the gating model output to pool/select predictions and makes the final decision.</p>
  <p><a href="https://i.sstatic.net/3zE0c.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3zE0c.png" alt="MoE Image" /></a></p>
  <p><a href="https://en.wikipedia.org/wiki/Mixture_of_experts" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Mixture_of_experts</a></p>
  <p><a href="https://www.cs.toronto.edu/%7Ehinton/csc321/notes/lec15.pdf" rel="nofollow noreferrer">https://www.cs.toronto.edu/~hinton/csc321/notes/lec15.pdf</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-05T09:33:22.443Z
Answers.Text: Can you please provide more details about these "Mixture of Experts"? On our site, we expect users to provide more details than just a name and a link to an external resource. If you don't feel like providing more details, I can convert this answer to a comment.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-06T01:25:58.203Z
Answers.Text: Sorry for the late response, I was busy last night and completely forgot about this question. I added some more details as you said.
Answers.UserDisplayName: DKDK
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-06T09:02:28.240Z
Answers.Text: I am not sure if "mixture of experts" is a widely used term. Maybe you mean "ensemble learning"?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-06T09:18:22.440Z
Answers.Text: I think mixture of experts is more suitable for the op's question. In Ensemble Learning, the seperate models doesn't necessarily have divided problem spaces.
Answers.UserDisplayName: DKDK
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-09T23:52:59.660Z
Answers.Text: Thanks. This is what I was seeking. The main drawback here is that each of the expert networks is evaluated for each input. In my architecture, these networks could be arbitrarily complex, and we may only want to use one expert for a given input. Anyway, good suggestion.
  
  > I am not sure if "mixture of experts" is a widely used term.
  
  The term is used widely in ML literature. Mixture of experts is a type of ensemble learning.
Answers.UserDisplayName: Wowee
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-05T02:00:26.387Z
Answers.LastActivityDate: 2022-01-06T01:24:34.960Z
Answers.LastEditDate: 2022-01-06T01:24:34.960Z
Answers.UserDisplayName: DKDK
Body: <p>I'd like to design a deep learning architecture in which the output of a primary neural network <span class="math-container">$M_{\theta}$</span> determines which neural network <span class="math-container">$N^i_{\alpha}$</span> in a set of secondary networks <span class="math-container">$\mathcal{N}$</span> to use next. For example, <span class="math-container">$M_{\theta}$</span> could be a multiclass classifier, where the predicted class determines <span class="math-container">$N^i_{\alpha}$</span>. The networks may have different dimensions and activation functions. Is there a name for this type of architecture?</p>
  
Tags: <neural-networks><algorithm-request><model-request><ensemble-learning><hierarchical-rl>
Title: Is there an approach where the output of one neural network is used to choose the next neural network?
UserDisplayName: Wowee
ThreadText: # Is there an approach where the output of one neural network is used to choose the next neural network?
  
  (Asked by: username_0 on 2022-01-05)
  
  I'd like to design a deep learning architecture in which the output of a primary neural network $M\_{\theta}$ determines which neural network $N^i\_{\alpha}$ in a set of secondary networks $\mathcal{N}$ to use next. For example, $M\_{\theta}$ could be a multiclass classifier, where the predicted class determines $N^i\_{\alpha}$. The networks may have different dimensions and activation functions. Is there a name for this type of architecture?
  
  **Tags:** neural-networks, algorithm-request, model-request, ensemble-learning, hierarchical-rl
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-01-05. Score: 1
  
  Mixture of Experts might be what you are looking for.
  
  A Mixture of Experts model (MoE), divides a task into subtasks and designs seperate models for each of the tasks (This would be `N` in your case). It also defines a gating model to decide which expert to use, and during inference it uses the gating model output to pool/select predictions and makes the final decision.
  
  [![MoE Image](https://i.sstatic.net/3zE0c.png)](https://i.sstatic.net/3zE0c.png)
  
  <https://en.wikipedia.org/wiki/Mixture_of_experts>
  
  [https://www.cs.toronto.edu/~hinton/csc321/notes/lec15.pdf](https://www.cs.toronto.edu/%7Ehinton/csc321/notes/lec15.pdf)
  
  > **Comments:**
  > **username_2**: Can you please provide more details about these "Mixture of Experts"? On our site, we expect users to provide more details than just a name and a link to an external resource. If you don't feel like providing more details, I can convert this answer to a comment.  
  > **username_1**: Sorry for the late response, I was busy last night and completely forgot about this question. I added some more details as you said.  
  > **username_2**: I am not sure if "mixture of experts" is a widely used term. Maybe you mean "ensemble learning"?  
  > **username_1**: I think mixture of experts is more suitable for the op's question. In Ensemble Learning, the seperate models doesn't necessarily have divided problem spaces.  
  > **username_0**: Thanks. This is what I was seeking. The main drawback here is that each of the expert networks is evaluated for each input. In my architecture, these networks could be arbitrarily complex, and we may only want to use one expert for a given input. Anyway, good suggestion.  > I am not sure if "mixture of experts" is a widely used term.  The term is used widely in ML literature. Mixture of experts is a type of ensemble learning.  
  
  ---
  
  