Answers.Body: <p>In tensorflow models can be their own layers as part of a larger architecture.<br>
  (If you want to see an example you can check out a notebook I have <a href="https://github.com/delta2echo/tensorflow_portfolio/blob/main/04_Transfer_Learning/04_Transfer_Learning_Project.ipynb" rel="nofollow noreferrer">here</a> and <a href="https://github.com/delta2echo/StackOverflow_Responses/blob/main/Multivariate_LSTM.ipynb" rel="nofollow noreferrer">here</a>)</p>
  <p>Specifically what you are trying to do is called <a href="https://www.tensorflow.org/js/tutorials/transfer/what_is_transfer_learning" rel="nofollow noreferrer"><strong>transfer learning</strong></a>
  but in general, a NN model can take <em>any</em> numerical input set and use it to output another numerical output set. The input can even be the outputs of other models, and their hidden layers.</p>
  <p>Suppose you have 2 models each with 4 layers, that take 2 datasets X1 &amp; X2 and output 2 other sets:&lt;br</p>
  <blockquote>
  <p>X1 -&gt; M1(L1,L2,L3,L4) -&gt; Y1<br>
  X2 -&gt; M2(L1,L2,L3,L4) -&gt; Y2</p>
  </blockquote>
  <p>You can directly <a href="https://www.tensorflow.org/api_docs/python/tf/concat" rel="nofollow noreferrer">concatenate</a> the outputs (Y's) of each model into a <em>new input</em> to a 3rd model:</p>
  <blockquote>
  <p>[Y1, Y2] = X3 -&gt; M3(L1,...) -&gt; Y3</p>
  </blockquote>
  <p><em>or</em>
  you can slice each model and at one of the hidden layers and feed <strong>those</strong> outputs to M3:</p>
  <blockquote>
  <p>[M1L3, M2L3] = X3 -&gt; M3(L1,...) -&gt; Y3</p>
  </blockquote>
  <p>You can even mix and match in any way you find that works:</p>
  <blockquote>
  <p>[M1L3, M2L2, Y1, Y2] = X3 -&gt; M3(...) -&gt; Y3</p>
  </blockquote>
  <p>If you use a TensorFlow model, 'layers' is a property of the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/Model#attributes_1" rel="nofollow noreferrer"><strong>model</strong></a> and you can slice them as needed to build <a href="https://www.tensorflow.org/tutorials/customization/custom_layers" rel="nofollow noreferrer">custom layers</a>.</p>
  <p>However, if you just want to feed the outputs of each model [Y1,Y2] to M3,
  I would probably do this using tensorflow's functional api creating two branches:</p>
  <pre><code>Input1 = tf.keras.Layers.Input(...)
  Input2 = tf.keras.Layers.Input(...)
  x1 = M1(Input1)   #pretrained model 1
  x2 = M2(Input2)   #pretrained model 2
  Output = M3(x1)
  Output = M3(x2)
  model = tf.keras.Model(inputs=[Input1,Input2],
                         outputs=[Output]
  </code></pre>
  <p>Hope this helps! :)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-17T02:25:19.560Z
Answers.Text: How do I tell TensorFlow to not update the borrowed models when training the conglomerate?
Answers.UserDisplayName: Brannon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-17T10:33:31.577Z
Answers.Text: to prevent layers from being trained you can explicitly set a 'trainable' [parameter](https://www.tensorflow.org/guide/keras/transfer_learning) for the models .
Answers.UserDisplayName: J.Kent
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-17T01:36:00.217Z
Answers.LastActivityDate: 2022-06-17T01:36:00.217Z
Answers.UserDisplayName: J.Kent
Body: <p>Suppose I have a NN that learns to predict the time it takes a robot to move between two jobs. That's three inputs (for starters): robot, job A, job B. Not all robots travel at the same speed, and jobs are not symmetrically spaced.</p>
  <p>Suppose I have another NN that learns how long a robot takes to do some given job. All robots can do all jobs, but the robots are not equally effective at all jobs.</p>
  <p>Now suppose that I want to make some new NN that takes a list of robots and a list of jobs and gives me robot-to-job map and sequence that minimizes the time to accomplish all jobs. <strong>Conceptually, how can I make this third NN utilize the knowledge contained in the other two NNs?</strong> What do you call this type of architecture? What structural form for this third NN do you recommend?</p>
  
Tags: <neural-networks><sequence-modeling><model-request><feature-extraction>
Title: Make an NN utilize other NNs as part of its decision process
UserDisplayName: Brannon
ThreadText: # Make an NN utilize other NNs as part of its decision process
  
  (Asked by: username_0 on 2022-06-16)
  
  Suppose I have a NN that learns to predict the time it takes a robot to move between two jobs. That's three inputs (for starters): robot, job A, job B. Not all robots travel at the same speed, and jobs are not symmetrically spaced.
  
  Suppose I have another NN that learns how long a robot takes to do some given job. All robots can do all jobs, but the robots are not equally effective at all jobs.
  
  Now suppose that I want to make some new NN that takes a list of robots and a list of jobs and gives me robot-to-job map and sequence that minimizes the time to accomplish all jobs. **Conceptually, how can I make this third NN utilize the knowledge contained in the other two NNs?** What do you call this type of architecture? What structural form for this third NN do you recommend?
  
  **Tags:** neural-networks, sequence-modeling, model-request, feature-extraction
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-06-17. Score: 1
  
  In tensorflow models can be their own layers as part of a larger architecture.  
  (If you want to see an example you can check out a notebook I have [here](https://github.com/delta2echo/tensorflow_portfolio/blob/main/04_Transfer_Learning/04_Transfer_Learning_Project.ipynb) and [here](https://github.com/delta2echo/StackOverflow_Responses/blob/main/Multivariate_LSTM.ipynb))
  
  Specifically what you are trying to do is called [**transfer learning**](https://www.tensorflow.org/js/tutorials/transfer/what_is_transfer_learning)
  but in general, a NN model can take *any* numerical input set and use it to output another numerical output set. The input can even be the outputs of other models, and their hidden layers.
  
  Suppose you have 2 models each with 4 layers, that take 2 datasets X1 & X2 and output 2 other sets:<br
  
  > X1 -> M1(L1,L2,L3,L4) -> Y1  
  > X2 -> M2(L1,L2,L3,L4) -> Y2
  
  You can directly [concatenate](https://www.tensorflow.org/api_docs/python/tf/concat) the outputs (Y's) of each model into a *new input* to a 3rd model:
  
  > [Y1, Y2] = X3 -> M3(L1,...) -> Y3
  
  *or*
  you can slice each model and at one of the hidden layers and feed **those** outputs to M3:
  
  > [M1L3, M2L3] = X3 -> M3(L1,...) -> Y3
  
  You can even mix and match in any way you find that works:
  
  > [M1L3, M2L2, Y1, Y2] = X3 -> M3(...) -> Y3
  
  If you use a TensorFlow model, 'layers' is a property of the [**model**](https://www.tensorflow.org/api_docs/python/tf/keras/Model#attributes_1) and you can slice them as needed to build [custom layers](https://www.tensorflow.org/tutorials/customization/custom_layers).
  
  However, if you just want to feed the outputs of each model [Y1,Y2] to M3,
  I would probably do this using tensorflow's functional api creating two branches:
  
  ```
  Input1 = tf.keras.Layers.Input(...)
  Input2 = tf.keras.Layers.Input(...)
  x1 = M1(Input1)   #pretrained model 1
  x2 = M2(Input2)   #pretrained model 2
  Output = M3(x1)
  Output = M3(x2)
  model = tf.keras.Model(inputs=[Input1,Input2],
                         outputs=[Output]
  
  ```
  
  Hope this helps! :)
  
  > **Comments:**
  > **username_0**: How do I tell TensorFlow to not update the borrowed models when training the conglomerate?  
  > **username_1**: to prevent layers from being trained you can explicitly set a 'trainable' [parameter](https://www.tensorflow.org/guide/keras/transfer_learning) for the models .  
  
  ---
  
  