Answers.Body: <p>A more efficient way would be creating a multi input model, with something like this:</p>
  
  <pre><code>___________    _____________
  |__Image__|    |Other input|            
  _____|_____     _____|_____
  |___CNN___|     |__Dense___|
  _____|______    _____|______
  |_Features1_|   |_Features2_|
           __|_____|__
           |__Merge___|
           _____|______
           |___Dense__|
           _____|_____
           |__Output__|
  </code></pre>
  
  <p>However, you could also combine the unstructured data to the image, as stated in the quora answer:</p>
  
  <blockquote>
    <h3>The out-of-the-box method:</h3>
    
    <p>If you want to just take your CNN library and use it without much
    thought, there's an easy way to do it.</p>
    
    <p>Your image has “channels”: red blue and green channels, for example.
    Just add another channel for each unstructured feature. Those channels
    will just be 2D-arrays whose entries are all the same value: the value
    of your outside feature.</p>
    
    <p>It means more memory and more parameters though. If you have a lot of
    unstructured data, this can become prohibitively expensive.</p>
    
    <h3>The more efficient method (and still not hard):</h3>
    
    <p>You use one or more deconvolutional filters to bring the unstructured
    data up to the size of the structured data, concatenate them along the
    channel dimension, and keep going as if nothing happened.</p>
  </blockquote>
  
  <p>Source: <a href="https://www.quora.com/How-can-l-train-a-CNN-with-extra-features-other-than-the-pixels" rel="nofollow noreferrer">How can l train a CNN with extra features other than the pixels? (Quora)</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T11:27:01.430Z
Answers.LastActivityDate: 2019-11-02T16:13:18.643Z
Answers.LastEditDate: 2019-11-02T16:13:18.643Z
Answers.UserDisplayName: Clement
Body: <p>I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.? </p>
  
  <p>It is certainly possible to put the output of a CNN and other data into, say, a densely connected network; but it seems inefficient. Are there well-established ways of doing something like this?</p>
  
Tags: <convolutional-neural-networks><architecture>
Title: Are there well-established ways of mixing different inputs (e.g. image and numbers)?
UserDisplayName: Mathieu Bouville
ThreadText: # Are there well-established ways of mixing different inputs (e.g. image and numbers)?
  
  (Asked by: username_0 on 2019-10-30)
  
  I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.?
  
  It is certainly possible to put the output of a CNN and other data into, say, a densely connected network; but it seems inefficient. Are there well-established ways of doing something like this?
  
  **Tags:** convolutional-neural-networks, architecture
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-30. Score: 4
  
  A more efficient way would be creating a multi input model, with something like this:
  ___________    _____________
  |__Image__|    |Other input|            
  _____|_____     _____|_____
  |___CNN___|     |__Dense___|
  _____|______    _____|______
  |_Features1_|   |_Features2_|
           __|_____|__
           |__Merge___|
           _____|______
           |___Dense__|
           _____|_____
           |__Output__|
  
  However, you could also combine the unstructured data to the image, as stated in the quora answer:
  
  The out-of-the-box method:
  If you want to just take your CNN library and use it without much
    thought, there's an easy way to do it.
  Your image has “channels”: red blue and green channels, for example.
    Just add another channel for each unstructured feature. Those channels
    will just be 2D-arrays whose entries are all the same value: the value
    of your outside feature.
  It means more memory and more parameters though. If you have a lot of
    unstructured data, this can become prohibitively expensive.
  The more efficient method (and still not hard):
  You use one or more deconvolutional filters to bring the unstructured
    data up to the size of the structured data, concatenate them along the
    channel dimension, and keep going as if nothing happened.
  
  Source: How can l train a CNN with extra features other than the pixels? (Quora)
  
  
  ---
  
  