Answers.Body: <p>Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work.
  Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.</p>
  <p>I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.</p>
  <pre><code>def create_mlp(self, arr, regress=False): 
          # define MLP network
          print(&quot;Array&quot;, arr)
          model = Sequential()
          model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation=&quot;relu&quot;))
          model.add(Dense((env.rows * env.cols)//2, activation=&quot;relu&quot;))
          **model.add.flatten() ### shape = (None, 1600)**
          # check to see if the regression node should be added
          if regress:
              model.add(Dense(1, activation=&quot;linear&quot;))
          # return our model
          return model
  </code></pre>
  <p>And just remove the reshape layer in <strong>create_cnn</strong> function.
  (output shape should be = (None, 600)).</p>
  <p>then concatenate two model</p>
  <p><code>combinedInput = concatenate([mlp.output, cnn.output])  ## output shape =(None, 2200)</code></p>
  <p>Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in <strong>create_mlp</strong> function.</p>
  <p>Your code should work this way.
  You can read <a href="https://stackoverflow.com/questions/57202016/concatenate-two-models-with-tensorflow-keras">this</a> simple one for better understanding.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-15T13:57:34.967Z
Answers.LastActivityDate: 2020-11-15T13:57:34.967Z
Answers.UserDisplayName: yang2x
Body: <p>I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s.
  The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.</p>
  <p>The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.</p>
  <p>This is my attempt so far, having this <a href="https://www.pyimagesearch.com/2019/02/04/keras-multiple-inputs-and-mixed-data/" rel="nofollow noreferrer">tutorial</a> as a reference.</p>
  <p>Here is the code:</p>
  <p>First, create the MLP network</p>
  <pre><code>def create_mlp(self, arr, regress=False): # for the position input
          # define MLP network
          print(&quot;Array&quot;, arr)
          model = Sequential()
          model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation=&quot;relu&quot;))
          model.add(Dense((env.rows * env.cols)//2, activation=&quot;relu&quot;))
          
          # check to see if the regression node should be added
          if regress:
              model.add(Dense(1, activation=&quot;linear&quot;))
              
          # return our model
          return model
  </code></pre>
  <p>Then, the CNN</p>
  <pre><code>def create_cnn(self, width, height, depth=1, regress=False): # for the matrix
          # initialize the input shape and channel dimension
          inputShape = (height, width, depth)
          output_nodes = 6e2
          
          # define the model input
          inputs = Input(shape=inputShape)
  
          # if this is the first CONV layer then set the input
          # appropriately
          x = inputs
          
          input_layer = Input(shape=(width, height, depth))
          conv1 = Conv2D(100, 3, padding=&quot;same&quot;, activation=&quot;relu&quot;, input_shape=inputShape) (input_layer)
          pool1 = MaxPooling2D(pool_size=(2,2), padding=&quot;same&quot;)(conv1)
          flat = Flatten()(pool1)
          hidden1 = Dense(200, activation='softmax')(flat) #relu
  
          batchnorm1 = BatchNormalization()(hidden1) 
          output_layer = Dense(output_nodes, activation=&quot;softmax&quot;)(batchnorm1) 
          output_layer2 = Dense(output_nodes, activation=&quot;relu&quot;)(output_layer) 
          output_reshape = Reshape((int(output_nodes), 1))(output_layer2)
          model = Model(inputs=input_layer, outputs=output_reshape)
  
          # return the CNN
          return model
  </code></pre>
  <p>Then, concatenate the two</p>
  <pre><code>def _build_model(self):
          # create the MLP and CNN models
          mlp = self.create_mlp(env.stateSpacePos)
          cnn = self.create_cnn(3, len(env.UEs))
          
          # create the input to our final set of layers as the *output* of both
          # the MLP and CNN
          combinedInput = concatenate([mlp.output, cnn.output])
          
          # our final FC layer head will have two dense layers, the final one
          # being our regression head
          x = Dense(len(env.stateSpacePos), activation=&quot;relu&quot;)(combinedInput)
          x = Dense(1, activation=&quot;linear&quot;)(x)
          
          # our final model will accept categorical/numerical data on the MLP
          # input and images on the CNN input, outputting a single value
          model = Model(inputs=[mlp.input, cnn.input], outputs=x)
          
          opt = Adam(lr=self.learning_rate, decay=self.epsilon_decay)
          model.compile(loss=&quot;mean_absolute_percentage_error&quot;, optimizer=opt)
          
          print(model.summary())
          
          return model
  </code></pre>
  <p>I have an error:</p>
  <pre><code>A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]
  </code></pre>
  <p>The line of code that gives the error is:</p>
  <pre><code>combinedInput = concatenate([mlp.output, cnn.output])
  </code></pre>
  <p>This is the MLP summary
  <a href="https://i.sstatic.net/JWVh3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JWVh3.png" alt="MLP Summary" /></a></p>
  <p>And this is the CNN summary
  <a href="https://i.sstatic.net/sU6ag.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/sU6ag.png" alt="CNN Summary" /></a></p>
  <p>I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-15T18:03:33.020Z
Comments.Text: Programming questions are off-topic here. Please, next time, ask this type of question on Stack Overflow. See https://ai.stackexchange.com/help/on-topic for more details.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-15T18:07:32.100Z
Comments.Text: Oh okay, sorry.
Comments.UserDisplayName: Ness
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-15T18:08:18.300Z
Comments.Text: Here we focus on the theoretical aspects of AI, including RL. So, if you have a question about the theory of RL, feel free to ask it here :)
Comments.UserDisplayName: nbro
Tags: <convolutional-neural-networks><python><tensorflow><dqn><multilayer-perceptrons>
Title: Keras DQN Model with Multiple Inputs and Multiple Outputs
UserDisplayName: Ness
ThreadText: # Keras DQN Model with Multiple Inputs and Multiple Outputs
  
  (Asked by: username_0 on 2020-11-14)
  
  I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s.
  The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.
  The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.
  This is my attempt so far, having this tutorial as a reference.
  Here is the code:
  First, create the MLP network
  def create_mlp(self, arr, regress=False): # for the position input
          # define MLP network
          print("Array", arr)
          model = Sequential()
          model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
          model.add(Dense((env.rows * env.cols)//2, activation="relu"))
          
          # check to see if the regression node should be added
          if regress:
              model.add(Dense(1, activation="linear"))
              
          # return our model
          return model
  
  Then, the CNN
  def create_cnn(self, width, height, depth=1, regress=False): # for the matrix
          # initialize the input shape and channel dimension
          inputShape = (height, width, depth)
          output_nodes = 6e2
          
          # define the model input
          inputs = Input(shape=inputShape)
  
          # if this is the first CONV layer then set the input
          # appropriately
          x = inputs
          
          input_layer = Input(shape=(width, height, depth))
          conv1 = Conv2D(100, 3, padding="same", activation="relu", input_shape=inputShape) (input_layer)
          pool1 = MaxPooling2D(pool_size=(2,2), padding="same")(conv1)
          flat = Flatten()(pool1)
          hidden1 = Dense(200, activation='softmax')(flat) #relu
  
          batchnorm1 = BatchNormalization()(hidden1) 
          output_layer = Dense(output_nodes, activation="softmax")(batchnorm1) 
          output_layer2 = Dense(output_nodes, activation="relu")(output_layer) 
          output_reshape = Reshape((int(output_nodes), 1))(output_layer2)
          model = Model(inputs=input_layer, outputs=output_reshape)
  
          # return the CNN
          return model
  
  Then, concatenate the two
  def _build_model(self):
          # create the MLP and CNN models
          mlp = self.create_mlp(env.stateSpacePos)
          cnn = self.create_cnn(3, len(env.UEs))
          
          # create the input to our final set of layers as the *output* of both
          # the MLP and CNN
          combinedInput = concatenate([mlp.output, cnn.output])
          
          # our final FC layer head will have two dense layers, the final one
          # being our regression head
          x = Dense(len(env.stateSpacePos), activation="relu")(combinedInput)
          x = Dense(1, activation="linear")(x)
          
          # our final model will accept categorical/numerical data on the MLP
          # input and images on the CNN input, outputting a single value
          model = Model(inputs=[mlp.input, cnn.input], outputs=x)
          
          opt = Adam(lr=self.learning_rate, decay=self.epsilon_decay)
          model.compile(loss="mean_absolute_percentage_error", optimizer=opt)
          
          print(model.summary())
          
          return model
  
  I have an error:
  A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]
  
  The line of code that gives the error is:
  combinedInput = concatenate([mlp.output, cnn.output])
  
  This is the MLP summary
  
  And this is the CNN summary
  
  I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.
  
  **Tags:** convolutional-neural-networks, python, tensorflow, dqn, multilayer-perceptrons
  > **Comments:**
  > **username_1**: Programming questions are off-topic here. Please, next time, ask this type of question on Stack Overflow. See https://ai.stackexchange.com/help/on-topic for more details.  
  > **username_0**: Oh okay, sorry.  
  > **username_1**: Here we focus on the theoretical aspects of AI, including RL. So, if you have a question about the theory of RL, feel free to ask it here :)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-11-15. Score: 1 | Accepted answer
  
  Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work.
  Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.
  I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.
  def create_mlp(self, arr, regress=False): 
          # define MLP network
          print("Array", arr)
          model = Sequential()
          model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
          model.add(Dense((env.rows * env.cols)//2, activation="relu"))
          **model.add.flatten() ### shape = (None, 1600)**
          # check to see if the regression node should be added
          if regress:
              model.add(Dense(1, activation="linear"))
          # return our model
          return model
  
  And just remove the reshape layer in create_cnn function.
  (output shape should be = (None, 600)).
  then concatenate two model
  combinedInput = concatenate([mlp.output, cnn.output])  ## output shape =(None, 2200)
  Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in create_mlp function.
  Your code should work this way.
  You can read this simple one for better understanding.
  
  
  ---
  
  