Answers.Body: <p>The error which you are facing is due to a shape mismatch while combining the dense layers of <code>text_dense</code> and <code>numeric_dense</code> using <code>Concatenate()</code></p>
  <p>The Answers of your question</p>
  <p>A1. Yes this issue is due to the way you process the outputs of dense layers of <code>text_dense</code> and <code>numeric_dense</code>.The issue lies in the fact that GlobalMaxPooling1D reduces the shape of the text_input data to (32, 128) (batch size × number of filters in the Conv1D layer), but the Dense(224) layer doesn't modify the batch size dimension. The text_dense output shape is (32, 224). but the <code>numeric_dense</code> output also has a shape <code>(32,224)</code> so they seems to be compatible. this shows  that the concatenation issue may stem from incorrect input processing outside this snippet (e.g., feeding the model incorrectly during training).</p>
  <p>A2. No , the issue is not related to <code>max_token_size</code> or the dense output size. Both inputs produce expected shapes, as shown in your debugging outputs. The error likely arises from feeding incompatible inputs during training or runtime.</p>
  <p>A3. to slove it you need to debug the shape of all intermediate layers by printing their shapes and make sure that <code>text_input</code> and <code>numeric_input</code> are correctly batched and shaped when passed to the model. and check that the data preprocessing pipeline to confirm <code>text_input</code> matches <code>(32, 1149)</code> and numeric_input matches <code>(32, 26)</code>. to determine the shape you can use <code>tf.print</code> function to print the shape.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-17T22:40:05.317Z
Answers.Text: I will accept this as the solution even though it does not include direct solution. But adding more log helped to solve.
Answers.UserDisplayName: Ferda-Ozdemir-Sonmez
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-13T07:28:20.937Z
Answers.LastActivityDate: 2024-12-13T07:28:20.937Z
Answers.UserDisplayName: Keval Pandya
Body: <p>I'm working on a Keras model using the Functional API that processes two inputs:</p>
  <p>Text Input: A padded integer array with a shape of (32, 1149) (batch size 32, sequence length 1149). Example: [5, 10, 3, ..., 0, 0, 0]. //this is a tokenized and padded text of lines with max word length is 1149 for a line of text.
  Numeric Input: A numeric feature array with a shape of (32, 26) (batch size 32, 26 numeric features). Example: [247., 47., 0., 0., ..., 1., 0.].
  The labels are binary with a shape of (32,). Example: [0, 1, 0, 0, ..., 1, 0].</p>
  <p>When I try to train the model, I get the following matrix size mismatch error:</p>
  <blockquote>
  <p>Matrix size-incompatible: In[0]: [32,1149], In[1]: [26,224]</p>
  </blockquote>
  <p>Here is the relevant code snippet for my model_builder function:</p>
  <pre><code>def model_builder(hp, vocab_size, max_token_size):
      # Text Input
      text_input = tf.keras.Input(shape=(max_token_size,), dtype=&quot;int32&quot;, name=&quot;text_input&quot;)
      embedding_layer = tf.keras.layers.Embedding(
          input_dim=vocab_size,
          output_dim=128,
          input_length=max_token_size
      )(text_input)
  
      conv_layer = tf.keras.layers.Conv1D(
          filters=128,
          kernel_size=3,
          activation=&quot;relu&quot;,
          padding=&quot;same&quot;
      )(embedding_layer)
      pooling_layer = tf.keras.layers.GlobalMaxPooling1D()(conv_layer)
      text_dense = tf.keras.layers.Dense(224, activation='relu')(pooling_layer)
  
      # Numeric Input
      numeric_input = tf.keras.Input(shape=(26,), name=&quot;numeric_input&quot;)
      numeric_dense = tf.keras.layers.Dense(224, activation='relu')(numeric_input)
  
      # Combine
      combined = tf.keras.layers.Concatenate()([text_dense, numeric_dense])
      combined_dense = tf.keras.layers.Dense(256, activation='relu')(combined)
      output = tf.keras.layers.Dense(1, activation='sigmoid')(combined_dense)
  
      # Compile
      model = tf.keras.Model(inputs=[text_input, numeric_input], outputs=output)
      model.compile(
          optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
          loss='binary_crossentropy',
          metrics=['accuracy']
      )
  
      return model
  </code></pre>
  <p>I've tried debugging the input shapes before passing them to the model with the following outputs:</p>
  <blockquote>
  <p>Text Input Shape: (32, None) Text Input Example: [ 5 10  3 ...  0  0
  0] Numeric Input Shape: (32, 26) Numeric Input Example: [247.  47.<br />
  0.   0.   0.   0.   1.   0.   0.   1.   0.   0.  19.  36.   0.   0.   0.   0.   0.   0.   1.   1.   0.   0.   0.   1.] Label Shape: (32,) Label Example: [0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 1 0 1 0 0 1 1 1 0 1
  1 0 0 1 0]</p>
  </blockquote>
  <p>The mismatch seems to occur when combining the dense layers (text_dense and numeric_dense) using Concatenate. However, both have been set to output 224 units.</p>
  <p>Questions:
  1-Could this issue be related to the way I process the text_input or numeric_input before combining them?
  2-Is the issue with how I defined the max_token_size for text_input or the dense layer output size for numeric_input?
  3-What is the correct way to debug and resolve this kind of size mismatch error?
  4-Any insights or debugging strategies to ensure that the inputs are compatible for concatenation would be greatly appreciated!
  Best Regards,
  Ferda</p>
  
Tags: <tensorflow><keras>
Title: How to Resolve Matrix Size Mismatch in TensorFlow Model Builder for Combined Text and Numeric Inputs?
UserDisplayName: Ferda-Ozdemir-Sonmez
ThreadText: # How to Resolve Matrix Size Mismatch in TensorFlow Model Builder for Combined Text and Numeric Inputs?
  
  (Asked by: username_0 on 2024-12-13)
  
  I'm working on a Keras model using the Functional API that processes two inputs:
  Text Input: A padded integer array with a shape of (32, 1149) (batch size 32, sequence length 1149). Example: [5, 10, 3, ..., 0, 0, 0]. //this is a tokenized and padded text of lines with max word length is 1149 for a line of text.
  Numeric Input: A numeric feature array with a shape of (32, 26) (batch size 32, 26 numeric features). Example: [247., 47., 0., 0., ..., 1., 0.].
  The labels are binary with a shape of (32,). Example: [0, 1, 0, 0, ..., 1, 0].
  When I try to train the model, I get the following matrix size mismatch error:
  
  Matrix size-incompatible: In[0]: [32,1149], In[1]: [26,224]
  
  Here is the relevant code snippet for my model_builder function:
  def model_builder(hp, vocab_size, max_token_size):
      # Text Input
      text_input = tf.keras.Input(shape=(max_token_size,), dtype="int32", name="text_input")
      embedding_layer = tf.keras.layers.Embedding(
          input_dim=vocab_size,
          output_dim=128,
          input_length=max_token_size
      )(text_input)
  
      conv_layer = tf.keras.layers.Conv1D(
          filters=128,
          kernel_size=3,
          activation="relu",
          padding="same"
      )(embedding_layer)
      pooling_layer = tf.keras.layers.GlobalMaxPooling1D()(conv_layer)
      text_dense = tf.keras.layers.Dense(224, activation='relu')(pooling_layer)
  
      # Numeric Input
      numeric_input = tf.keras.Input(shape=(26,), name="numeric_input")
      numeric_dense = tf.keras.layers.Dense(224, activation='relu')(numeric_input)
  
      # Combine
      combined = tf.keras.layers.Concatenate()([text_dense, numeric_dense])
      combined_dense = tf.keras.layers.Dense(256, activation='relu')(combined)
      output = tf.keras.layers.Dense(1, activation='sigmoid')(combined_dense)
  
      # Compile
      model = tf.keras.Model(inputs=[text_input, numeric_input], outputs=output)
      model.compile(
          optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
          loss='binary_crossentropy',
          metrics=['accuracy']
      )
  
      return model
  
  I've tried debugging the input shapes before passing them to the model with the following outputs:
  
  Text Input Shape: (32, None) Text Input Example: [ 5 10  3 ...  0  0
  0] Numeric Input Shape: (32, 26) Numeric Input Example: [247.  47.
  0.   0.   0.   0.   1.   0.   0.   1.   0.   0.  19.  36.   0.   0.   0.   0.   0.   0.   1.   1.   0.   0.   0.   1.] Label Shape: (32,) Label Example: [0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 1 0 1 0 0 1 1 1 0 1
  1 0 0 1 0]
  
  The mismatch seems to occur when combining the dense layers (text_dense and numeric_dense) using Concatenate. However, both have been set to output 224 units.
  Questions:
  1-Could this issue be related to the way I process the text_input or numeric_input before combining them?
  2-Is the issue with how I defined the max_token_size for text_input or the dense layer output size for numeric_input?
  3-What is the correct way to debug and resolve this kind of size mismatch error?
  4-Any insights or debugging strategies to ensure that the inputs are compatible for concatenation would be greatly appreciated!
  Best Regards,
  Ferda
  
  **Tags:** tensorflow, keras
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-12-13. Score: 0 | Accepted answer
  
  The error which you are facing is due to a shape mismatch while combining the dense layers of `text_dense` and `numeric_dense` using `Concatenate()`
  
  The Answers of your question
  
  A1. Yes this issue is due to the way you process the outputs of dense layers of `text_dense` and `numeric_dense`.The issue lies in the fact that GlobalMaxPooling1D reduces the shape of the text\_input data to (32, 128) (batch size × number of filters in the Conv1D layer), but the Dense(224) layer doesn't modify the batch size dimension. The text\_dense output shape is (32, 224). but the `numeric_dense` output also has a shape `(32,224)` so they seems to be compatible. this shows that the concatenation issue may stem from incorrect input processing outside this snippet (e.g., feeding the model incorrectly during training).
  
  A2. No , the issue is not related to `max_token_size` or the dense output size. Both inputs produce expected shapes, as shown in your debugging outputs. The error likely arises from feeding incompatible inputs during training or runtime.
  
  A3. to slove it you need to debug the shape of all intermediate layers by printing their shapes and make sure that `text_input` and `numeric_input` are correctly batched and shaped when passed to the model. and check that the data preprocessing pipeline to confirm `text_input` matches `(32, 1149)` and numeric\_input matches `(32, 26)`. to determine the shape you can use `tf.print` function to print the shape.
  
  > **Comments:**
  > **username_0**: I will accept this as the solution even though it does not include direct solution. But adding more log helped to solve.  
  
  ---
  
  