<p>In the keras Functional API, intermediate tensors are symbolic that means their shapes cannot directly evaluated or printed during the model construction , but if you want to determine their shape you can determine it after building the model , you can do it by</p>
<pre><code>
def model_builder(hp, vocab_size, max_token_size):
    # Input layer of the model
    text_input = tf.keras.Input(shape=(max_token_size,), dtype=&quot;int32&quot;, name=&quot;text_input&quot;)
    
    # Embedding layer with shape debugging to know the shape betterly 
    embedding_layer = tf.keras.layers.Embedding(
        input_dim=vocab_size,
        output_dim=128
    )(text_input)

    # Debugging intermediate tensor shape using a temporary model
    temp_model = tf.keras.Model(inputs=text_input, outputs=embedding_layer)
    print(&quot;Embedding Layer Shape (Static):&quot;, temp_model.output_shape)
    
    # Adding a Lambda layer for dynamic shape debugging during training
    embedding_layer = tf.keras.layers.Lambda(
        lambda x: tf.print(&quot;Embedding Layer Output Shape (Dynamic):&quot;, tf.shape(x)) or x,
        output_shape=(max_token_size, 128)
    )(embedding_layer)

    # Example intermediate layer (for example using a global average polling layer)
    x = tf.keras.layers.GlobalAveragePooling1D()(embedding_layer)
    temp_model2 = tf.keras.Model(inputs=text_input, outputs=x)
    print(&quot;GlobalAveragePooling1D Output Shape (Static):&quot;, temp_model2.output_shape)

    # Fully connected dense layer as Fully connected NN
    output = tf.keras.layers.Dense(1, activation=&quot;sigmoid&quot;)(x)
    
    # Final model building
    model = tf.keras.Model(inputs=text_input, outputs=output)

    # Print overall model structure with their layers and shapes
    model.summary()

    # Visualize the model
    from tensorflow.keras.utils import plot_model
    plot_model(model, show_shapes=True, to_file=&quot;model_structure.png&quot;)

    return model

if __name__==&quot;__main__&quot;:
  hp = None  # Replace with Hyperparameter object if needed
  vocab_size = 5000
  max_token_size = 100

  # Build and debug the model
  model = model_builder(hp, vocab_size, max_token_size)

  # Compile the model with eager execution for additional debugging during training
  model.compile(optimizer=&quot;adam&quot;, loss=&quot;binary_crossentropy&quot;, metrics=[&quot;accuracy&quot;], run_eagerly=True)
</code></pre>
<p>it print the shapes like below<a href="https://i.sstatic.net/3KIwhabl.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3KIwhabl.png" alt="Image shape of model" /></a></p>
<p>I am using <code>2.17.1</code> version of the tensorflow</p>

CC BY-SA 4.0
2024-12-13T17:21:49.260Z
This worked wonderfully. Thank you very much. I wish I asked before...
Ferda-Ozdemir-Sonmez
CC BY-SA 4.0
2024-12-13T07:14:08.723Z
2024-12-13T07:14:08.723Z
Keval Pandya
<p>I am trying to debug my model_builder function in Keras Functional API by printing the shapes of intermediate tensors. However, none of the methods I have tried so far seem to work as expected.</p>
<p>Here's the code snippet for my model_builder function:</p>
<pre><code>def model_builder(hp, vocab_size, max_token_size):
    text_input = tf.keras.Input(shape=(max_token_size,), dtype=&quot;int32&quot;, name=&quot;text_input&quot;)

    # Embedding layer
    embedding_layer = tf.keras.layers.Embedding(
        input_dim=vocab_size,
        output_dim=128,
        input_length=max_token_size
    )(text_input)
# Attempt 1: Using Python's print
print(&quot;Embedding Layer Shape (Attempt 1):&quot;, embedding_layer.getOutput(0))  # This throws an error

# Attempt 2: Using a Lambda layer with tf.print
embedding_layer = tf.keras.layers.Lambda(
    lambda x: tf.print(&quot;Embedding Layer Output Shape (Attempt 2):&quot;, tf.shape(x)) or x
)(embedding_layer)

# Attempt 3: Using tf.print directly
tf.print(&quot;Embedding Layer Output Shape (Attempt 3):&quot;, tf.shape(embedding_layer))  
</code></pre>
<p>I understand that Keras Functional API constructs a symbolic graph and print or tf.print might not work directly in graph construction. How can I print intermediate tensor shapes while building the model? This is necessary because I have multiple inputs and am getting unmatched matrix size errors. I need to understand at which layer this occurs.
Best Regards,
Ferda</p>

<tensorflow><keras><models>
How to print intermediate shapes in a Keras Functional API model_builder during graph construction?
Ferda-Ozdemir-Sonmez
# How to print intermediate shapes in a Keras Functional API model_builder during graph construction?

(Asked by: username_0 on 2024-12-13)

I am trying to debug my model_builder function in Keras Functional API by printing the shapes of intermediate tensors. However, none of the methods I have tried so far seem to work as expected.
Here's the code snippet for my model_builder function:
def model_builder(hp, vocab_size, max_token_size):
    text_input = tf.keras.Input(shape=(max_token_size,), dtype="int32", name="text_input")

    # Embedding layer
    embedding_layer = tf.keras.layers.Embedding(
        input_dim=vocab_size,
        output_dim=128,
        input_length=max_token_size
    )(text_input)
# Attempt 1: Using Python's print
print("Embedding Layer Shape (Attempt 1):", embedding_layer.getOutput(0))  # This throws an error

# Attempt 2: Using a Lambda layer with tf.print
embedding_layer = tf.keras.layers.Lambda(
    lambda x: tf.print("Embedding Layer Output Shape (Attempt 2):", tf.shape(x)) or x
)(embedding_layer)

# Attempt 3: Using tf.print directly
tf.print("Embedding Layer Output Shape (Attempt 3):", tf.shape(embedding_layer))  

I understand that Keras Functional API constructs a symbolic graph and print or tf.print might not work directly in graph construction. How can I print intermediate tensor shapes while building the model? This is necessary because I have multiple inputs and am getting unmatched matrix size errors. I need to understand at which layer this occurs.
Best Regards,
Ferda

**Tags:** tensorflow, keras, models


---

## 1 answers

### Answer by username_1 on 2024-12-13. Score: 2 | Accepted answer

In the keras Functional API, intermediate tensors are symbolic that means their shapes cannot directly evaluated or printed during the model construction , but if you want to determine their shape you can determine it after building the model , you can do it by

def model_builder(hp, vocab_size, max_token_size):
    # Input layer of the model
    text_input = tf.keras.Input(shape=(max_token_size,), dtype="int32", name="text_input")
    
    # Embedding layer with shape debugging to know the shape betterly 
    embedding_layer = tf.keras.layers.Embedding(
        input_dim=vocab_size,
        output_dim=128
    )(text_input)

    # Debugging intermediate tensor shape using a temporary model
    temp_model = tf.keras.Model(inputs=text_input, outputs=embedding_layer)
    print("Embedding Layer Shape (Static):", temp_model.output_shape)
    
    # Adding a Lambda layer for dynamic shape debugging during training
    embedding_layer = tf.keras.layers.Lambda(
        lambda x: tf.print("Embedding Layer Output Shape (Dynamic):", tf.shape(x)) or x,
        output_shape=(max_token_size, 128)
    )(embedding_layer)

    # Example intermediate layer (for example using a global average polling layer)
    x = tf.keras.layers.GlobalAveragePooling1D()(embedding_layer)
    temp_model2 = tf.keras.Model(inputs=text_input, outputs=x)
    print("GlobalAveragePooling1D Output Shape (Static):", temp_model2.output_shape)

    # Fully connected dense layer as Fully connected NN
    output = tf.keras.layers.Dense(1, activation="sigmoid")(x)
    
    # Final model building
    model = tf.keras.Model(inputs=text_input, outputs=output)

    # Print overall model structure with their layers and shapes
    model.summary()

    # Visualize the model
    from tensorflow.keras.utils import plot_model
    plot_model(model, show_shapes=True, to_file="model_structure.png")

    return model

if __name__=="__main__":
  hp = None  # Replace with Hyperparameter object if needed
  vocab_size = 5000
  max_token_size = 100

  # Build and debug the model
  model = model_builder(hp, vocab_size, max_token_size)

  # Compile the model with eager execution for additional debugging during training
  model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"], run_eagerly=True)

it print the shapes like below
I am using 2.17.1 version of the tensorflow

> **Comments:**
> **username_0**: This worked wonderfully. Thank you very much. I wish I asked before...  

---

