title: Tensorflow 2 getting "WARNING:tensorflow:x out of the last x calls to <function> triggered tf.function retracing."
body: <p>I'm working on a project where I have trained a series of binary classifiers with <strong>Keras</strong>, with <strong>Tensorflow</strong> as the backend engine. The input data I have is a series of images, where each binary classifier must make the prediction on the images, later I save the predictions on a CSV file.</p>
  <p>The problem I have is when I get the predictions from the first series of binary classifiers there isn't any warning, but when the 5th or 6th binary classifier calls the method <strong>predict</strong> on the input data I get the following warning:</p>
  <blockquote>
  <p>WARNING:tensorflow:5 out of the last 5 calls to &lt;function
  Model.make_predict_function..predict_function at
  0x2b280ff5c158&gt; triggered tf.function retracing. Tracing is expensive
  and the excessive number of tracings could be due to (1) creating
  @tf.function repeatedly in a loop, (2) passing tensors with different
  shapes, (3) passing Python objects instead of tensors. For (1), please
  define your @tf.function outside of the loop. For (2), @tf.function
  has experimental_relax_shapes=True option that relaxes argument shapes
  that can avoid unnecessary retracing. For (3), please refer to
  <a href="https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args" rel="noreferrer">https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args</a>
  and <a href="https://www.tensorflow.org/api_docs/python/tf/function" rel="noreferrer">https://www.tensorflow.org/api_docs/python/tf/function</a> for  more
  details.</p>
  </blockquote>
  <p>To answer each point in the parenthesis, here are my answers:</p>
  <ol>
  <li>The <strong>predict</strong> method is called inside a for loop.</li>
  <li>I don't pass tensors but a list of <strong>NumPy arrays</strong> of gray scale images, all of them with the same size in width and height. The only thing that can change is the batch size because the list can have only 1 image or more than one.</li>
  <li>As I wrote in point 2, I pass a list of NumPy arrays.</li>
  </ol>
  <p>I have debugged my program and found that this warning always happens when the method predict is called. To summarize the code I have written is the following:</p>
  <pre><code>import cv2 as cv
  import tensorflow as tf
  from tensorflow.keras.models import load_model
  # Load the models
  binary_classifiers = [load_model(path) for path in path2models]
  # Get the images
  images = [#Load the images with OpenCV]
  # Apply the resizing and reshapes on the images.
  my_list = list()
  for image in images:
      image_reworked = # Apply the resizing and reshaping on images
      my_list.append(image_reworked)
  
  # Get the prediction from each model
  # This is where I get the warning
  predictions = [model.predict(x=my_list,verbose=0) for model in binary_classifiers]
  </code></pre>
  <h3>What I have tried</h3>
  <p>I have defined a function as tf.function and putted the code of the predictions inside the tf.function like this</p>
  <pre><code>@tf.function
  def testing(models, faces):
      return [model.predict(x=faces,verbose=0) for model in models]
  </code></pre>
  <p>But I ended up getting the following error:</p>
  <blockquote>
  <p>RuntimeError: Detected a call to <code>Model.predict</code> inside a
  <code>tf.function</code>. Model.predict is a high-level endpoint that manages
  its own <code>tf.function</code>. Please move the call to <code>Model.predict</code> outside
  of all enclosing <code>tf.function</code>s. Note that you can call a <code>Model</code>
  directly on Tensors inside a <code>tf.function</code> like: <code>model(x)</code>.</p>
  </blockquote>
  <p>So calling the method <code>predict</code> is basically already a tf.function. So it's useless to define a tf.function when the warning I get it's from that method.</p>
  <p>I have also checked those other two questions:</p>
  <ol>
  <li><a href="https://stackoverflow.com/questions/61647404/tensorflow-2-getting-warningtensorflow9-out-of-the-last-9-calls-to-function">Tensorflow 2: Getting &quot;WARNING:tensorflow:9 out of the last 9 calls to  triggered tf.function retracing. Tracing is expensive&quot;</a></li>
  <li><a href="https://stackoverflow.com/questions/65563185/loading-multiple-saved-tensorflow-keras-models-for-prediction">Loading multiple saved tensorflow/keras models for prediction</a></li>
  </ol>
  <p>But neither of the two questions answers my question about how to avoid this warning. Plus I have also checked the links in the warning message but I couldn't solve my problem.</p>
  <h3>What I want</h3>
  <p>I simply want to avoid this warning. While I'm still getting the predictions from the models I noticed that the python program takes way too much time on doing predictions for a list of images.</p>
  <h3>What I'm using</h3>
  <ul>
  <li>Python 3.6.13</li>
  <li>Tensorflow 2.3.0</li>
  </ul>
  <h3>Solution</h3>
  <p>After some tries to suppress the warning from the <code>predict</code> method, I have checked the documentation of Tensorflow and in one of the first tutorials on how to use Tensorflow it is explained that, by default, Tensorflow is executed in eager mode, which is useful for testing and debugging the network models. Since I have already tested my models many times, it was only required to disable the eager mode by writing this single python line of code:</p>
  <p><code>tf.compat.v1.disable_eager_execution()</code></p>
  <p>Now the warning doesn't show up anymore.</p>