title: Cannot convert a symbolic Keras input/output to a numpy array TypeError when using sampled_softmax in tensorflow 2.4
body: <p>I'm trying to train a word embedding classifier using TF2.4 with Keras and using the <code>tf.nn.sampled_softmax_loss</code>. However, when calling the <code>fit</code> method of the model, &quot;Cannot convert a symbolic Keras input/output to a numpy array&quot; TypeError occurs. Please help me to fix the error or with an alternative approach to do candidate sampling.</p>
  <pre class="lang-py prettyprint-override"><code>import tensorflow as tf
  import numpy as np
  
   
  TextVectorization = tf.keras.layers.experimental.preprocessing.TextVectorization
  
  class SampledSoftmaxLoss: #(tf.keras.losses.Loss):
  
    def __init__(self, model, n_classes):
      self.model = model
      output_layer = model.layers[-1]
      self.input = output_layer.input
      self.weights = output_layer.weights
      self.n_classes = n_classes
  
  
    def loss(self, y_true, y_pred, **kwargs):
      labels = tf.argmax(y_true, axis=1)
      labels = tf.expand_dims(labels, -1)
      loss = tf.nn.sampled_softmax_loss(
          weights=self.weights[0],
          biases=self.weights[1],
          labels=labels,
          inputs=self.input,
          num_sampled = 3,
          num_classes = self.n_classes
      )
      return loss
  
  
  max_features = 50  # Maximum vocab size.
  max_len = 10  # Sequence length to pad the outputs to.
  embedding_dims = 5
  
  input_data = np.array([ 
      &quot;Python Machine Learning&quot;,
      &quot;Data Science from Scratch: First Principles with Python&quot;, 
      &quot;Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems&quot;,
      &quot;Introduction to Machine Learning with Python: A Guide for Data Scientists&quot;,
      &quot;Vital Introduction to Machine Learning with Python: Best Practices to Improve and Optimize Machine Learning Systems and Algorithms&quot;,
      &quot;Machine Learning in Python: Essential Techniques for Predictive Analysis&quot;,
      &quot;Python Data Science Handbook: Essential Tools for Working with Data&quot;,
      &quot;Introducing Data Science: Big Data, Machine Learning, and more, using Python tools&quot;,
      &quot;Real-World Machine Learning&quot;])
  
  labels_one_hot = []
  for i in range(len(input_data)):
      labels = np.random.randint(0, 6, max_features)
      labels[labels&gt;1] = 1
      labels_one_hot.append(labels)
  
  labels_one_hot = np.array(labels_one_hot)
   
  # Create the text categorical encoding layer.
  vectorize_layer = TextVectorization(
           max_tokens=max_features,
           output_mode='int',
           output_sequence_length=max_len)
   
  vectorize_layer.adapt(text_dataset)
  inp = tf.keras.Input(shape=(1,), dtype=tf.string)
  idxs = vectorize_layer(inp)
  embed = tf.keras.layers.Embedding(max_features + 1, embedding_dims,input_length=max_len)(idxs)
  flat = tf.keras.layers.Flatten()(embed)
  out = tf.keras.layers.Dense(labels_one_hot.shape[1])(flat)
  
  model = tf.keras.models.Model(inp, out)
  
  softmax = SampledSoftmaxLoss(model, labels_one_hot.shape[1])
  
  model.compile('adam', loss=softmax.loss)
  model.summary()
  model.fit(input_data, labels_one_hot)
  model.predict(input_data)
  </code></pre>
  <p>This is the log:</p>
  <pre class="lang-py prettyprint-override"><code>INFO:tensorflow:Assets written to: model/assets
  Model: &quot;model_5&quot;
  _________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  input_84 (InputLayer)        [(None, 1)]               0         
  _________________________________________________________________
  text_vectorization_83 (TextV (None, 10)                0         
  _________________________________________________________________
  embedding_83 (Embedding)     (None, 10, 5)             255       
  _________________________________________________________________
  flatten_25 (Flatten)         (None, 50)                0         
  _________________________________________________________________
  dense_69 (Dense)             (None, 50)                2550      
  =================================================================
  Total params: 2,805
  Trainable params: 2,805
  Non-trainable params: 0
  _________________________________________________________________
  
  ---------------------------------------------------------------------------
  
  TypeError                                 Traceback (most recent call last)
  
  &lt;ipython-input-87-c35b8d1321a1&gt; in &lt;module&gt;()
      106 model.compile('adam', loss=softmax.loss)
      107 model.summary()
  --&gt; 108 model.fit(input_data, labels_one_hot)
      109 # Now, the model can map strings to integers, and you can add an embedding
      110 # layer to map these integers to learned embeddings.
  
  9 frames
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
      975           except Exception as e:  # pylint:disable=broad-except
      976             if hasattr(e, &quot;ag_error_metadata&quot;):
  --&gt; 977               raise e.ag_error_metadata.to_exception(e)
      978             else:
      979               raise
  
  TypeError: in user code:
  
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:805 train_function  *
          return step_function(self, iterator)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:795 step_function  **
          outputs = model.distribute_strategy.run(run_step, args=(data,))
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
          return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
          return self._call_for_each_replica(fn, args, kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
          return fn(*args, **kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:788 run_step  **
          outputs = model.train_step(data)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:756 train_step
          y, y_pred, sample_weight, regularization_losses=self.losses)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/compile_utils.py:238 __call__
          total_loss_metric_value, sample_weight=batch_dim)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/metrics_utils.py:90 decorated
          update_op = update_state_fn(*args, **kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/metrics.py:177 update_state_fn
          return ag_update_state(*args, **kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/metrics.py:364 update_state  **
          sample_weight, values)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/weights_broadcast_ops.py:155 broadcast_weights
          values = ops.convert_to_tensor(values, name=&quot;values&quot;)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/profiler/trace.py:163 wrapped
          return func(*args, **kwargs)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1540 convert_to_tensor
          ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:339 _constant_tensor_conversion_function
          return constant(v, dtype=dtype, name=name)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:265 constant
          allow_broadcast=True)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py:283 _constant_impl
          allow_broadcast=allow_broadcast))
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py:435 make_tensor_proto
          values = np.asarray(values)
      /usr/local/lib/python3.6/dist-packages/numpy/core/_asarray.py:83 asarray
          return array(a, dtype, copy=False, order=order)
      /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/keras_tensor.py:274 __array__
          'Cannot convert a symbolic Keras input/output to a numpy array. '
  
      TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
  </code></pre>