I’m not sure I see the advantage if doing this inside tf.data. WHgy not just filter it first?
AFAIK you’ll need you to use a tf.py_function for the actual check.
Then your options are either to return a zero-length tensor for the result if a url fails, or use:

  
      

      TensorFlow
  

  
    

tf.data.experimental.ignore_errors  |  TensorFlow Core v2.7.0

  Creates a Dataset from another Dataset and silently ignores any errors.