Have you tried to write your decode function like in:

  
      

      TensorFlow
  

  
    

tf.data.TFRecordDataset  |  TensorFlow Core v2.6.0 8

  A Dataset comprising records from one or more TFRecord files.