Could you read the data once and save it in another format?
I just had a look at the code on how to read TFRecord files, and it looks like you need a session object and some transformations.
The easiest way would probably be to read the data from another source.
If that’s not possible, you could try to transform the data somehow from tf.float32 to a Tensor.
I’m not sure, if you can fit this code somehow into a Dataset and DataLoader.