No, session is not needed. TFRecord format is straightforward protocol buffers container. It can be read into array of integers or floats quite easily (and efficiently). The reason I am asking is to avoid coding, maybe someone already solved this?
One technical difficulty may be that TFRecords are streaming (one does not know the number of data points upfront), and Dataset interface requires len() and random-access indexing…
Thanks