thanks @sgugger I have read this link. My main issue is that my data is stored in a Pandas dataframe so I am not sure how to adapt your code (and in particular mimic the dataset.map) function. Is there an easy way to transfer a pandas dataframe to a tensorflow dataset? does that make sense?