Hi Mac,
Having both TF and Pytorch in the same script is a little bit weird, and I am afraid that there might be some efficiency problem if we load data by TF and then convert it to Pytorch Tensor for Dataloader. So I think converting tfrecord to arvo and read avro to the dataloader is really a good idea.
Could you please show me a sketch about how to convert tf data to arvo and load them to pytorch tensor? Thanks very much! I really appreciate it.
I have also find the link you sent to me. However, it seems this problem is not addressed in that post. People still complain about the efficiency.
Regards,
Yin