Thanks @xzhu1900!
@bencherian based on the PR you mentioned, you could write a chunk datareader entirely in python if and only if your batch is a vector basic types. Currently, there are bindings for the following basic type readers available on python:
ChunkDataReaderDouble, ChunkDataReaderInt16T, ChunkDataReaderInt64T, ChunkDataReaderUint8T, ChunkDataReaderFloat, ChunkDataReaderInt32T, ChunkDataReaderInt8T
In that case, you could do this:
import torch

# A batch is a array of floats
class PythonChunkDatareaderDouble(torch._C.data.ChunkDataReaderDouble):
    def __init__(self):
        print('constructor')
    def read_chunk(self, index):
        print('read_chunk {}'.format(index))
    def chunk_count(self):
        print('chunk_count')
        return 0
    def reset(self, size=None):
        print('reset {}'.format(size))

y=PythonChunkDatareaderDouble()
y.reset()
y.chunk_count()
y.read_chunk(0)