Since Pytorch is in Python, inevitably the code will involve some Python overhead compared C++ code.
For example, with the data loading utility from Pytorch, we could code up a data-loader to load our data, be it text or image etc.
But what I observed is this Python data-loader is unbelievably slow than my C++ loader, since my dataset is huge, I cannot load it all into memory,so I have to iterate this dataset chunk-by-chunk, one by one epoch.
The slow data-loading in Python is un-tolerable, which also makes me think even if I fully load the data into memory, the network training Python code will also bring in substantial overhead.
BTW, in order to speed up the data-loading, should I discard this Python data-loader, and write Py-wrapper over my C++ loader?