Thanks for your reply.
Sorry, maybe my expression is not accurate.
I mean why the training time of the first epoch is pretty longer than the training time of the subsequent epochs in tensorflow.
When performing the same operation (caching some batchs of data) in pytorch, I cannot find such a significant time reduction.
Does tensorflow use some other optimization operations that pytorch does not?
if index in self.cache:
    img, cls = self.cache[index]
else:
    ... ...
    if len(self.cache) < self.cache_size:
        self.cache[index] = (img, cls)