Solved by SMMousavi in post #8 
              
              
                I found a solution for now. Just before tokenizing, I’m converting my datasets to pandas data frame and converting them back to datasets. By doing this, the datasets library doesn’t load the same cache each time and recognizes that my datasets are different. 
new_datasets = DatasetDict({
    "train"…