UPD: this workaround is not feasible in my case even with 1Tb of RAM =(
Here’s a workaround that I figured out, but it is slow and requires to put all of your data into RAM (and I don’t have enough).
dataset_dict = datasets.DatasetDict({
    "train": datasets.arrow_dataset.Dataset.from_dict(dataset_dict["train"].data.to_pydict()),
    "validation": datasets.arrow_dataset.Dataset.from_dict(dataset_dict["validation"].data.to_pydict()),
    "test": datasets.arrow_dataset.Dataset.from_dict(dataset_dict["test"].data.to_pydict()),
})

dataset_dict.save_to_disk("../data/wikipedia_rank_nocache")