I am trying to write a custom dataset using GeneratorBasedBuilder and I faced some samples have missing columns sometimes.
I really don’t know how to deal with it. Maybe we need creating a “nullable” feature value? Is it possible?
something like:
datasets.features.Sequence({"text": datasets.Value("string",nullable=True),
                            "answer_start": datasets.Value("int32"),})