Solved by lhoestq in post #3 
              
              
                Hi ! I like the idea of using a name convention. By default it could use something like 
data_files = {
    "train": ["*train*"],
    "test": ["*test*"],
    "validation": ["*dev*", "*valid*"]
}

Another alternative would be to let the user specify the data_files patterns in an additional JSON file …