Hi,
I’m trying to recreate the OpenAI subset of YFCC100M and since the dataset is very large, I’d like to push it to the hub and allow streaming.
I was seeing it as:

large metadata file (containing title, description, file, path): I had planned a JSON but I guess I should use a tsv or csv (because a JSON probably needs to be read entirely to be parsed?)
an archive with all images: it seems that it needs to be a zip file

Would streaming be feasible? Is there a loading script I can get inspiration from?