From what I found in the code, the format transformation occuring in the dataset.map function is driven by the transmit_format(func) wrapper defined in datasets.arrow_dataset.py [184:219].
My guess is that this is necessary so that methods such as .to_pandas() or .to_csv() can be applied on the mapped dataset.
I used these functions with your toy dataset:
def zero_shot_classify_sequences_v2(examples, threshold=0.5):
  with classifier.device_placement():
    outputs = classifier(examples['text'], candidate_labels, multi_label=True)

  final_outputs = []
  labels_set = set()
  for result in outputs:
    try:
      final_output = dict([(result['labels'][i], score) for i, score in enumerate(result['scores']) if score > threshold])
      labels_set.update(list(final_output.keys()))
    except TypeError:
      final_output = {}

    print("final_output before return: \n", final_output)
    final_outputs.append(final_output)

  print("Set of labels in threshold with this batch: ", len(labels_set), labels_set)
  examples['predicted_labels'] = final_outputs

  return examples


def print_dataset(dataset):
  for i, results in enumerate(dataset):
    pp((i, results))

Colab settings:
transformers: 4.5.0
python: sys.version_info(major=3, minor=7, micro=10, releaselevel='final', serial=0)

I found a few quirks with the map function:
A. Batching problems

Using zero_shot_classify_sequences_v2, setting batch=False returns empty score dicts.
Let b = the batch size, with N=4 (as in your toy example): if b={1,3} (i.e. odd and < N), this error is raised: ArrowInvalid: Column 1 named text expected length 3 but got length 1.

B. Reformatting problems:

The columns reformatting (in the map transformation) is dependent on the batch-size. Examples:

# 1. batch_size=2
updated_dataset_v2b2 = dataset.map(zero_shot_classify_sequences_v2, batched=True, batch_size=2)
#last printed line:
>>>Set of labels in threshold with this batch:  7 {'Plant-based', 'Vertical & Indoor farming', 'Functional foods & Other alternative ingredients', 'B2B marketplaces', 'Meal planning & Nutrition', 'Meal replacements & Supplements', 'Food safety & Quality'}

# 2. batch_size=5
updated_dataset_v2b5 = dataset.map(zero_shot_classify_sequences_v2, batched=True, batch_size=5)
#last printed line:
>>>Set of labels in threshold with this batch:  13 {'Plant-based', 'Vertical & Indoor farming', 'Wine tech', 'Retail tech', 'B2B marketplaces', 'Supply chain monitoring & traceability', 'Functional foods & Other alternative ingredients', 'Ag marketplace', 'Meal planning & Nutrition', 'Delivery & fulfilment technology', 'Meal replacements & Supplements', 'Food safety & Quality', 'Drinks'}


Missing new columns: the union over the labels appearing in the dict is somehow missing the entries from the ultimate batch if b < N. This is why these 6 predicted labels are returned when b=2:

{'Ag marketplace',
 'Delivery & fulfilment technology',
 'Drinks',
 'Retail tech',
 'Supply chain monitoring & traceability',
 'Wine tech'}

But if b >= N, the resulting (common) columns are correct (i.e. 13 for b={4,5}):
{'Ag marketplace',
 'B2B marketplaces',
 'Delivery & fulfilment technology',
 'Drinks',
 'Food safety & Quality',
 'Functional foods & Other alternative ingredients',
 'Meal planning & Nutrition',
 'Meal replacements & Supplements',
 'Plant-based',
 'Retail tech',
 'Supply chain monitoring & traceability',
 'Vertical & Indoor farming',
 'Wine tech'}

It smells like a bug to me.