Args:

dataset: A tf.data.Dataset with dictionaries containing the key

‘input_feature_key‘.

sequence_length: dict mapping of feature key to int length for that feature.
output_features: mapping of keys to features.
use_prefix_lm_task: <bool> If True, include PrefixLM in the task mix.
rates: <Optional<List<float>> List of rates per task. If None, tasks are
9https://github.com/google/seqio

36

sampled uniformly.

example.

mean_noise_span_lengths: List of mean number of tokens per masked span per

noise_densities: List of what fraction of the tokens to mask.
shard_ds: <bool> If True, shard dataset per objective.
optional_task_prefixes: <Optional<list<str>> Strings to prepend for each

input_feature_key: which feature to use from the dataset as the input text

corruption scheme. NOTE: If including prefixLM task,
it must be the last prefix.

tokens.

to reduce padding.

merge_examples_to_reduce_padding: if True, combines multiple input examples
UL2- Unifying Language Learning Paradigms