transformers: ‘4.10.3’
datasets: 1.12.1
Im trying to follow the examples from here to make my own custom metric: datasets/super_glue.py at master · huggingface/datasets · GitHub 1
def _info(self):
return datasets.MetricInfo(
description=’_DESCRIPTION’,
citation=’_CITATION’,
inputs_description=’_KWARGS_DESCRIPTION’,
features=datasets.Features({
“predictions”: {
“1”: datasets.Value(“int64”),
“2”: datasets.Value(“int64”),
“3”: datasets.Value(“int64”),
},
‘references’: datasets.Value(“int64”),
}),
codebase_urls=[],
reference_urls=[],
# format=‘numpy’
)
and somewhere in code I call this:
metric.add_batch(predictions=outputs[‘predictions’], references=outputs[‘references’])
If  my predictions is not nested but just  equal to datasets.Value(“int64”) then it works.
What am I doing wrong here?
Stack trace:
metric.add_batch(predictions=outputs[‘predictions’], references=outputs[‘references’])
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/metric.py”, line 431, in add_batch
batch = self.info.features.encode_batch(batch)
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/features.py”, line 1080, in encode_batch
encoded_batch[key] = [encode_nested_example(self[key], obj) for obj in column]
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/features.py”, line 1080, in 
encoded_batch[key] = [encode_nested_example(self[key], obj) for obj in column]
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/features.py”, line 886, in encode_nested_example
k: encode_nested_example(sub_schema, sub_obj) for k, (sub_schema, sub_obj) in utils.zip_dict(schema, obj)
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/features.py”, line 885, in 
return {
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/utils/py_utils.py”, line 99, in zip_dict
yield key, tuple(d[key] for d in dicts)
File “/home/dspokoyn/.cache/pypoetry/virtualenvs/unit-7vxG7edj-py3.7/lib/python3.7/site-packages/datasets/utils/py_utils.py”, line 99, in 
yield key, tuple(d[key] for d in dicts)
TypeError: string indices must be integers