Don’t worry figured out my problem in that last post, but I still have the same original issue, despite changing to integers.
# output of p.predictions:
[ 2 3 13 5 6 6 3 14 2 23 5 11 2 2 53 2 24 67 3 2 59 5 11 6 55 12 19 24 3 24]

---------------------------------------------------------------------------

ArrowInvalid Traceback (most recent call last)

[/usr/local/lib/python3.7/dist-packages/datasets/metric.py](https://zntzdj4nnfh-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20210616-060129-RC00_379666131#) in add_batch(self, predictions, references) 434 try: --> 435 self.writer.write_batch(batch) 436 except pa.ArrowInvalid:

10 frames

ArrowInvalid: Column 1 named references expected length 30 but got length 32

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)

[/usr/local/lib/python3.7/dist-packages/datasets/metric.py](https://zntzdj4nnfh-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20210616-060129-RC00_379666131#) in add_batch(self, predictions, references) 436 except pa.ArrowInvalid: 437 raise ValueError( --> 438 f"Predictions and/or references don't match the expected format.\n" 439 f"Expected format: {self.features},\n" 440 f"Input predictions: {predictions},\n"

ValueError: Predictions and/or references don't match the expected format. Expected format: {'predictions': Value(dtype='int32', id=None), 'references': Value(dtype='int32', id=None)}, Input predictions: [ 2 3 13 5 6 6 3 14 2 23 5 11 2 2 53 2 24 67 3 2 59 5 11 6 55 12 19 24 3 24], Input references: [ 3 0 3 8 4 2 3 4 8 2 16 3 11 3 1 38 3 2 8 35 3 0 32 4 8 3 28 7 10 20 4 15]

In fact done some more snooping and find the first number in the batch is being removed:
so this is the output the scores from the model and from the compute_loss function:
 tensor([ 4,  2,  3, 13,  5,  6,  6,  3, 14,  2, 23,  5, 11,  2,  2, 53],
       dtype=torch.int32)

and this is the output at the metrics function
 [ 2  3 13  5  6  6  3 14  2 23  5 11  2  2 53  2 24 67  3  2 59  5 11  6
 55 12 19 24  3 24]
---------------------