I don’t believe that this is the issue, because when I print out the encoded inputs (print(inputs)), the numbers are the same, and the max length is far below the 512.
That said, let me test this and get back on it!
EDIT1: Tokenization length made a difference, and was the cause of differences while using bart-large.
EDIT2: However, XSum is still hallucinating strange results. CNN has consistent behavior.
So this does confirm that the automodel works exactly as the bartconditional, and the difference there was only user error.
I guess the remaining question is just…  does anybody have insight into how pipeline actually works? I have no idea what it’s doing behind the scenes, or why it’s giving me different results here with XSUM.