we just print the shape of attention_mask for brevity.


Okay, got it.


Seems like you are truncating more aggressively for T5.


Yeah, but I think it should cover over 90% of the distribution.

+Weird that the t5 version is missing special tokens, whereas bart has them. I suspect this is the problem.

Yeah I think we have a tokenization-related problem. See further findings below:
This 1 is where these log files are dumped; it uses batch_decode(.) to convert the indices into strings. And by default, batch_decode(.) has to dump the special tokens 1. So we should definitely be able to see the special tokens.
However, when I looked into tok_batch.json file (snippet shown below), I see that:

All the instances in input_ids end with the special id of 1, which corresponds to the end-of-sentence special token. So they seem to be good. Not sure why this is not correctly shown in text_batch.json file.

attention_mask are fine.

decoder_input_ids start with a 0 (padding) and end with 1 (end of sentence); so they’re good. Again, not sure why they’re not shown in the output.

{
  "input_ids": [
    [
      113,
      10159,
      7,
      405,
      3,
      88,
      333,
      140,
      28,
      3,
      60,
      1
    ],
    [
      149,
      186,
      1688,
      19,
      17472,
      388,
      57,
      3,
      4900,
      102,
      107,
      1
    ],
    [
      213,
      103,
      8,
      248,
      17721,
      942,
      8,
      5431,
      58,
      1,
      0,
      0
    ]
  ],
  "attention_mask": [
    [
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1
    ],
    [
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1
    ],
    [
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      1,
      0,
      0
    ]
  ],
  "labels": [
    [
      16121,
      8688,
      1,
      0,
      0,
      0,
      0,
      0
    ],
    [
      305,
      4959,
      41,
      511,
      4182,
      3,
      61,
      1
    ],
    [
      8,
      2788,
      16617,
      2473,
      1,
      0,
      0,
      0
    ]
  ],
  "decoder_input_ids": [
    [
      0,
      16121,
      8688,
      1,
      0,
      0,
      0,
      0
    ],
    [
      0,
      305,
      4959,
      41,
      511,
      4182,
      3,
      61
    ],
    [
      0,
      8,
      2788,
      16617,
      2473,
      1,
      0,
      0
    ]
  ]
}

Any additional suggestions for debugging, @sshleifer @valhalla?