Use dedicated PAD token to preserve EOS training labels
DataCollatorForLanguageModeling(mlm=False) masks every label equal to pad_token_id. The current tokenizer uses EOS ID 1 as PAD, so real document-ending EOS tokens lose supervision in padded batches.
Use the existing dedicated <｜▁pad▁｜> token (ID 2) consistently in the tokenizer, model configuration/defaults, and generation configuration. EOS stays at ID 1. The vocabulary, tokenizer.json, raw unpadded encoding, and 24-layer vanilla architecture are unchanged. Update tokenizer provenance and the padding instructions.
For documents [42, 43, EOS] and [42, EOS], the standard collator now produces:
input_ids: [[42, 43, 1], [42, 1, 2]]
labels: [[42, 43, 1], [42, 1, -100]]
Validation: 13 tests passed — the 11 existing dense-model behavioral tests plus regressions for standard-collator EOS preservation/backward and PAD/EOS serialization/raw-encoding parity. The dedicated embedding padding row has no gradient; EOS supervision is retained. Full-size meta parameter count remains 1,076,700,672.
The previous 100-step experiments used padding-free custom batches, kept EOS labels, and did not use the standard collator. Their exact training slices and evaluation panels contain no ID 2; their reported metrics are unaffected by this issue. Those trials were not rerun for this fix.
This PR is independent of the recurrent architecture candidate in PR #1. No loop fields or model execution changes are included. No tests, datasets, credentials, or weights are uploaded to the model repository.