Probably an easy one, but not having any luck in finding the solution, so thought I’d make a post.
To use tensor cores effectively with mixed precision training a NVIDIA guide recommends to “pad vocabulary to be a multiple of 8”.
I’ve searched the tokenizers documentation for answers but haven’t found much luck. The closest I could find is the pp_tokenizer.vocab_size method that returns the current vocab size, but I can’t assign it a new value.
Any idea how I can do this?