Well, you can call help on an object or a specific method to see more info. For instance, help(tokenizer.__call__) will display the documentation on the method that you’re using in your example. It’s the safest bet, in my opinion. However, the implementation of the method is inherited from PreTrainedTokenizerBase and accordingly the related docs can be found here 1.
Although, I do agree with you that not seeing this info on the child classes’ pages may be quite confusing.