Here is an example of doing named entity recognition, using a model and a tokenizer. The process is the following:
Instantiate a tokenizer and a model from the checkpoint name. The model is identified as a BERT model and loads it
   with the weights stored in the checkpoint.
Define a sequence with known entities, such as "Hugging Face" as an organisation and "New York City" as a location.
Split words into tokens so that they can be mapped to predictions. We use a small hack by, first, completely
   encoding and decoding the sequence, so that we're left with a string that contains the special tokens.
Encode that sequence into IDs (special tokens are added automatically).
Retrieve the predictions by passing the input to the model and getting the first output. This results in a
   distribution over the 9 possible classes for each token. We take the argmax to retrieve the most likely class for
