Solved by BramVanroy in post #18 
              
              
                You can use the scripts provided in the transformers library. These are well-tested and provide many useful options. For BERT in particular, you can have a look here. If you have additional questions, you can use their dedicated forum.