Solved by sgugger in post #2 
              
              
                You need to add agregation_strategy="simple" to your pipeline creation, to tell the pipeline to group together the tokens in the same entities.