<h1 dir="auto">🐛 Bug</h1>
<h2 dir="auto">Information</h2>
<p dir="auto">"mrm8488/bert-spanish-cased-finetuned-ner"</p>
<p dir="auto">Language I am using the model on (English, Chinese ...):  Spanish</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>create a <code class="notranslate">ner</code> pipeline</li>
<li>pass flag <code class="notranslate">grouped_entities</code></li>
<li>entities are not grouped as expected see sample below</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="NER_MODEL = &quot;mrm8488/bert-spanish-cased-finetuned-ner&quot;
nlp_ner = pipeline(&quot;ner&quot;, model=NER_MODEL,
                   grouped_entities=True,
                   tokenizer=(NER_MODEL, {&quot;use_fast&quot;: False}))

t = &quot;&quot;&quot;Consuelo Araújo Noguera, ministra de cultura del presidente Andrés Pastrana (1998.2002) fue asesinada por las Farc luego de haber permanecido secuestrada por algunos meses.&quot;&quot;&quot;
ner(t)
&gt;&gt;&gt; 
[ {'entity_group': 'B-PER', 'score': 0.901019960641861, 'word': 'Consuelo'}, 
 {'entity_group': 'I-PER', 'score': 0.9990904808044434, 'word': 'Araújo Noguera'}, 
 {'entity_group': 'B-PER', 'score': 0.9998136162757874, 'word': 'Andrés'}, 
 {'entity_group': 'I-PER', 'score': 0.9996985991795858, 'word': 'Pastrana'}, 
 {'entity_group': 'B-ORG', 'score': 0.9989739060401917, 'word': 'Far'}]"><pre class="notranslate"><span class="pl-v">NER_MODEL</span> <span class="pl-c1">=</span> <span class="pl-s">"mrm8488/bert-spanish-cased-finetuned-ner"</span>
<span class="pl-s1">nlp_ner</span> <span class="pl-c1">=</span> <span class="pl-en">pipeline</span>(<span class="pl-s">"ner"</span>, <span class="pl-s1">model</span><span class="pl-c1">=</span><span class="pl-v">NER_MODEL</span>,
                   <span class="pl-s1">grouped_entities</span><span class="pl-c1">=</span><span class="pl-c1">True</span>,
                   <span class="pl-s1">tokenizer</span><span class="pl-c1">=</span>(<span class="pl-v">NER_MODEL</span>, {<span class="pl-s">"use_fast"</span>: <span class="pl-c1">False</span>}))

<span class="pl-s1">t</span> <span class="pl-c1">=</span> <span class="pl-s">"""Consuelo Araújo Noguera, ministra de cultura del presidente Andrés Pastrana (1998.2002) fue asesinada por las Farc luego de haber permanecido secuestrada por algunos meses."""</span>
<span class="pl-en">ner</span>(<span class="pl-s1">t</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> 
[ {<span class="pl-s">'entity_group'</span>: <span class="pl-s">'B-PER'</span>, <span class="pl-s">'score'</span>: <span class="pl-c1">0.901019960641861</span>, <span class="pl-s">'word'</span>: <span class="pl-s">'Consuelo'</span>}, 
 {<span class="pl-s">'entity_group'</span>: <span class="pl-s">'I-PER'</span>, <span class="pl-s">'score'</span>: <span class="pl-c1">0.9990904808044434</span>, <span class="pl-s">'word'</span>: <span class="pl-s">'Araújo Noguera'</span>}, 
 {<span class="pl-s">'entity_group'</span>: <span class="pl-s">'B-PER'</span>, <span class="pl-s">'score'</span>: <span class="pl-c1">0.9998136162757874</span>, <span class="pl-s">'word'</span>: <span class="pl-s">'Andrés'</span>}, 
 {<span class="pl-s">'entity_group'</span>: <span class="pl-s">'I-PER'</span>, <span class="pl-s">'score'</span>: <span class="pl-c1">0.9996985991795858</span>, <span class="pl-s">'word'</span>: <span class="pl-s">'Pastrana'</span>}, 
 {<span class="pl-s">'entity_group'</span>: <span class="pl-s">'B-ORG'</span>, <span class="pl-s">'score'</span>: <span class="pl-c1">0.9989739060401917</span>, <span class="pl-s">'word'</span>: <span class="pl-s">'Far'</span>}]</pre></div>
<h2 dir="auto">Expected behavior</h2>
<h3 dir="auto">Inconsistent grouping</h3>
<p dir="auto">I expect the first two items of the given sample( <code class="notranslate">B-PER</code>, and <code class="notranslate">I-PER</code>) to be grouped. As they are contiguous tokens and correspond to a single entity spot.   It seems the current code does not take into account <code class="notranslate">B</code> and <code class="notranslate">I</code> tokens.</p>
<p dir="auto">expected output:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=" {'entity_group': 'I-PER', 'score': 0.9990904808044434, 'word': ' Consuelo Araújo Noguera'}, 
 {'entity_group': 'I-PER', 'score': 0.9998136162757874, 'word': 'Andrés Pastrana'}, 
 {'entity_group': 'B-ORG', 'score': 0.9989739060401917, 'word': 'Farc'}]"><pre class="notranslate"><code class="notranslate"> {'entity_group': 'I-PER', 'score': 0.9990904808044434, 'word': ' Consuelo Araújo Noguera'}, 
 {'entity_group': 'I-PER', 'score': 0.9998136162757874, 'word': 'Andrés Pastrana'}, 
 {'entity_group': 'B-ORG', 'score': 0.9989739060401917, 'word': 'Farc'}]
</code></pre></div>
<h3 dir="auto">Lost tokens?</h3>
<p dir="auto">for the same input, passing <code class="notranslate">grouped_entities=False</code> generates the following output:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[
{'word': 'Cons', 'score': 0.9994944930076599, 'entity': 'B-PER', 'index': 1},
{'word': '##uelo', 'score': 0.802545428276062, 'entity': 'B-PER', 'index': 2}, 
{'word': 'Ara', 'score': 0.9993102550506592, 'entity': 'I-PER', 'index': 3}, 
{'word': '##új', 'score': 0.9993743896484375, 'entity': 'I-PER', 'index': 4}, 
{'word': '##o', 'score': 0.9992871880531311, 'entity': 'I-PER', 'index': 5}, 
{'word': 'No', 'score': 0.9993029236793518, 'entity': 'I-PER', 'index': 6}, 
{'word': '##guera', 'score': 0.9981776475906372, 'entity': 'I-PER', 'index': 7}, 
{'word': 'Andrés', 'score': 0.9998136162757874, 'entity': 'B-PER', 'index': 15}, 
{'word': 'Pas', 'score': 0.999740719795227, 'entity': 'I-PER', 'index': 16}, 
{'word': '##tran', 'score': 0.9997414350509644, 'entity': 'I-PER', 'index': 17}, 
{'word': '##a', 'score': 0.9996136426925659, 'entity': 'I-PER', 'index': 18}, 
{'word': 'Far', 'score': 0.9989739060401917, 'entity': 'B-ORG', 'index': 28}, 
{'word': '##c', 'score': 0.7188423275947571, 'entity': 'I-ORG', 'index': 29}]"><pre class="notranslate"><code class="notranslate">[
{'word': 'Cons', 'score': 0.9994944930076599, 'entity': 'B-PER', 'index': 1},
{'word': '##uelo', 'score': 0.802545428276062, 'entity': 'B-PER', 'index': 2}, 
{'word': 'Ara', 'score': 0.9993102550506592, 'entity': 'I-PER', 'index': 3}, 
{'word': '##új', 'score': 0.9993743896484375, 'entity': 'I-PER', 'index': 4}, 
{'word': '##o', 'score': 0.9992871880531311, 'entity': 'I-PER', 'index': 5}, 
{'word': 'No', 'score': 0.9993029236793518, 'entity': 'I-PER', 'index': 6}, 
{'word': '##guera', 'score': 0.9981776475906372, 'entity': 'I-PER', 'index': 7}, 
{'word': 'Andrés', 'score': 0.9998136162757874, 'entity': 'B-PER', 'index': 15}, 
{'word': 'Pas', 'score': 0.999740719795227, 'entity': 'I-PER', 'index': 16}, 
{'word': '##tran', 'score': 0.9997414350509644, 'entity': 'I-PER', 'index': 17}, 
{'word': '##a', 'score': 0.9996136426925659, 'entity': 'I-PER', 'index': 18}, 
{'word': 'Far', 'score': 0.9989739060401917, 'entity': 'B-ORG', 'index': 28}, 
{'word': '##c', 'score': 0.7188423275947571, 'entity': 'I-ORG', 'index': 29}]
</code></pre></div>
<p dir="auto">when using <code class="notranslate">grouped_entities</code>  the last entity <code class="notranslate">word</code> (<code class="notranslate">##c</code>) got lost, it is not even considered as a different group</p>
<p dir="auto"><code class="notranslate"> {'entity_group': 'B-ORG', 'score': 0.9989739060401917, 'word': 'Far'}]</code></p>
<h2 dir="auto">Environment info</h2>

<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 2.11.0</li>
<li>Platform: OSX</li>
<li>Python version: 3.7</li>
<li>PyTorch version (GPU?):  1.5.0</li>
<li>Tensorflow version (GPU?):</li>
<li>Using GPU in script?: no</li>
<li>Using distributed or parallel set-up in script?: no</li>
</ul>
<h1 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji> Feature request</h1>
<p dir="auto">output probabilities of generated sequences in generate function (generation utils)</p>
<p dir="auto">thank you so much! :)</p>