<h2 dir="auto"><g-emoji class="g-emoji" alias="question" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/2753.png">❓</g-emoji> Questions &amp; Help</h2>
<p dir="auto">I want to use <code class="notranslate">mixed_precision</code>, and I found <a href="https://www.tensorflow.org/api_docs/python/tf/keras/mixed_precision/experimental/Policy" rel="nofollow">tf.keras.mixed_precision.experimental.Policy</a>.</p>
<p dir="auto">So I put <code class="notranslate">tf.keras.mixed_precision.experimental.set_policy("mixed_float16")</code> before <code class="notranslate">TFBertModel.from_pretrained(pretrained_weights)</code>. When I run the code, I got the following error:</p>
<blockquote>
<p dir="auto">InvalidArgumentError: cannot compute AddV2 as input <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="377057813" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/1" data-hovercard-type="pull_request" data-hovercard-url="/huggingface/transformers/pull/1/hovercard" href="https://github.com/huggingface/transformers/pull/1">#1</a>(zero-based) was expected to be a half tensor but is a float tensor [Op:AddV2] name: tf_bert_model_1/bert/embeddings/add/</p>
</blockquote>
<p dir="auto">which happened at <code class="notranslate">ret = model(model.dummy_inputs, training=False)  # build the network with dummy inputs</code>.</p>
<p dir="auto">I am not sure if I used it correctly. I think <code class="notranslate">tf.keras.mixed_precision.experimental.set_policy</code> is supposed to be used before constructing / build the model, as the tf page says <code class="notranslate">Policies can be passed to the 'dtype' argument of layer constructors, or a global policy can be set with 'tf.keras.mixed_precision.experimental.set_policy'</code>.</p>
<p dir="auto">I wonder if I can use AMP with tf based transformer models and how. Thanks.</p>
<p dir="auto"><a href="https://github.com/huggingface/transformers/files/3907032/error.txt">error.txt</a></p>
<h1 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h1>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (Bert, XLNet ...): 'ner' pipeline</p>
<p dir="auto">Language I am using the model on (English, Chinese ...): English</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Have transformers 3.0.2 installed</li>
<li>Run the below code</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from transformers import pipeline
nlp = pipeline('ner', grouped_entities=True)
nlp('Welcome to New York')"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">transformers</span> <span class="pl-k">import</span> <span class="pl-s1">pipeline</span>
<span class="pl-s1">nlp</span> <span class="pl-c1">=</span> <span class="pl-en">pipeline</span>(<span class="pl-s">'ner'</span>, <span class="pl-s1">grouped_entities</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-en">nlp</span>(<span class="pl-s">'Welcome to New York'</span>)</pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">We should receive <code class="notranslate">[{'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}</code>, but instead the output has duplicated 'New York': <code class="notranslate">[{'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}, {'entity_group': 'I-LOC', 'score': 0.9984402656555176, 'word': 'New York'}]</code>.</p>
<h3 dir="auto">The Cause of the Issue According to Me</h3>
<p dir="auto">After reading 3.0.2, I noticed that lines 1047-1049 were added.  I think this was done to fix a prior issue that caused the last named entity in the sequence to be occasionally omitted when <code class="notranslate">grouped_entities=True</code>.  Long story short, I think this snippet was a patch that only shifted the problem from being an occasional named entity omission to an occasional named entity duplicate.</p>
<p dir="auto">The for-loop that precedes this snippet is inconsistent in that sometimes the last named entity gets successfully added anyway (e.g. if the <code class="notranslate">if</code> clause on 1025 (first iteration) or 1032 is entered on the last iteration).  In this case, there is a duplicate entry upon the calling of the new code at 1047.  On the converse, the last named entity won’t be added if the <code class="notranslate">else</code> clause in line 1041 is entered on the last iteration.  In this case, the final named entity correctly gets added after the new code snippet is run.</p>
<p dir="auto">In short, there is a duplicate (I think) if (i) there is only one recognized named entity or (ii) the last named entity is one such that the tokenizer cut it up into multiple tokens.  Otherwise, there is no duplicate.</p>
<p dir="auto">nlp(‘Welcome to Dallas’) -&gt; duplicate 'Dallas' because 'Dallas' is the only named entity<br>
nlp(‘HuggingFace is not located in Dallas’) -&gt; no duplicate because there are multiple entities and the final one 'Dallas' is not tokenized into multiple tokens<br>
nlp(‘HuggingFace is located in New York City’) -&gt; duplicate ‘New York City’ because the final named entity 'New York City' is tokenized into multiple tokens</p>
<h2 dir="auto">Environment info</h2>
<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 3.0.2</li>
<li>Platform: Linux-5.3.0-1031-azure-x86_64-with-glibc2.10</li>
<li>Python version: 3.8.1</li>
<li>PyTorch version (GPU?): 1.5.1 (False)</li>
<li>Tensorflow version (GPU?): not installed (NA)</li>
<li>Using GPU in script?: no</li>
<li>Using distributed or parallel set-up in script?: no</li>
</ul>