<h1 dir="auto"><g-emoji class="g-emoji" alias="star2" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f31f.png">🌟</g-emoji> New model addition</h1>
<p dir="auto">MobileBERT</p>
<h2 dir="auto">Model description</h2>
<p dir="auto">MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks.</p>
<h2 dir="auto">Open source status</h2>
<ul class="contains-task-list">
<li class="task-list-item">
<p dir="auto"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the model implementation is available: (give details)<br>
<a href="https://github.com/google-research/google-research/tree/master/mobilebert">https://github.com/google-research/google-research/tree/master/mobilebert</a></p>
</li>
<li class="task-list-item">
<p dir="auto"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the model weights are available: (give details)<br>
<a href="https://storage.googleapis.com/cloud-tpu-checkpoints/mobilebert/uncased_L-24_H-128_B-512_A-4_F-4_OPT.tar.gz" rel="nofollow">https://storage.googleapis.com/cloud-tpu-checkpoints/mobilebert/uncased_L-24_H-128_B-512_A-4_F-4_OPT.tar.gz</a></p>
</li>
<li class="task-list-item">
<p dir="auto"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> who are the authors: (mention them, if possible by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gh-username/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gh-username">@gh-username</a>)<br>
Google LLC<br>
Xiaodan Song<br>
Zhiqing Sun<br>
Hongkun Yu<br>
Denny Zou</p>
</li>
</ul>
<h1 dir="auto">🐛 Bug</h1>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (Bert, XLNet ...): No specified</p>
<p dir="auto">Language I am using the model on (English, Chinese ...): English</p>
<p dir="auto">The problem arises when using:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the official example scripts: (give details below)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own modified scripts: (give details below)</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> an official GLUE/SQUaD task: (give the name)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>I have tried using pipeline on my own purpose, but I realized it will cause errors if I input long sentence on some tasks, it should do truncation automatically, but it does not. And the pipeline function does not take extra argument so we cannot add something like <code class="notranslate">truncation=True</code>. Here is an example on sentiment-analysis task:</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from transformers import pipeline
nlp  = pipeline('sentiment-analysis')
text = &quot;This is an example&quot;*300
nlp(text)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">transformers</span> <span class="pl-k">import</span> <span class="pl-s1">pipeline</span>
<span class="pl-s1">nlp</span>  <span class="pl-c1">=</span> <span class="pl-en">pipeline</span>(<span class="pl-s">'sentiment-analysis'</span>)
<span class="pl-s1">text</span> <span class="pl-c1">=</span> <span class="pl-s">"This is an example"</span><span class="pl-c1">*</span><span class="pl-c1">300</span>
<span class="pl-en">nlp</span>(<span class="pl-s1">text</span>)</pre></div>
<ol start="2" dir="auto">
<li>The error message is below:</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
&lt;ipython-input-31-0a8ce849cc29&gt; in &lt;module&gt;()
      2 nlp  = pipeline('sentiment-analysis')
      3 text = &quot;This is an example&quot;*300
----&gt; 4 nlp(text)

11 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1722         # remove once script supports set_grad_enabled
   1723         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-&gt; 1724     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1725 
   1726 

IndexError: index out of range in self"><pre class="notranslate"><code class="notranslate">---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
&lt;ipython-input-31-0a8ce849cc29&gt; in &lt;module&gt;()
      2 nlp  = pipeline('sentiment-analysis')
      3 text = "This is an example"*300
----&gt; 4 nlp(text)

11 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1722         # remove once script supports set_grad_enabled
   1723         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-&gt; 1724     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1725 
   1726 

IndexError: index out of range in self
</code></pre></div>
<ol start="3" dir="auto">
<li>same error on NER task, maybe some other task as well.</li>
</ol>

<h2 dir="auto">Expected behavior</h2>

<h2 dir="auto">Environment info</h2>

<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 3.0.2</li>
<li>Platform: Google Colab</li>
<li>Python version:  Python 3.6.9</li>
<li>PyTorch version (GPU?):1.5.1+cu101, no GPU</li>
<li>Tensorflow version (GPU?): no</li>
<li>Using GPU in script?: no</li>
<li>Using distributed or parallel set-up in script?: no</li>
</ul>