<h1 dir="auto">What I'm doing</h1>
<p dir="auto">Trying to estimate the emotion of tweets using <a href="https://huggingface.co/cardiffnlp/twitter-roberta-base-emotion" rel="nofollow">https://huggingface.co/cardiffnlp/twitter-roberta-base-emotion</a></p>
<h1 dir="auto">The error</h1>
<p dir="auto"><code class="notranslate">RuntimeError: The expanded size of the tensor (601) must match the existing size (514) at non-singleton dimension 1.  Target sizes: [1, 601].  Tensor sizes: [1, 514]</code></p>
<p dir="auto">Does anyone what the problem might be? I tried <code class="notranslate">truncating=True</code> as well</p>
<h1 dir="auto">Code and data to reproduce</h1>
<h3 dir="auto">View only</h3>
<p dir="auto"><a href="https://deepnote.com/project/Code-to-reproduce-error-RuntimeError-The-expanded-size-of-the-tensor-601-must-match-the-existing-size-514-at-non-singleton-dimension-1-ZfBSsqbKQ7-XWrir593tKQ/%2Fnotebook.ipynb" rel="nofollow">https://deepnote.com/project/Code-to-reproduce-error-RuntimeError-The-expanded-size-of-the-tensor-601-must-match-the-existing-size-514-at-non-singleton-dimension-1-ZfBSsqbKQ7-XWrir593tKQ/%2Fnotebook.ipynb</a></p>
<h3 dir="auto">Interactive (can run and/or make changes)</h3>
<p dir="auto"><a href="https://deepnote.com/project/Interactive-Code-to-reproduce-error-RuntimeError-The-expanded-size-of-the-tensor-601-must-match-the-existing-size-514-at-non-singleton-dimension-1-Duplicate-qJTy9jxRTPWhXhwytQjU4Q/%2Fnotebook.ipynb" rel="nofollow">https://deepnote.com/project/Interactive-Code-to-reproduce-error-RuntimeError-The-expanded-size-of-the-tensor-601-must-match-the-existing-size-514-at-non-singleton-dimension-1-Duplicate-qJTy9jxRTPWhXhwytQjU4Q/%2Fnotebook.ipynb</a></p>
<h3 dir="auto">Environment</h3>
<p dir="auto">"Deepnote projects run in containers on Debian Buster with Python 3.7"</p>
<h3 dir="auto">System Info</h3>
<p dir="auto">transformers==4.20.1<br>
torch==1.11.0+cu113<br>
Python 3.9.13</p>
<h3 dir="auto">Who can help?</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/patrickvonplaten/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/patrickvonplaten">@patrickvonplaten</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Narsil/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Narsil">@Narsil</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ola13/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ola13">@ola13</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gante/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gante">@gante</a></p>
<h3 dir="auto">Information</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> The official example scripts</li>
</ul>
<h3 dir="auto">Reproduction</h3>
<p dir="auto">Hello everyone,</p>
<p dir="auto">I am using BART and I have enabled <code class="notranslate">return_scores = True</code> with <code class="notranslate">beam_size = 4</code>.<br>
The shape of the Scores vector is <code class="notranslate">(seq_len, batch_size * beam_size * num_returned sequence, vocab_size)</code>.</p>
<p dir="auto">I would like to know how I can subdivide the vector obtaining the shape<br>
<code class="notranslate">(seq_len, batch_size, beam_size, num_returned_sequence, vocab_size)</code>.</p>
<p dir="auto">Because at the moment it is not possible to map the input sentences in a batch with the output.</p>
<p dir="auto">Kind regards,<br>
Andrea</p>
<p dir="auto">ps: I also opened a <a href="https://discuss.huggingface.co/t/strange-shape-of-scores-vector/24780" rel="nofollow">blog post</a>, I did not know which was the most appropriate place, sorry for the duplicate.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="outputs = model.generate(
            input_ids=source_ids,
            max_length = 500,
            return_dict_in_generate=True, 
            output_scores=True, 
            num_beams=4
        )

print(len(outputs.scores)) # seq_len
print(outputs.scores[0].shape) # (batch_size * beam_size * num_returned sequence, vocab_size)"><pre class="notranslate"><span class="pl-s1">outputs</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-en">generate</span>(
            <span class="pl-s1">input_ids</span><span class="pl-c1">=</span><span class="pl-s1">source_ids</span>,
            <span class="pl-s1">max_length</span> <span class="pl-c1">=</span> <span class="pl-c1">500</span>,
            <span class="pl-s1">return_dict_in_generate</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, 
            <span class="pl-s1">output_scores</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, 
            <span class="pl-s1">num_beams</span><span class="pl-c1">=</span><span class="pl-c1">4</span>
        )

<span class="pl-en">print</span>(<span class="pl-en">len</span>(<span class="pl-s1">outputs</span>.<span class="pl-s1">scores</span>)) <span class="pl-c"># seq_len</span>
<span class="pl-en">print</span>(<span class="pl-s1">outputs</span>.<span class="pl-s1">scores</span>[<span class="pl-c1">0</span>].<span class="pl-s1">shape</span>) <span class="pl-c"># (batch_size * beam_size * num_returned sequence, vocab_size)</span></pre></div>
<h3 dir="auto">Expected behavior</h3>
<p dir="auto">a score vector with a shape of (seq_len, batch_size, beam_size, num_returned_sequence, vocab_size)</p>