<p dir="auto">Two first points of the translation process are duplicating two first points of summarization process:<br>
<a href="https://huggingface.co/transformers/task_summary.html#translation" rel="nofollow">https://huggingface.co/transformers/task_summary.html#translation</a></p>
<blockquote>
<ol dir="auto">
<li>Instantiate a tokenizer and a model from the checkpoint name. Summarization is usually done using an encoder-decoder model, such as Bart or T5.</li>
<li>Define the article that should be summarized.</li>
</ol>
</blockquote>
<h2 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji></h2>
<p dir="auto">Use Pytorch's own attention and transformer modules.</p>
<h2 dir="auto">Motivation</h2>
<p dir="auto">Pytorch now offers modules like <a href="https://pytorch.org/docs/stable/nn.html?highlight=attention#torch.nn.MultiheadAttention" rel="nofollow">nn.MultiheadAttention</a> and <a href="https://pytorch.org/docs/stable/nn.html#transformer-layers" rel="nofollow">nn.Transformer</a>. It would be nice to use the official Pytorch implementations in <code class="notranslate">transformers</code> now that they are available.</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">There is an offical Pytorch <a href="https://pytorch.org/tutorials/beginner/transformer_tutorial.html" rel="nofollow">tutorial</a> that shows how nn.Transformer can be used and customized.</p>
<p dir="auto">These modules are only available in Pytorch 1.1 (<code class="notranslate">nn.MultiHeadAttention</code>) and 1.2 (<code class="notranslate">nn.Transformer</code>). Using them would mean that anyone with Pytorch 1.0 would have to update their own version.</p>