<h2 dir="auto">Environment info</h2>
<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 3.3.1</li>
<li>Platform: Linux-4.15.0-38-generic-x86_64-with-Ubuntu-18.04-bionic</li>
<li>Python version: 3.6.9</li>
<li>PyTorch version (GPU?): 1.6.0 (True)</li>
<li>Tensorflow version (GPU?): not installed (NA)</li>
<li>Using GPU in script?: True</li>
<li>Using distributed or parallel set-up in script?: True (Retriever is distributed)</li>
</ul>
<h3 dir="auto">Who can help</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/patrickvonplaten/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/patrickvonplaten">@patrickvonplaten</a>, <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/lhoestq/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/lhoestq">@lhoestq</a></p>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using (Bert, XLNet ...):</p>
<p dir="auto"><strong>facebook/rag-sequence-base</strong></p>
<p dir="auto">The problem arises when using:</p>
<ul dir="auto">
<li>[x ] the official example scripts: (give details below)<br>
examples/rag/finetune.sh</li>
</ul>
<p dir="auto">The tasks I am working on is:</p>
<ul dir="auto">
<li>[x ] my own task or dataset: (give details below)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<p dir="auto">run <code class="notranslate">sh finetune.sh</code><br>
with</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="DATA_DIR=data_dir
OUTPUT_DIR=output_dir
MODEL_NAME_OR_PATH=&quot;facebook/rag-sequence-base&quot;"><pre class="notranslate"><code class="notranslate">DATA_DIR=data_dir
OUTPUT_DIR=output_dir
MODEL_NAME_OR_PATH="facebook/rag-sequence-base"
</code></pre></div>
<p dir="auto">gives:</p>
<p dir="auto"><strong>Model name 'facebook/rag-sequence-base/question_encoder_tokenizer' not found in model shortcut name list (facebook/dpr-question_encoder-single-nq-base). Assuming 'facebook/rag-sequence-base/question_encoder_tokenizer' is a path, a model identifier, or url to a directory containing tokenizer files</strong>.<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/vocab.txt" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/vocab.txt</a> from cache at /h/asabet/.cache/torch/transformers/14d599f015518cd5b95b5d567b8c06b265dbbf04047e44b3654efd7cbbacb697.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/added_tokens.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/added_tokens.json</a> from cache at None<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/special_tokens_map.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/special_tokens_map.json</a> from cache at /h/asabet/.cache/torch/transformers/70614c7a84151409876eaaaecb3b5185213aa5c560926855e35753b9909f1116.275045728fbf41c11d3dae08b8742c054377e18d92cc7b72b6351152a99b64e4<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/tokenizer_config.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/tokenizer_config.json</a> from cache at /h/asabet/.cache/torch/transformers/8ade9cf561f8c0a47d1c3785e850c57414d776b3795e21bd01e58483399d2de4.11f57497ee659e26f830788489816dbcb678d91ae48c06c50c9dc0e4438ec05b<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/tokenizer.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/question_encoder_tokenizer/tokenizer.json</a> from cache at None<br>
<strong>Model name 'facebook/rag-sequence-base/generator_tokenizer' not found in model shortcut name list (facebook/bart-base, facebook/bart-large, facebook/bart-large-mnli, facebook/bart-large-cnn, facebook/bart-large-xsum, yjernite/bart_eli5). Assuming 'facebook/rag-sequence-base/generator_tokenizer' is a path, a model identifier, or url to a directory containing tokenizer files.</strong><br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/vocab.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/vocab.json</a> from cache at /h/asabet/.cache/torch/transformers/3b9637b6eab4a48cf2bc596e5992aebb74de6e32c9ee660a27366a63a8020557.6a4061e8fc00057d21d80413635a86fdcf55b6e7594ad9e25257d2f99a02f4be<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/merges.txt" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/merges.txt</a> from cache at /h/asabet/.cache/torch/transformers/b2a6adcb3b8a4c39e056d80a133951b99a56010158602cf85dee775936690c6a.70bec105b4158ed9a1747fea67a43f5dee97855c64d62b6ec3742f4cfdb5feda<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/added_tokens.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/added_tokens.json</a> from cache at None<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/special_tokens_map.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/special_tokens_map.json</a> from cache at /h/asabet/.cache/torch/transformers/342599872fb2f45f954699d3c67790c33b574cc552a4b433fedddc97e6a3c58e.6e217123a3ada61145de1f20b1443a1ec9aac93492a4bd1ce6a695935f0fd97a<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/tokenizer_config.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/tokenizer_config.json</a> from cache at /h/asabet/.cache/torch/transformers/e5f72dc4c0b1ba585d7afb7fa5e3e52ff0e1f101e49572e2caaf38fab070d4d6.d596a549211eb890d3bb341f3a03307b199bc2d5ed81b3451618cbcb04d1f1bc<br>
loading file <a href="https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/tokenizer.json" rel="nofollow">https://s3.amazonaws.com/models.huggingface.co/bert/facebook/rag-sequence-base/generator_tokenizer/tokenizer.json</a> from cache at None<br>
Traceback (most recent call last):<br>
File "finetune.py", line 499, in <br>
main(args)<br>
File "finetune.py", line 439, in main<br>
model: GenerativeQAModule = GenerativeQAModule(args)<br>
File "finetune.py", line 105, in <strong>init</strong><br>
retriever = RagPyTorchDistributedRetriever.from_pretrained(hparams.model_name_or_path, config=config)<br>
File "/h/asabet/.local/lib/python3.6/site-packages/transformers/retrieval_rag.py", line 308, in from_pretrained<br>
config, question_encoder_tokenizer=question_encoder_tokenizer, generator_tokenizer=generator_tokenizer<br>
File "/scratch/ssd001/home/asabet/transformers/examples/rag/distributed_retriever.py", line 41, in <strong>init</strong><br>
index=index,<br>
<strong>TypeError: init() got an unexpected keyword argument 'index'</strong></p>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">finetune.sh should launch and run</p>
<h1 dir="auto"><g-emoji class="g-emoji" alias="star2" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f31f.png">🌟</g-emoji>New model addition</h1>
<h2 dir="auto">Model description</h2>
<p dir="auto">method for pre-training seq2seq models by de-noising text. BART outperforms previous work on a bunch of generation tasks (summarization/dialogue/QA), while getting similar performance to RoBERTa on SQuAD/GLUE</p>
<h2 dir="auto">Open Source status</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the model implementation is available: not yet</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> the model weights are available: not yet</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> who are the authors: <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/yinhanliu/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/yinhanliu">@yinhanliu</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ernamangoyal/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ernamangoyal">@ernamangoyal</a></li>
</ul>