<p dir="auto">In our research group we all have user accounts on a server where we each run our own experiments (Ubuntu behind the scenes). By default, everyone is downloading <code class="notranslate">transformers</code> models to their own home directory. Let's say we have 20 researchers, that might mean that we have 20 duplicates of "bert-base-cased" on the server (and of many other models). This is not efficient at all and takes too much room to our liking.</p>
<p dir="auto">We have tried creating a 777 directory as TRANSFORMERS_CACHE globally, but that does not work. If I download a model, some of the downloaded files get a read/write access for me as the creator of the file. This means that others cannot use the model (permission denied).</p>
<p dir="auto">Our suggestion or request would be to have an option when downloading a model to also set its permissions for all downloaded files. Preferably adjustable via a (system-wide) environment variable. This would probably need to be added in file_utils.py, similar to other options like "local_files_only".</p>
<p dir="auto">I currently do not have time to work on this myself, but I am open to any feedback of course.</p>
<h2 dir="auto">EDIT: <code class="notranslate">DataCollatorWithPadding</code> already has the requested feature</h2>
<h1 dir="auto"><g-emoji class="g-emoji" alias="rocket" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f680.png">🚀</g-emoji> Feature request</h1>

<p dir="auto">A <code class="notranslate">DataCollatorWithPadding</code> class that can pad <code class="notranslate">attention_mask</code> with any tokenizer class compatible with 🤗Transformers.</p>
<h2 dir="auto">Motivation</h2>
<p dir="auto">It is common to have training examples with both <code class="notranslate">input_ids</code> and <code class="notranslate">attention_mask</code>. For batches of size larger than 1, you need to pad the examples to form congruent tensors. However, <a href="https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding" rel="nofollow">DataCollatorWithPadding</a> pads <code class="notranslate">input_ids</code> but not <code class="notranslate">attention_mask</code> and there is apparently no way to instruct it to do so.</p>

<h2 dir="auto">Your contribution</h2>
<p dir="auto">The method <code class="notranslate">__call__</code> from <code class="notranslate">DataCollatorWithPadding</code> (<a href="https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/data/data_collator.py#L246">code</a>) is a wrapper for <code class="notranslate">self.tokenizer.pad</code>. For tokenizers of the class <a href="https://huggingface.co/docs/transformers/internal/tokenization_utils#transformers.PreTrainedTokenizerBase" rel="nofollow">PreTrainedTokenizerBase</a>, the <code class="notranslate">pad</code> method (<a href="https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/tokenization_utils_base.py#L2683">code</a>) has an argument <code class="notranslate">return_attention_mask</code> that cannot be forwarded from the <code class="notranslate">__call__</code> method of DataCollatorWithPadding.</p>
<p dir="auto">Perhaps an easy way to implement the issue without breaking everything is to add to DataCollatorWithPadding an attribute <code class="notranslate">pad_attention_mask</code> (maybe set to <code class="notranslate">False</code> for compatibility with the current code) and pass <code class="notranslate">return_attention_mask=self.pad_attention_mask</code> in the call to <code class="notranslate">self.tokenizer.pad</code>.</p>
<p dir="auto">The current docstring of the <code class="notranslate">pad</code> method from PreTrainedTokenizerBase is quite confusing (see <a href="https://github.com/huggingface/transformers/blob/db7d6a80e82d66127b2a44b6e3382969fdc8b207/src/transformers/tokenization_utils_base.py#L2740">here</a>). It refers to the <code class="notranslate">return_outputs</code> attribute, which is not defined anywhere and is not used anywhere in the code. It is perhaps an obsolete feature and I suppose that the issue arose when it was removed.</p>
<p dir="auto">I could submit a PR, but I suppose that this is best brought to the attention of <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a>. If not, sorry for bothering you Sylvain.</p>