<h3 dir="auto">Feature request</h3>
<p dir="auto">Thank you for the awesome framework!<br>
For my work I wanted to use <code class="notranslate">transformers.pipelines.token_classification.TokenClassificationPipeline</code> in batch mode, since it is much faster on GPU, but I wanted to keep all the functionality for grouping entities.<br>
So I want to suggest something like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="nlp = pipeline(&quot;ner&quot;, model=model, 
               tokenizer=tokenizer,
               device = 0 if torch.cuda.is_available() else -1,
               aggregation_strategy=&quot;average&quot;, batch_size=16)"><pre class="notranslate"><code class="notranslate">nlp = pipeline("ner", model=model, 
               tokenizer=tokenizer,
               device = 0 if torch.cuda.is_available() else -1,
               aggregation_strategy="average", batch_size=16)
</code></pre></div>
<h3 dir="auto">Motivation</h3>
<p dir="auto">I implemented it for myself and think it would be cool to have this functionality "out-of-the-box" for community to enjoy the speed up. (And it really gives a huge speed up)</p>
<h3 dir="auto">Your contribution</h3>
<p dir="auto">I am willing to contribute and implement this change for TokenClassification task (also for TextClassification, FeatureExtraction should be pretty much same). Have not worked with other pipelines, so not sure how batching is implemented there, but I am willing to try and contribute.</p>
<h3 dir="auto">System Info</h3>
<p dir="auto">python: 3.8<br>
transformers: 4.23.1</p>
<h3 dir="auto">Who can help?</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/patrickvonplaten/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/patrickvonplaten">@patrickvonplaten</a></p>
<h3 dir="auto">Information</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> The official example scripts</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> My own modified scripts</li>
</ul>
<h3 dir="auto">Tasks</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> An officially supported task in the <code class="notranslate">examples</code> folder (such as GLUE/SQuAD, ...)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> My own task or dataset (give details below)</li>
</ul>
<h3 dir="auto">Reproduction</h3>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from transformers import Wav2Vec2Tokenizer, Wav2Vec2CTCTokenizer


def main() -&gt; None:
    # [NOTE]: plsace insert save_dir path
    output_dir = r&quot;./output_dir&quot;

    encoder1_tokenizer = Wav2Vec2CTCTokenizer.from_pretrained(&quot;facebook/wav2vec2-base-960h&quot;)
    encoder2_tokenizer = Wav2Vec2Tokenizer.from_pretrained(&quot;facebook/wav2vec2-base-960h&quot;)

    encoder1_tokenizer.vocab_files_name = {
        &quot;vocab_file&quot;: &quot;encoder1_vocab.json&quot;,
        &quot;tokenizer_config_file&quot;: &quot;encoder1_tokenizer_config.json&quot;,
    }
    encoder2_tokenizer.vocab_files_name = {
        &quot;vocab_file&quot;: &quot;encoder2_vocab.json&quot;,
        &quot;tokenizer_config_file&quot;: &quot;encoder2_tokenizer_config.json&quot;,
    }

    encoder1_tokenizer.save_vocabulary(save_diretory=output_dir)
    encoder2_tokenizer.save_vocabulary(save_diretory=output_dir)

if &quot;__main__&quot; in __name__:
    main()"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">transformers</span> <span class="pl-k">import</span> <span class="pl-v">Wav2Vec2Tokenizer</span>, <span class="pl-v">Wav2Vec2CTCTokenizer</span>


<span class="pl-k">def</span> <span class="pl-en">main</span>() <span class="pl-c1">-&gt;</span> <span class="pl-c1">None</span>:
    <span class="pl-c"># [NOTE]: plsace insert save_dir path</span>
    <span class="pl-s1">output_dir</span> <span class="pl-c1">=</span> <span class="pl-s">r"./output_dir"</span>

    <span class="pl-s1">encoder1_tokenizer</span> <span class="pl-c1">=</span> <span class="pl-v">Wav2Vec2CTCTokenizer</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"facebook/wav2vec2-base-960h"</span>)
    <span class="pl-s1">encoder2_tokenizer</span> <span class="pl-c1">=</span> <span class="pl-v">Wav2Vec2Tokenizer</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"facebook/wav2vec2-base-960h"</span>)

    <span class="pl-s1">encoder1_tokenizer</span>.<span class="pl-s1">vocab_files_name</span> <span class="pl-c1">=</span> {
        <span class="pl-s">"vocab_file"</span>: <span class="pl-s">"encoder1_vocab.json"</span>,
        <span class="pl-s">"tokenizer_config_file"</span>: <span class="pl-s">"encoder1_tokenizer_config.json"</span>,
    }
    <span class="pl-s1">encoder2_tokenizer</span>.<span class="pl-s1">vocab_files_name</span> <span class="pl-c1">=</span> {
        <span class="pl-s">"vocab_file"</span>: <span class="pl-s">"encoder2_vocab.json"</span>,
        <span class="pl-s">"tokenizer_config_file"</span>: <span class="pl-s">"encoder2_tokenizer_config.json"</span>,
    }

    <span class="pl-s1">encoder1_tokenizer</span>.<span class="pl-en">save_vocabulary</span>(<span class="pl-s1">save_diretory</span><span class="pl-c1">=</span><span class="pl-s1">output_dir</span>)
    <span class="pl-s1">encoder2_tokenizer</span>.<span class="pl-en">save_vocabulary</span>(<span class="pl-s1">save_diretory</span><span class="pl-c1">=</span><span class="pl-s1">output_dir</span>)

<span class="pl-k">if</span> <span class="pl-s">"__main__"</span> <span class="pl-c1">in</span> <span class="pl-s1">__name__</span>:
    <span class="pl-en">main</span>()</pre></div>
<h3 dir="auto">Expected behavior</h3>
<p dir="auto">This is the problem that occurred when the bi-encoder had to be placed on Wav2Vec2.<br>
When i saving the model<br>
To prevent overwriting due to duplication when saving in tokenizer, i may have named files that are saved differently.</p>
<p dir="auto">I overridden vocab_files_name as shown in Reproduction.</p>
<p dir="auto">But later I found out that the file hasn't been renamed</p>
<p dir="auto">So I looked up the problem and found the following problem</p>
<p dir="auto">tokenization_wav2vec2.py &gt; <a href="https://github.com/huggingface/transformers/blob/v4.24.0/src/transformers/models/wav2vec2/tokenization_wav2vec2.py#L127-L161">L127-161</a></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="class Wav2Vec2CTCTokenizer(PreTrainedTokenizer):

    &quot;&quot;&quot;
    Constructs a Wav2Vec2CTC tokenizer.
    This tokenizer inherits from [`PreTrainedTokenizer`] which contains some of the main methods. Users should refer to
    the superclass for more information regarding such methods.
    Args:
        vocab_file (`str`):
            File containing the vocabulary.
        bos_token (`str`, *optional*, defaults to `&quot;&lt;s&gt;&quot;`):
            The beginning of sentence token.
        eos_token (`str`, *optional*, defaults to `&quot;&lt;/s&gt;&quot;`):
            The end of sentence token.
        unk_token (`str`, *optional*, defaults to `&quot;&lt;unk&gt;&quot;`):
            The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
            token instead.
        pad_token (`str`, *optional*, defaults to `&quot;&lt;pad&gt;&quot;`):
            The token used for padding, for example when batching sequences of different lengths.
        word_delimiter_token (`str`, *optional*, defaults to `&quot;|&quot;`):
            The token used for defining the end of a word.
        do_lower_case (`bool`, *optional*, defaults to `False`):
            Whether or not to accept lowercase input and lowercase the output when decoding.
        **kwargs
            Additional keyword arguments passed along to [`PreTrainedTokenizer`]
    &quot;&quot;&quot;

    vocab_files_names = VOCAB_FILES_NAMES # [NOTE]: here
    pretrained_vocab_files_map = PRETRAINED_VOCAB_FILES_MAP
    max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES
    model_input_names = [&quot;input_ids&quot;, &quot;attention_mask&quot;]

    def __init__(
"><pre class="notranslate"><span class="pl-k">class</span> <span class="pl-v">Wav2Vec2CTCTokenizer</span>(<span class="pl-v">PreTrainedTokenizer</span>):

    <span class="pl-s">"""</span>
<span class="pl-s">    Constructs a Wav2Vec2CTC tokenizer.</span>
<span class="pl-s">    This tokenizer inherits from [`PreTrainedTokenizer`] which contains some of the main methods. Users should refer to</span>
<span class="pl-s">    the superclass for more information regarding such methods.</span>
<span class="pl-s">    Args:</span>
<span class="pl-s">        vocab_file (`str`):</span>
<span class="pl-s">            File containing the vocabulary.</span>
<span class="pl-s">        bos_token (`str`, *optional*, defaults to `"&lt;s&gt;"`):</span>
<span class="pl-s">            The beginning of sentence token.</span>
<span class="pl-s">        eos_token (`str`, *optional*, defaults to `"&lt;/s&gt;"`):</span>
<span class="pl-s">            The end of sentence token.</span>
<span class="pl-s">        unk_token (`str`, *optional*, defaults to `"&lt;unk&gt;"`):</span>
<span class="pl-s">            The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this</span>
<span class="pl-s">            token instead.</span>
<span class="pl-s">        pad_token (`str`, *optional*, defaults to `"&lt;pad&gt;"`):</span>
<span class="pl-s">            The token used for padding, for example when batching sequences of different lengths.</span>
<span class="pl-s">        word_delimiter_token (`str`, *optional*, defaults to `"|"`):</span>
<span class="pl-s">            The token used for defining the end of a word.</span>
<span class="pl-s">        do_lower_case (`bool`, *optional*, defaults to `False`):</span>
<span class="pl-s">            Whether or not to accept lowercase input and lowercase the output when decoding.</span>
<span class="pl-s">        **kwargs</span>
<span class="pl-s">            Additional keyword arguments passed along to [`PreTrainedTokenizer`]</span>
<span class="pl-s">    """</span>

    <span class="pl-s1">vocab_files_names</span> <span class="pl-c1">=</span> <span class="pl-v">VOCAB_FILES_NAMES</span> <span class="pl-c"># [NOTE]: here</span>
    <span class="pl-s1">pretrained_vocab_files_map</span> <span class="pl-c1">=</span> <span class="pl-v">PRETRAINED_VOCAB_FILES_MAP</span>
    <span class="pl-s1">max_model_input_sizes</span> <span class="pl-c1">=</span> <span class="pl-v">PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES</span>
    <span class="pl-s1">model_input_names</span> <span class="pl-c1">=</span> [<span class="pl-s">"input_ids"</span>, <span class="pl-s">"attention_mask"</span>]

    <span class="pl-k">def</span> <span class="pl-s1">__init__</span>(</pre></div>
<p dir="auto">VOCAB_FILES_NAMES sets the default value for vocab_files_names</p>
<p dir="auto">tokenization_wav2vec2.py &gt; <a href="https://github.com/huggingface/transformers/blob/v4.24.0/src/transformers/models/wav2vec2/tokenization_wav2vec2.py#L595-L606">L595-606</a></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="    def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None) -&gt; Tuple[str]:
        if not os.path.isdir(save_directory):
            logger.error(f&quot;Vocabulary path ({save_directory}) should be a directory&quot;)
            return
        vocab_file = os.path.join(
            save_directory, (filename_prefix + &quot;-&quot; if filename_prefix else &quot;&quot;) + VOCAB_FILES_NAMES[&quot;vocab_file&quot;] # [NOTE]: here
        )

        with open(vocab_file, &quot;w&quot;, encoding=&quot;utf-8&quot;) as f:
            f.write(json.dumps(self.encoder, indent=2, sort_keys=True, ensure_ascii=False) + &quot;\n&quot;)

        return (vocab_file,)"><pre class="notranslate">    <span class="pl-k">def</span> <span class="pl-en">save_vocabulary</span>(<span class="pl-s1">self</span>, <span class="pl-s1">save_directory</span>: <span class="pl-s1">str</span>, <span class="pl-s1">filename_prefix</span>: <span class="pl-v">Optional</span>[<span class="pl-s1">str</span>] <span class="pl-c1">=</span> <span class="pl-c1">None</span>) <span class="pl-c1">-&gt;</span> <span class="pl-v">Tuple</span>[<span class="pl-s1">str</span>]:
        <span class="pl-k">if</span> <span class="pl-c1">not</span> <span class="pl-s1">os</span>.<span class="pl-s1">path</span>.<span class="pl-en">isdir</span>(<span class="pl-s1">save_directory</span>):
            <span class="pl-s1">logger</span>.<span class="pl-en">error</span>(<span class="pl-s">f"Vocabulary path (<span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">save_directory</span><span class="pl-kos">}</span></span>) should be a directory"</span>)
            <span class="pl-k">return</span>
        <span class="pl-s1">vocab_file</span> <span class="pl-c1">=</span> <span class="pl-s1">os</span>.<span class="pl-s1">path</span>.<span class="pl-en">join</span>(
            <span class="pl-s1">save_directory</span>, (<span class="pl-s1">filename_prefix</span> <span class="pl-c1">+</span> <span class="pl-s">"-"</span> <span class="pl-k">if</span> <span class="pl-s1">filename_prefix</span> <span class="pl-k">else</span> <span class="pl-s">""</span>) <span class="pl-c1">+</span> <span class="pl-v">VOCAB_FILES_NAMES</span>[<span class="pl-s">"vocab_file"</span>] <span class="pl-c"># [NOTE]: here</span>
        )

        <span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s1">vocab_file</span>, <span class="pl-s">"w"</span>, <span class="pl-s1">encoding</span><span class="pl-c1">=</span><span class="pl-s">"utf-8"</span>) <span class="pl-k">as</span> <span class="pl-s1">f</span>:
            <span class="pl-s1">f</span>.<span class="pl-en">write</span>(<span class="pl-s1">json</span>.<span class="pl-en">dumps</span>(<span class="pl-s1">self</span>.<span class="pl-s1">encoder</span>, <span class="pl-s1">indent</span><span class="pl-c1">=</span><span class="pl-c1">2</span>, <span class="pl-s1">sort_keys</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">ensure_ascii</span><span class="pl-c1">=</span><span class="pl-c1">False</span>) <span class="pl-c1">+</span> <span class="pl-s">"<span class="pl-cce">\n</span>"</span>)

        <span class="pl-k">return</span> (<span class="pl-s1">vocab_file</span>,)</pre></div>
<p dir="auto">However, if you look at the save_vocabulary function, "vocab_files_names" is not included in the checked part, but "VOCAB_FILES_NAMES" is included, so the file name does not change even if you override "vocab_files_names"</p>
<p dir="auto">So I want to change that "VOCAB_FILES_NAMES" part to "vocab_files_names"<br>
And this was on Wav2Vec2CTCTokenizer and Wav2Vec2Tokenizer</p>