<p dir="auto">This is not urgent. There is a ton of deprecation warnings across many modules with pytorch-1.7+ and a few with python-3.8:<br>
(I hard-wrapped the lines to avoid the need to scroll, but it makes somewhat harder to see the warnings):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working from collections import Sequence

tests/test_logging.py::HfArgumentParserTest::test_integration
  tests/test_logging.py:40:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(msg)

tests/test_logging.py::HfArgumentParserTest::test_integration
  tests/test_logging.py:48:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(msg)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:164:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / (float(v.size(-1)) ** 0.5)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:169:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! mask = self.bias[:, :, ns - nd : ns, :ns]

tests/test_modeling_auto.py::AutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_auto.py::AutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_auto.py:821:
FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed
in a future version. Please use `AutoModelForCausalLM` for causal language
models, `AutoModelForMaskedLM` for masked language models and
`AutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs
tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:432:
UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape.
This may consume a large amount of memory. warnings.warn(

tests/test_modeling_albert.py::AlbertModelTest::test_torchscript
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_attentions
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_albert.py:229:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_albert.py: 3 warnings tests/test_modeling_bert.py: 3
warnings tests/test_modeling_bert_generation.py: 3 warnings
tests/test_modeling_distilbert.py: 2 warnings tests/test_modeling_dpr.py: 3
warnings tests/test_modeling_flaubert.py: 3 warnings
tests/test_modeling_electra.py: 3 warnings tests/test_modeling_layoutlm.py: 3
warnings tests/test_modeling_roberta.py: 3 warnings tests/test_modeling_xlm.py:
3 warnings tests/test_modeling_xlnet.py: 3 warnings
src/transformers/modeling_utils.py:1670:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! input_tensor.shape == tensor_shape for input_tensor
in input_tensors

tests/test_modeling_bert_generation.py: 32 warnings
  src/transformers/modeling_bert_generation.py:417:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(&quot;If you want to use `BertGenerationDecoder` as a standalone, add
  `is_decoder=True.`&quot;)

tests/test_modeling_bert.py::BertModelTest::test_torchscript
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_attentions
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_attentions
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert.py:191:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:175:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if decoder_padding_mask is not None and
decoder_padding_mask.shape[1] &gt; 1:

tests/test_modeling_bart.py: 3 warnings tests/test_modeling_flaubert.py: 3
warnings tests/test_modeling_fsmt.py: 3 warnings tests/test_modeling_roberta.py:
3 warnings tests/test_modeling_xlm.py: 3 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/functional.py:1836:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert padding_idx &lt; weight.size(0), 'Padding_idx
must be within num_embeddings'

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:720:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.shape == (bsz, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:722:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:740:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:287:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if torch.isinf(x).any() or torch.isnan(x).any():

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:1190:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! if len(torch.unique(eos_mask.sum(1))) &gt; 1:

tests/test_modeling_common.py::UtilsFunctionsTest::test_top_k_top_p_filtering
  tests/test_modeling_common.py:1196:
  UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
  one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
  internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  non_inf_idx = (output != -float(&quot;inf&quot;)).nonzero().to(device=torch_device)

tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_attentions
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert_generation.py:156:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_flaubert.py: 14 warnings tests/test_modeling_xlm.py: 14
warnings
src/transformers/modeling_xlm.py:1220:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_flaubert.py::FlaubertModelTest::test_flaubert_lm_head
tests/test_modeling_flaubert.py::FlaubertModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_xlm_lm_head
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/_reduction.py:14:
UserWarning: reduction='elementwise_mean' is deprecated, please use
reduction='mean' instead. warnings.warn(&quot;reduction='elementwise_mean' is
deprecated, please use reduction='mean' instead.&quot;)

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:188:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:106:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert mask.size() == (bs, slen)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:1224:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if max_pos &gt; self.weight.size(0):

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:763:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert embed_dim == self.embed_dim

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:764:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert list(query.size()) == [tgt_len, bsz,
embed_dim]

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:805:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:814:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.size()[:2] == (

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:833:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_att_mask_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past_large_inputs
src/transformers/modeling_gpt2.py:530:
FutureWarning: The `past` argument is deprecated and will be removed in a future
version, use `past_key_values` instead. warnings.warn(

tests/test_modeling_electra.py::ElectraModelTest::test_torchscript
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_attentions
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_electra.py:180:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_attentions
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_layoutlm.py:87:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/tensor.py:547:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect. return
torch.tensor(other, dtype=dtype, device=self.device) ** self

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:314:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! num_remove = shift * len(pooled_pos)

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:638:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooling_flag = pooling_flag and block_index &gt; 0

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:481:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! shift = 2 if q_head.shape[1] != context_len else 1

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:431:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! positional_attn = positional_attn[..., :context_len]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:678:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! output = output[:, : target_len - 1]

tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:1058:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:467:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[None, :
input_shape[-1]]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:180:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / math.sqrt(v.size(-1))

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:183:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! b = self.bias[:, :, : w.size(-2), : w.size(-1)]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:823:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_rag.py: 12 warnings tests/test_retrieval_rag.py: 1 warning
src/transformers/tokenization_utils_base.py:613:
UserWarning: To copy construct from a tensor, it is recommended to use
sourceTensor.clone().detach() or
sourceTensor.clone().detach().requires_grad_(True), rather than
torch.tensor(sourceTensor). tensor = as_tensor(value)

tests/test_modeling_reformer.py: 58 warnings tests/test_modeling_transfo_xl.py:
18 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/modules/container.py:434:
UserWarning: Setting attributes on ParameterList is not supported.
warnings.warn(&quot;Setting attributes on ParameterList is not supported.&quot;)

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:192:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:534:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect.
torch.tensor(1000),

tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_cached_inference
  src/transformers/modeling_reformer.py:899:
  UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
  one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
  internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  relevant_bucket_idx = (bucket_idx == (bucket_idx.shape[-1] - 1)).nonzero()

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_attentions
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_utils.py:244:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if causal_mask.shape[1] &lt; attention_mask.shape[1]:

tests/test_modeling_t5.py: 95 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/utils.py:760:
  DeprecationWarning: an integer is required (got type
  torch._C._onnx.TensorProtoDataType). Implicit conversion to integers using
  __int__ is deprecated, and may be removed in a future version of Python.
  return getattr(node, kind + &quot;_&quot;)(name, value)

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:1638:
DeprecationWarning: an integer is required (got type float). Implicit conversion
to integers using __int__ is deprecated, and may be removed in a future version
of Python. value_t=torch.tensor([fill_value],
dtype=sym_help.scalar_type_to_pytorch_type[dtype]))

tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_tf_auto.py:697:
FutureWarning: The class `TFAutoModelWithLMHead` is deprecated and will be
removed in a future version. Please use `TFAutoModelForCausalLM` for causal
language models, `TFAutoModelForMaskedLM` for masked language models and
`TFAutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_attentions
tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_squeezebert.py:78:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_tf_flaubert.py: 9 warnings tests/test_modeling_tf_xlm.py: 9
warnings
src/transformers/modeling_tf_xlm.py:994:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_graph_mode
tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_graph_mode
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:493:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. return
py_builtins.overload_of(f)(*args)

tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_compile_tf_model
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_config
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_keras_save_load
tests/test_modeling_xlnet.py::XLNetModelTest::test_config
tests/test_modeling_xlnet.py::XLNetModelTest::test_correct_missing_keys
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_save_load
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_train_pipeline_custom_model
tests/test_modeling_xlnet.py::XLNetModelTest::test_save_load
src/transformers/configuration_xlnet.py:205:
FutureWarning: This config doesn't use attention memories, a core feature of
XLNet. Consider setting `mem_len` to a non-zero value, for example `xlnet =
XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024)`, for
accurate training performance as well as an order of magnitude faster inference.
Starting from version 3.5.0, the default parameter will be 1024, following the
implementation in https://arxiv.org/abs/1906.08237 warnings.warn(

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:531:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:546:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :slen]

tests/test_optimization.py::OptimizationTest::test_adafactor
  src/transformers/optimization.py:512:
  UserWarning: This overload of add_ is deprecated: add_(Number alpha, Tensor
  other) Consider using one of the following signatures instead: add_(Tensor
  other, *, Number alpha) (Triggered internally at
  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  exp_avg_sq.mul_(beta2t).add_(1.0 - beta2t, update)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:247:
  UserWarning: To get the last learning rate computed by the scheduler, please
  use `get_last_lr()`. warnings.warn(&quot;To get the last learning rate computed by
  the scheduler, &quot;

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131:
  UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.
  In PyTorch 1.1.0 and later, you should call them in the opposite order:
  `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will
  result in PyTorch skipping the first value of the learning rate schedule. See
  more details at
  https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn(&quot;Detected call of `lr_scheduler.step()` before
  `optimizer.step()`. &quot;

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:216:
  UserWarning: Please also save or load the state of the optimizer when saving
  or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:234:
  UserWarning: Please also save or load the state of the optimizer when saving
  or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_identifier_with_correct_config
tests/test_tokenization_mbart.py::MBartEnroIntegrationTest::test_batch_fairseq_parity
tests/test_tokenization_t5.py::T5TokenizationTest::test_empty_target_text
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_max_target_length
tests/test_tokenization_t5.py::T5TokenizationTest::test_outputs_not_longer_than_maxlen
tests/test_tokenization_t5.py::T5TokenizationTest::test_prepare_seq2seq_batch
src/transformers/tokenization_utils_base.py:1421:
FutureWarning: The `max_len` attribute has been deprecated and will be removed
in a future version, use `model_max_length` instead. warnings.warn(

tests/test_tokenization_albert.py: 2 warnings tests/test_tokenization_bart.py: 2
warnings tests/test_tokenization_bert.py: 2 warnings
tests/test_tokenization_bert_generation.py: 1 warning
tests/test_tokenization_bertweet.py: 1 warning
tests/test_tokenization_blenderbot.py: 1 warning
tests/test_tokenization_ctrl.py: 1 warning tests/test_tokenization_camembert.py:
2 warnings tests/test_tokenization_distilbert.py: 4 warnings
tests/test_tokenization_dpr.py: 8 warnings tests/test_tokenization_fsmt.py: 1
warning tests/test_tokenization_funnel.py: 2 warnings
tests/test_tokenization_herbert.py: 2 warnings tests/test_tokenization_gpt2.py:
1 warning tests/test_tokenization_layoutlm.py: 2 warnings
tests/test_tokenization_marian.py: 1 warning tests/test_tokenization_lxmert.py:
2 warnings tests/test_tokenization_mbart.py: 2 warnings
tests/test_tokenization_pegasus.py: 2 warnings
tests/test_tokenization_openai.py: 1 warning tests/test_tokenization_phobert.py:
1 warning tests/test_tokenization_deberta.py: 1 warning
tests/test_tokenization_prophetnet.py: 1 warning
tests/test_tokenization_reformer.py: 1 warning
tests/test_tokenization_squeezebert.py: 4 warnings
tests/test_tokenization_t5.py: 2 warnings tests/test_tokenization_roberta.py: 2
warnings tests/test_tokenization_transfo_xl.py: 1 warning
tests/test_tokenization_xlm.py: 1 warning
tests/test_tokenization_xlm_prophetnet.py: 1 warning
tests/test_tokenization_xlnet.py: 2 warnings
tests/test_tokenization_xlm_roberta.py: 2 warnings
src/transformers/tokenization_utils_base.py:2025:
FutureWarning: The `pad_to_max_length` argument is deprecated and will be
removed in a future version, use `padding=True` or `padding='longest'` to pad to
the longest sequence in the batch, or use `padding='max_length'` to pad to a max
length. In this case, you can give a specific length with `max_length` (e.g.
`max_length=45`) or leave max_length to None to pad to the maximal input size of
the model (e.g. 512 for Bert). warnings.warn(

tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_treatment
src/transformers/tokenization_t5.py:183:
UserWarning: This sequence already has &lt;/s&gt;. In future versions this behavior
may lead to duplicated eos tokens being added. warnings.warn(

tests/test_trainer.py: 44 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64:
  UserWarning: Was asked to gather along dimension 0, but all input tensors were
  scalars; will instead unsqueeze and return a vector. warnings.warn('Was asked
  to gather along dimension 0, but all '

tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training
tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/cuda/nccl.py:48:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working if not isinstance(inputs, collections.Container) or isinstance(inputs,
torch.Tensor):

-- Docs: https://docs.pytest.org/en/stable/warnings.html"><pre class="notranslate"><code class="notranslate">src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18 src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18
src/transformers/modeling_deberta.py:18:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working from collections import Sequence

tests/test_logging.py::HfArgumentParserTest::test_integration
  tests/test_logging.py:40:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(msg)

tests/test_logging.py::HfArgumentParserTest::test_integration
  tests/test_logging.py:48:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(msg)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:164:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / (float(v.size(-1)) ** 0.5)

tests/test_benchmark.py::BenchmarkTest::test_inference_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:169:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! mask = self.bias[:, :, ns - nd : ns, :ns]

tests/test_modeling_auto.py::AutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_auto.py::AutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_auto.py:821:
FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed
in a future version. Please use `AutoModelForCausalLM` for causal language
models, `AutoModelForMaskedLM` for masked language models and
`AutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_no_configs
tests/test_benchmark_tf.py::TFBenchmarkTest::test_train_with_configs
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:432:
UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape.
This may consume a large amount of memory. warnings.warn(

tests/test_modeling_albert.py::AlbertModelTest::test_torchscript
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_attentions
tests/test_modeling_albert.py::AlbertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_albert.py:229:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_albert.py: 3 warnings tests/test_modeling_bert.py: 3
warnings tests/test_modeling_bert_generation.py: 3 warnings
tests/test_modeling_distilbert.py: 2 warnings tests/test_modeling_dpr.py: 3
warnings tests/test_modeling_flaubert.py: 3 warnings
tests/test_modeling_electra.py: 3 warnings tests/test_modeling_layoutlm.py: 3
warnings tests/test_modeling_roberta.py: 3 warnings tests/test_modeling_xlm.py:
3 warnings tests/test_modeling_xlnet.py: 3 warnings
src/transformers/modeling_utils.py:1670:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! input_tensor.shape == tensor_shape for input_tensor
in input_tensors

tests/test_modeling_bert_generation.py: 32 warnings
  src/transformers/modeling_bert_generation.py:417:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn("If you want to use `BertGenerationDecoder` as a standalone, add
  `is_decoder=True.`")

tests/test_modeling_bert.py::BertModelTest::test_torchscript
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_attentions
tests/test_modeling_bert.py::BertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_attentions
tests/test_modeling_dpr.py::DPRModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert.py:191:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:175:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if decoder_padding_mask is not None and
decoder_padding_mask.shape[1] &gt; 1:

tests/test_modeling_bart.py: 3 warnings tests/test_modeling_flaubert.py: 3
warnings tests/test_modeling_fsmt.py: 3 warnings tests/test_modeling_roberta.py:
3 warnings tests/test_modeling_xlm.py: 3 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/functional.py:1836:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert padding_idx &lt; weight.size(0), 'Padding_idx
must be within num_embeddings'

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:720:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.shape == (bsz, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:722:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:740:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:287:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if torch.isinf(x).any() or torch.isnan(x).any():

tests/test_modeling_bart.py::BARTModelTest::test_torchscript
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_attentions
tests/test_modeling_bart.py::BARTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_bart.py:1190:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! if len(torch.unique(eos_mask.sum(1))) &gt; 1:

tests/test_modeling_common.py::UtilsFunctionsTest::test_top_k_top_p_filtering
  tests/test_modeling_common.py:1196:
  UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
  one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
  internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  non_inf_idx = (output != -float("inf")).nonzero().to(device=torch_device)

tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_attentions
tests/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_torchscript_output_hidden_state
src/transformers/modeling_bert_generation.py:156:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_flaubert.py: 14 warnings tests/test_modeling_xlm.py: 14
warnings
src/transformers/modeling_xlm.py:1220:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_flaubert.py::FlaubertModelTest::test_flaubert_lm_head
tests/test_modeling_flaubert.py::FlaubertModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_model_outputs_equivalence
tests/test_modeling_xlm.py::XLMModelTest::test_xlm_lm_head
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/_reduction.py:14:
UserWarning: reduction='elementwise_mean' is deprecated, please use
reduction='mean' instead. warnings.warn("reduction='elementwise_mean' is
deprecated, please use reduction='mean' instead.")

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:188:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_flaubert.py:189:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:95:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_attentions
tests/test_modeling_flaubert.py::FlaubertModelTest::test_torchscript_output_hidden_state
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:106:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert mask.size() == (bs, slen)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:1224:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if max_pos &gt; self.weight.size(0):

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:763:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert embed_dim == self.embed_dim

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:764:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert list(query.size()) == [tgt_len, bsz,
embed_dim]

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:805:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_weights.size() == (bsz * self.num_heads,
tgt_len, src_len)

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:814:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert key_padding_mask is None or
key_padding_mask.size()[:2] == (

tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_attentions
tests/test_modeling_fsmt.py::FSMTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_fsmt.py:833:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert attn_output.size() == (bsz * self.num_heads,
tgt_len, self.head_dim)

tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_att_mask_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past
tests/test_modeling_gpt2.py::GPT2ModelTest::test_gpt2_model_past_large_inputs
src/transformers/modeling_gpt2.py:530:
FutureWarning: The `past` argument is deprecated and will be removed in a future
version, use `past_key_values` instead. warnings.warn(

tests/test_modeling_electra.py::ElectraModelTest::test_torchscript
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_attentions
tests/test_modeling_electra.py::ElectraModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_electra.py:180:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_attentions
tests/test_modeling_layoutlm.py::LayoutLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_layoutlm.py:87:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/tensor.py:547:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect. return
torch.tensor(other, dtype=dtype, device=self.device) ** self

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:314:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! num_remove = shift * len(pooled_pos)

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:638:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooling_flag = pooling_flag and block_index &gt; 0

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:481:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! shift = 2 if q_head.shape[1] != context_len else 1

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelBaseModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:431:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! positional_attn = positional_attn[..., :context_len]

tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_attentions
tests/test_modeling_funnel.py::FunnelModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_funnel.py:678:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! output = output[:, : target_len - 1]

tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_attentions
tests/test_modeling_gpt2.py::GPT2ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_gpt2.py:1058:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:467:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[None, :
input_shape[-1]]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:180:
TracerWarning: Converting a tensor to a Python float might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! w = w / math.sqrt(v.size(-1))

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:183:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! b = self.bias[:, :, : w.size(-2), : w.size(-1)]

tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_attentions
tests/test_modeling_openai.py::OpenAIGPTModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_openai.py:823:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! pooled_logits = logits[range(batch_size),
sequence_lengths]

tests/test_modeling_rag.py: 12 warnings tests/test_retrieval_rag.py: 1 warning
src/transformers/tokenization_utils_base.py:613:
UserWarning: To copy construct from a tensor, it is recommended to use
sourceTensor.clone().detach() or
sourceTensor.clone().detach().requires_grad_(True), rather than
torch.tensor(sourceTensor). tensor = as_tensor(value)

tests/test_modeling_reformer.py: 58 warnings tests/test_modeling_transfo_xl.py:
18 warnings
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/modules/container.py:434:
UserWarning: Setting attributes on ParameterList is not supported.
warnings.warn("Setting attributes on ParameterList is not supported.")

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:192:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_attentions
tests/test_modeling_mobilebert.py::MobileBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_mobilebert.py:534:
TracerWarning: torch.tensor results are registered as constants in the trace.
You can safely ignore this warning if you use this function to create tensors
out of constant variables that would be the same every time you call this
function. In any other case, this might cause the trace to be incorrect.
torch.tensor(1000),

tests/test_modeling_reformer.py::ReformerLSHAttnModelTest::test_reformer_cached_inference
  src/transformers/modeling_reformer.py:899:
  UserWarning: This overload of nonzero is deprecated: nonzero() Consider using
  one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered
  internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  relevant_bucket_idx = (bucket_idx == (bucket_idx.shape[-1] - 1)).nonzero()

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_attentions
tests/test_modeling_t5.py::T5ModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_utils.py:244:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! if causal_mask.shape[1] &lt; attention_mask.shape[1]:

tests/test_modeling_t5.py: 95 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/utils.py:760:
  DeprecationWarning: an integer is required (got type
  torch._C._onnx.TensorProtoDataType). Implicit conversion to integers using
  __int__ is deprecated, and may be removed in a future version of Python.
  return getattr(node, kind + "_")(name, value)

tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
tests/test_modeling_t5.py::T5ModelTest::test_export_to_onnx
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:1638:
DeprecationWarning: an integer is required (got type float). Implicit conversion
to integers using __int__ is deprecated, and may be removed in a future version
of Python. value_t=torch.tensor([fill_value],
dtype=sym_help.scalar_type_to_pytorch_type[dtype]))

tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_identifier_from_model_type
tests/test_modeling_tf_auto.py::TFAutoModelTest::test_from_pretrained_identifier
src/transformers/modeling_tf_auto.py:697:
FutureWarning: The class `TFAutoModelWithLMHead` is deprecated and will be
removed in a future version. Please use `TFAutoModelForCausalLM` for causal
language models, `TFAutoModelForMaskedLM` for masked language models and
`TFAutoModelForSeq2SeqLM` for encoder-decoder models. warnings.warn(

tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_attentions
tests/test_modeling_squeezebert.py::SqueezeBertModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_squeezebert.py:78:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :seq_length]

tests/test_modeling_tf_flaubert.py: 9 warnings tests/test_modeling_tf_xlm.py: 9
warnings
src/transformers/modeling_tf_xlm.py:994:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. warnings.warn(

tests/test_modeling_tf_flaubert.py::TFFlaubertModelTest::test_graph_mode
tests/test_modeling_tf_xlm.py::TFXLMModelTest::test_graph_mode
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:493:
FutureWarning: The `lengths` parameter cannot be used with the XLM multiple
choice models. Please use the attention mask instead. return
py_builtins.overload_of(f)(*args)

tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_compile_tf_model
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_config
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_keras_save_load
tests/test_modeling_xlnet.py::XLNetModelTest::test_config
tests/test_modeling_xlnet.py::XLNetModelTest::test_correct_missing_keys
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_save_load
tests/test_modeling_tf_xlnet.py::TFXLNetModelTest::test_train_pipeline_custom_model
tests/test_modeling_xlnet.py::XLNetModelTest::test_save_load
src/transformers/configuration_xlnet.py:205:
FutureWarning: This config doesn't use attention memories, a core feature of
XLNet. Consider setting `mem_len` to a non-zero value, for example `xlnet =
XLNetLMHeadModel.from_pretrained('xlnet-base-cased'', mem_len=1024)`, for
accurate training performance as well as an order of magnitude faster inference.
Starting from version 3.5.0, the default parameter will be 1024, following the
implementation in https://arxiv.org/abs/1906.08237 warnings.warn(

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:531:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.size(0) == bs

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python number might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:532:
TracerWarning: Converting a tensor to a Python boolean might cause the trace to
be incorrect. We can't record the data flow of Python values, so this value will
be treated as a constant in the future. This means that the trace might not
generalize to other inputs! assert lengths.max().item() &lt;= slen

tests/test_modeling_xlm.py::XLMModelTest::test_torchscript
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_attentions
tests/test_modeling_xlm.py::XLMModelTest::test_torchscript_output_hidden_state
src/transformers/modeling_xlm.py:546:
TracerWarning: Converting a tensor to a Python index might cause the trace to be
incorrect. We can't record the data flow of Python values, so this value will be
treated as a constant in the future. This means that the trace might not
generalize to other inputs! position_ids = self.position_ids[:, :slen]

tests/test_optimization.py::OptimizationTest::test_adafactor
  src/transformers/optimization.py:512:
  UserWarning: This overload of add_ is deprecated: add_(Number alpha, Tensor
  other) Consider using one of the following signatures instead: add_(Tensor
  other, *, Number alpha) (Triggered internally at
  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)
  exp_avg_sq.mul_(beta2t).add_(1.0 - beta2t, update)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:247:
  UserWarning: To get the last learning rate computed by the scheduler, please
  use `get_last_lr()`. warnings.warn("To get the last learning rate computed by
  the scheduler, "

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131:
  UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.
  In PyTorch 1.1.0 and later, you should call them in the opposite order:
  `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will
  result in PyTorch skipping the first value of the learning rate schedule. See
  more details at
  https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before
  `optimizer.step()`. "

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:216:
  UserWarning: Please also save or load the state of the optimizer when saving
  or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_optimization.py::ScheduleInitTest::test_schedulers
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:234:
  UserWarning: Please also save or load the state of the optimizer when saving
  or loading the scheduler. warnings.warn(SAVE_STATE_WARNING, UserWarning)

tests/test_tokenization_auto.py::AutoTokenizerTest::test_tokenizer_identifier_with_correct_config
tests/test_tokenization_mbart.py::MBartEnroIntegrationTest::test_batch_fairseq_parity
tests/test_tokenization_t5.py::T5TokenizationTest::test_empty_target_text
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_max_target_length
tests/test_tokenization_t5.py::T5TokenizationTest::test_outputs_not_longer_than_maxlen
tests/test_tokenization_t5.py::T5TokenizationTest::test_prepare_seq2seq_batch
src/transformers/tokenization_utils_base.py:1421:
FutureWarning: The `max_len` attribute has been deprecated and will be removed
in a future version, use `model_max_length` instead. warnings.warn(

tests/test_tokenization_albert.py: 2 warnings tests/test_tokenization_bart.py: 2
warnings tests/test_tokenization_bert.py: 2 warnings
tests/test_tokenization_bert_generation.py: 1 warning
tests/test_tokenization_bertweet.py: 1 warning
tests/test_tokenization_blenderbot.py: 1 warning
tests/test_tokenization_ctrl.py: 1 warning tests/test_tokenization_camembert.py:
2 warnings tests/test_tokenization_distilbert.py: 4 warnings
tests/test_tokenization_dpr.py: 8 warnings tests/test_tokenization_fsmt.py: 1
warning tests/test_tokenization_funnel.py: 2 warnings
tests/test_tokenization_herbert.py: 2 warnings tests/test_tokenization_gpt2.py:
1 warning tests/test_tokenization_layoutlm.py: 2 warnings
tests/test_tokenization_marian.py: 1 warning tests/test_tokenization_lxmert.py:
2 warnings tests/test_tokenization_mbart.py: 2 warnings
tests/test_tokenization_pegasus.py: 2 warnings
tests/test_tokenization_openai.py: 1 warning tests/test_tokenization_phobert.py:
1 warning tests/test_tokenization_deberta.py: 1 warning
tests/test_tokenization_prophetnet.py: 1 warning
tests/test_tokenization_reformer.py: 1 warning
tests/test_tokenization_squeezebert.py: 4 warnings
tests/test_tokenization_t5.py: 2 warnings tests/test_tokenization_roberta.py: 2
warnings tests/test_tokenization_transfo_xl.py: 1 warning
tests/test_tokenization_xlm.py: 1 warning
tests/test_tokenization_xlm_prophetnet.py: 1 warning
tests/test_tokenization_xlnet.py: 2 warnings
tests/test_tokenization_xlm_roberta.py: 2 warnings
src/transformers/tokenization_utils_base.py:2025:
FutureWarning: The `pad_to_max_length` argument is deprecated and will be
removed in a future version, use `padding=True` or `padding='longest'` to pad to
the longest sequence in the batch, or use `padding='max_length'` to pad to a max
length. In this case, you can give a specific length with `max_length` (e.g.
`max_length=45`) or leave max_length to None to pad to the maximal input size of
the model (e.g. 512 for Bert). warnings.warn(

tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_in_input
tests/test_tokenization_t5.py::T5TokenizationTest::test_eos_treatment
src/transformers/tokenization_t5.py:183:
UserWarning: This sequence already has &lt;/s&gt;. In future versions this behavior
may lead to duplicated eos tokens being added. warnings.warn(

tests/test_trainer.py: 44 warnings
  /home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/nn/parallel/_functions.py:64:
  UserWarning: Was asked to gather along dimension 0, but all input tensors were
  scalars; will instead unsqueeze and return a vector. warnings.warn('Was asked
  to gather along dimension 0, but all '

tests/test_trainer.py::TrainerIntegrationTest::test_can_resume_training
tests/test_trainer_callback.py::TrainerCallbackTest::test_event_flow
/home/stas/anaconda3/envs/py38-pt17/lib/python3.8/site-packages/torch/cuda/nccl.py:48:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop
working if not isinstance(inputs, collections.Container) or isinstance(inputs,
torch.Tensor):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
</code></pre></div>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/LysandreJik/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/LysandreJik">@LysandreJik</a></p>
<h3 dir="auto">System Info</h3>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="- `transformers` version: 4.20.1
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.8.12
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no"><pre class="notranslate">- <span class="pl-s"><span class="pl-pds">`</span>transformers<span class="pl-pds">`</span></span> version: 4.20.1
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.8.12
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU<span class="pl-k">?</span>): 1.11.0 (True)
- Tensorflow version (GPU<span class="pl-k">?</span>): 2.8.0 (True)
- Flax version (CPU<span class="pl-k">?</span>/GPU<span class="pl-k">?</span>/TPU<span class="pl-k">?</span>): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU <span class="pl-k">in</span> script<span class="pl-k">?</span>: no
- Using distributed or parallel set-up <span class="pl-k">in</span> script<span class="pl-k">?</span>: no</pre></div>
<h3 dir="auto">Who can help?</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/patrickvonplaten/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/patrickvonplaten">@patrickvonplaten</a></p>
<h3 dir="auto">Information</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> The official example scripts</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> My own modified scripts</li>
</ul>
<h3 dir="auto">Tasks</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> An officially supported task in the <code class="notranslate">examples</code> folder (such as GLUE/SQuAD, ...)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> My own task or dataset (give details below)</li>
</ul>
<h3 dir="auto">Reproduction</h3>
<ol dir="auto">
<li>duplication case: the outputs should not contain the same sequence.</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; from transformers import GPT2LMHeadModel, GPT2Tokenizer

&gt;&gt;&gt; model = GPT2LMHeadModel.from_pretrained(&quot;gpt2&quot;)
&gt;&gt;&gt; tokenizer = GPT2Tokenizer.from_pretrained(&quot;gpt2&quot;)

&gt;&gt;&gt; force_word = &quot;are&quot;

&gt;&gt;&gt; force_words_ids = [
&gt;&gt;&gt;     tokenizer([force_word], add_prefix_space=True, add_special_tokens=False).input_ids,
&gt;&gt;&gt; ]

&gt;&gt;&gt; starting_text = [&quot;The soldiers&quot;]

&gt;&gt;&gt; input_ids = tokenizer(starting_text, return_tensors=&quot;pt&quot;).input_ids

&gt;&gt;&gt; outputs = model.generate(
&gt;&gt;&gt;     input_ids,
&gt;&gt;&gt;     max_new_tokens=3,
&gt;&gt;&gt;     force_words_ids=force_words_ids,
&gt;&gt;&gt;     num_beams=10,
&gt;&gt;&gt;     num_return_sequences=10,
&gt;&gt;&gt;     no_repeat_ngram_size=1,
&gt;&gt;&gt;     remove_invalid_values=True,
&gt;&gt;&gt; )

&gt;&gt;&gt; outputs = outputs[:, input_ids.shape[-1]:]
&gt;&gt;&gt; print(&quot;Output:\n&quot; + 100 * '-')
&gt;&gt;&gt; for s in tokenizer.batch_decode(outputs, skip_special_tokens=True):
&gt;&gt;&gt;   print(s)
&gt;&gt;&gt; import collections
&gt;&gt;&gt; print(collections.Counter(map(tuple, outputs.tolist())).most_common(1))
Output:
----------------------------------------------------------------------------------------------------
, who are
, who were
 who are in
, who had
, who are
 who are fighting
 who are still
 who are killed
 who are not
 who are on
[((11, 508, 389), 2)]"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">from</span> <span class="pl-s1">transformers</span> <span class="pl-k">import</span> <span class="pl-v">GPT2LMHeadModel</span>, <span class="pl-v">GPT2Tokenizer</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-v">GPT2LMHeadModel</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"gpt2"</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">tokenizer</span> <span class="pl-c1">=</span> <span class="pl-v">GPT2Tokenizer</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"gpt2"</span>)

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">force_word</span> <span class="pl-c1">=</span> <span class="pl-s">"are"</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">force_words_ids</span> <span class="pl-c1">=</span> [
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-en">tokenizer</span>([<span class="pl-s1">force_word</span>], <span class="pl-s1">add_prefix_space</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">add_special_tokens</span><span class="pl-c1">=</span><span class="pl-c1">False</span>).<span class="pl-s1">input_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> ]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">starting_text</span> <span class="pl-c1">=</span> [<span class="pl-s">"The soldiers"</span>]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">input_ids</span> <span class="pl-c1">=</span> <span class="pl-en">tokenizer</span>(<span class="pl-s1">starting_text</span>, <span class="pl-s1">return_tensors</span><span class="pl-c1">=</span><span class="pl-s">"pt"</span>).<span class="pl-s1">input_ids</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">outputs</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-s1">generate</span>(
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">input_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">max_new_tokens</span><span class="pl-c1">=</span><span class="pl-c1">3</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">force_words_ids</span><span class="pl-c1">=</span><span class="pl-s1">force_words_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">num_beams</span><span class="pl-c1">=</span><span class="pl-c1">10</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">num_return_sequences</span><span class="pl-c1">=</span><span class="pl-c1">10</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">no_repeat_ngram_size</span><span class="pl-c1">=</span><span class="pl-c1">1</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">remove_invalid_values</span><span class="pl-c1">=</span><span class="pl-c1">True</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> )

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">outputs</span> <span class="pl-c1">=</span> <span class="pl-s1">outputs</span>[:, <span class="pl-s1">input_ids</span>.<span class="pl-s1">shape</span>[<span class="pl-c1">-</span><span class="pl-c1">1</span>]:]
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-en">print</span>(<span class="pl-s">"Output:<span class="pl-cce">\n</span>"</span> <span class="pl-c1">+</span> <span class="pl-c1">100</span> <span class="pl-c1">*</span> <span class="pl-s">'-'</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">for</span> <span class="pl-s1">s</span> <span class="pl-c1">in</span> <span class="pl-s1">tokenizer</span>.<span class="pl-en">batch_decode</span>(<span class="pl-s1">outputs</span>, <span class="pl-s1">skip_special_tokens</span><span class="pl-c1">=</span><span class="pl-c1">True</span>):
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>   <span class="pl-en">print</span>(<span class="pl-s1">s</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">import</span> <span class="pl-s1">collections</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-en">print</span>(<span class="pl-s1">collections</span>.<span class="pl-v">Counter</span>(<span class="pl-en">map</span>(<span class="pl-s1">tuple</span>, <span class="pl-s1">outputs</span>.<span class="pl-en">tolist</span>())).<span class="pl-en">most_common</span>(<span class="pl-c1">1</span>))
<span class="pl-v">Output</span>:
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span>
, <span class="pl-s1">who</span> <span class="pl-s1">are</span>
, <span class="pl-s1">who</span> <span class="pl-s1">were</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-c1">in</span>
, <span class="pl-s1">who</span> <span class="pl-s1">had</span>
, <span class="pl-s1">who</span> <span class="pl-s1">are</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-s1">fighting</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-s1">still</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-s1">killed</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-c1">not</span>
 <span class="pl-s1">who</span> <span class="pl-s1">are</span> <span class="pl-s1">on</span>
[((<span class="pl-c1">11</span>, <span class="pl-c1">508</span>, <span class="pl-c1">389</span>), <span class="pl-c1">2</span>)]</pre></div>
<ol start="2" dir="auto">
<li>weird case: the output looks weird, repeat progressing constraints with unreasonable tokens</li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; from transformers import GPT2LMHeadModel, GPT2Tokenizer

&gt;&gt;&gt; model = GPT2LMHeadModel.from_pretrained(&quot;gpt2&quot;)
&gt;&gt;&gt; tokenizer = GPT2Tokenizer.from_pretrained(&quot;gpt2&quot;)

&gt;&gt;&gt; force_word = &quot;were not allowed&quot;

&gt;&gt;&gt; force_words_ids = [
&gt;&gt;&gt;     tokenizer([force_word], add_prefix_space=True, add_special_tokens=False).input_ids,
&gt;&gt;&gt; ]

&gt;&gt;&gt; starting_text = [&quot;The soldiers&quot;]

&gt;&gt;&gt; input_ids = tokenizer(starting_text, return_tensors=&quot;pt&quot;).input_ids

&gt;&gt;&gt; outputs = model.generate(
&gt;&gt;&gt;     input_ids,
&gt;&gt;&gt;     force_words_ids=force_words_ids,
&gt;&gt;&gt;     num_beams=10,
&gt;&gt;&gt;     num_return_sequences=10,
&gt;&gt;&gt;     no_repeat_ngram_size=1,
&gt;&gt;&gt;     remove_invalid_values=True,
&gt;&gt;&gt; )

&gt;&gt;&gt; print(&quot;Output:\n&quot; + 100 * '-')
&gt;&gt;&gt; for s in tokenizer.batch_decode(outputs, skip_special_tokens=True):
&gt;&gt;&gt;   print(s)
Output:
----------------------------------------------------------------------------------------------------
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander of
The soldiers, who were wearing were not in were not allowed to leave the barracks. The military said
The soldiers, who were wearing were not in were not allowed to enter the building. The military said
The soldiers, who were wearing were not in were not allowed to enter the building. The police said
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander said
The soldiers, who were wearing were not in were not allowed to enter the building. The police had
The soldiers, who were wearing were not in were not allowed to leave the barracks. The army said
The soldiers, who were wearing were not in were not allowed to enter the building. The police then
The soldiers, who were wearing were not in were not allowed to leave the barracks. The commander told
The soldiers, who were wearing were not in were not allowed to enter the building. The police and"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">from</span> <span class="pl-s1">transformers</span> <span class="pl-k">import</span> <span class="pl-v">GPT2LMHeadModel</span>, <span class="pl-v">GPT2Tokenizer</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-v">GPT2LMHeadModel</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"gpt2"</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">tokenizer</span> <span class="pl-c1">=</span> <span class="pl-v">GPT2Tokenizer</span>.<span class="pl-en">from_pretrained</span>(<span class="pl-s">"gpt2"</span>)

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">force_word</span> <span class="pl-c1">=</span> <span class="pl-s">"were not allowed"</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">force_words_ids</span> <span class="pl-c1">=</span> [
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-en">tokenizer</span>([<span class="pl-s1">force_word</span>], <span class="pl-s1">add_prefix_space</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">add_special_tokens</span><span class="pl-c1">=</span><span class="pl-c1">False</span>).<span class="pl-s1">input_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> ]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">starting_text</span> <span class="pl-c1">=</span> [<span class="pl-s">"The soldiers"</span>]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">input_ids</span> <span class="pl-c1">=</span> <span class="pl-en">tokenizer</span>(<span class="pl-s1">starting_text</span>, <span class="pl-s1">return_tensors</span><span class="pl-c1">=</span><span class="pl-s">"pt"</span>).<span class="pl-s1">input_ids</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">outputs</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-s1">generate</span>(
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">input_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">force_words_ids</span><span class="pl-c1">=</span><span class="pl-s1">force_words_ids</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">num_beams</span><span class="pl-c1">=</span><span class="pl-c1">10</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">num_return_sequences</span><span class="pl-c1">=</span><span class="pl-c1">10</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">no_repeat_ngram_size</span><span class="pl-c1">=</span><span class="pl-c1">1</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">remove_invalid_values</span><span class="pl-c1">=</span><span class="pl-c1">True</span>,
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> )

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-en">print</span>(<span class="pl-s">"Output:<span class="pl-cce">\n</span>"</span> <span class="pl-c1">+</span> <span class="pl-c1">100</span> <span class="pl-c1">*</span> <span class="pl-s">'-'</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">for</span> <span class="pl-s1">s</span> <span class="pl-c1">in</span> <span class="pl-s1">tokenizer</span>.<span class="pl-en">batch_decode</span>(<span class="pl-s1">outputs</span>, <span class="pl-s1">skip_special_tokens</span><span class="pl-c1">=</span><span class="pl-c1">True</span>):
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>   <span class="pl-en">print</span>(<span class="pl-s1">s</span>)
<span class="pl-v">Output</span>:
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">leave</span> <span class="pl-s1">the</span> <span class="pl-s1">barracks</span>. <span class="pl-v">The</span> <span class="pl-s1">commander</span> <span class="pl-s1">of</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">leave</span> <span class="pl-s1">the</span> <span class="pl-s1">barracks</span>. <span class="pl-v">The</span> <span class="pl-s1">military</span> <span class="pl-s1">said</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">enter</span> <span class="pl-s1">the</span> <span class="pl-s1">building</span>. <span class="pl-v">The</span> <span class="pl-s1">military</span> <span class="pl-s1">said</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">enter</span> <span class="pl-s1">the</span> <span class="pl-s1">building</span>. <span class="pl-v">The</span> <span class="pl-s1">police</span> <span class="pl-s1">said</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">leave</span> <span class="pl-s1">the</span> <span class="pl-s1">barracks</span>. <span class="pl-v">The</span> <span class="pl-s1">commander</span> <span class="pl-s1">said</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">enter</span> <span class="pl-s1">the</span> <span class="pl-s1">building</span>. <span class="pl-v">The</span> <span class="pl-s1">police</span> <span class="pl-s1">had</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">leave</span> <span class="pl-s1">the</span> <span class="pl-s1">barracks</span>. <span class="pl-v">The</span> <span class="pl-s1">army</span> <span class="pl-s1">said</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">enter</span> <span class="pl-s1">the</span> <span class="pl-s1">building</span>. <span class="pl-v">The</span> <span class="pl-s1">police</span> <span class="pl-s1">then</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">leave</span> <span class="pl-s1">the</span> <span class="pl-s1">barracks</span>. <span class="pl-v">The</span> <span class="pl-s1">commander</span> <span class="pl-s1">told</span>
<span class="pl-v">The</span> <span class="pl-s1">soldiers</span>, <span class="pl-s1">who</span> <span class="pl-s1">were</span> <span class="pl-s1">wearing</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-c1">in</span> <span class="pl-s1">were</span> <span class="pl-c1">not</span> <span class="pl-s1">allowed</span> <span class="pl-s1">to</span> <span class="pl-s1">enter</span> <span class="pl-s1">the</span> <span class="pl-s1">building</span>. <span class="pl-v">The</span> <span class="pl-s1">police</span> <span class="pl-c1">and</span></pre></div>
<h3 dir="auto">Expected behavior</h3>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="1. I believe the bug is due to the insufficient initialization of [`track_new[&quot;new_seqs&quot;]`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L658).
It appears if the top-k hypothesis also advances the constraints.
Then this hypothesis may appear multiple times in the beam as the example output.
The fix is one line in-place.
Updated to `track_new = {&quot;new_seqs&quot;: full_hypotheses.tolist(), &quot;new_states&quot;: [], &quot;new_indices&quot;: [], &quot;new_tokens&quot;: [], &quot;new_scores&quot;: []}`

2. I believe the bug is due to the incorrect value assignment of [`scores_for_all_vocab`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3223),
what stores the next **token scores**.
`scores_for_all_vocab` is first passed to [`constrained_beam_scorer.process()`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3262), and later passed to [`constrained_beam_scorer.step_sentence_constraint()`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L596) as `vocab_scores`.
Within its scope, `vocab_scores` is sliced to [`this_batch_token_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L654).
`this_batch_token_scores` is finally added to [`track_new[&quot;new_scores&quot;]`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L686).
However, the derived [`new_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L727) is concated with [`sent_beam_scores`](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L731),
reveal it should represent **beam scores**.
The **token scores** is larger than the expected **beam scores** as ignoring past token scores.
So the unfulfilled hypothesis will advance with an unexpected higher score, and dominate the beam as the example output.
The fix is also straightforward in-place.


# scores_for_all_vocab = next_token_scores_processed.clone()

next_token_scores = next_token_scores_processed + beam_scores[:, None].expand_as(next_token_scores)

scores_for_all_vocab = next_token_scores.clone()"><pre class="notranslate">1. I believe the bug is due to the insufficient initialization of [<span class="pl-s"><span class="pl-pds">`</span>track_new[<span class="pl-s"><span class="pl-pds">"</span>new_seqs<span class="pl-pds">"</span></span>]<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L658).
It appears <span class="pl-k">if</span> the top-k hypothesis also advances the constraints.
Then this hypothesis may appear multiple <span class="pl-c1">times</span> <span class="pl-k">in</span> the beam as the example output.
The fix is one line in-place.
Updated to <span class="pl-s"><span class="pl-pds">`</span>track_new = {<span class="pl-s"><span class="pl-pds">"</span>new_seqs<span class="pl-pds">"</span></span>: <span class="pl-en">full_hypotheses.tolist</span>(), <span class="pl-s"><span class="pl-pds">"</span>new_states<span class="pl-pds">"</span></span>: [], <span class="pl-s"><span class="pl-pds">"</span>new_indices<span class="pl-pds">"</span></span>: [], <span class="pl-s"><span class="pl-pds">"</span>new_tokens<span class="pl-pds">"</span></span>: [], <span class="pl-s"><span class="pl-pds">"</span>new_scores<span class="pl-pds">"</span></span>: []}<span class="pl-s"><span class="pl-pds">`</span></span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">2. I believe the bug is due to the incorrect value assignment of [<span class="pl-s"><span class="pl-pds">`</span>scores_for_all_vocab<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3223),</span></span>
<span class="pl-s"><span class="pl-s">what stores the next <span class="pl-k">**</span>token scores<span class="pl-k">**</span>.</span></span>
<span class="pl-s"><span class="pl-s"><span class="pl-pds">`</span></span>scores_for_all_vocab<span class="pl-s"><span class="pl-pds">`</span> is first passed to [<span class="pl-s"><span class="pl-pds">`</span>constrained_beam_scorer.process()<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_utils.py#L3262), and later passed to [<span class="pl-s"><span class="pl-pds">`</span>constrained_beam_scorer.step_sentence_constraint()<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L596) as <span class="pl-pds">`</span></span>vocab_scores<span class="pl-s"><span class="pl-pds">`</span>.</span></span>
<span class="pl-s"><span class="pl-s">Within its scope, <span class="pl-pds">`</span></span>vocab_scores<span class="pl-s"><span class="pl-pds">`</span> is sliced to [<span class="pl-s"><span class="pl-pds">`</span>this_batch_token_scores<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L654).</span></span>
<span class="pl-s"><span class="pl-s"><span class="pl-pds">`</span></span>this_batch_token_scores<span class="pl-s"><span class="pl-pds">`</span> is finally added to [<span class="pl-s"><span class="pl-pds">`</span>track_new[<span class="pl-s"><span class="pl-pds">"</span>new_scores<span class="pl-pds">"</span></span>]<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L686).</span></span>
<span class="pl-s"><span class="pl-s">However, the derived [<span class="pl-s"><span class="pl-pds">`</span>new_scores<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L727) is concated with [<span class="pl-s"><span class="pl-pds">`</span>sent_beam_scores<span class="pl-pds">`</span></span>](https://github.com/huggingface/transformers/blob/3b00b623b7cad9e1b7c71c97fff24a0286b37045/src/transformers/generation_beam_search.py#L731),</span></span>
<span class="pl-s"><span class="pl-s">reveal it should represent <span class="pl-k">**</span>beam scores<span class="pl-k">**</span>.</span></span>
<span class="pl-s"><span class="pl-s">The <span class="pl-k">**</span>token scores<span class="pl-k">**</span> is larger than the expected <span class="pl-k">**</span>beam scores<span class="pl-k">**</span> as ignoring past token scores.</span></span>
<span class="pl-s"><span class="pl-s">So the unfulfilled hypothesis will advance with an unexpected higher score, and dominate the beam as the example output.</span></span>
<span class="pl-s"><span class="pl-s">The fix is also straightforward in-place.</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s"><span class="pl-c"><span class="pl-c">#</span> scores_for_all_vocab = next_token_scores_processed.clone()</span></span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">next_token_scores = next_token_scores_processed + beam_scores[:, None].expand_as(next_token_scores)</span></span>
<span class="pl-s"><span class="pl-s"></span></span>
<span class="pl-s"><span class="pl-s">scores_for_all_vocab = <span class="pl-en">next_token_scores.clone</span>()</span></span></pre></div>