[examples/research_projects/onnx/summarization] is outdated
### System Info

- `transformers` version: 4.25.0
- Platform: Linux-4.19.91-24.1.al7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.10
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed

### Who can help?

@fatcat-z 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

With the above env set, just clone the repo and run

```
python run_onnx_exporter.py --model_name_or_path facebook/bart-base
```

### Expected behavior

**Expected**: BART with BeamSearch exported with no error.

BTW, I notice that projects in `research_projects` are not maintained actively.  I want to export an `XXXModelForConditionalGeneration` with BeamSearch. And this demo is the best reference I have found so far. So I truly hope someone can fix it.

**Actual**:

```pytb
/home/root/.local/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py:232: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/home/root/.local/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py:239: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
/home/root/.local/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py:271: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
/home/root/.local/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py:915: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
/home/root/.local/lib/python3.8/site-packages/transformers/models/bart/modeling_bart.py:96: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  mask = torch.full((tgt_len, tgt_len), torch.tensor(torch.finfo(dtype).min))
/home/root/.local/lib/python3.8/site-packages/torch/jit/_trace.py:976: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module's inputs. Consider using a constant container instead (e.g. for `list`, use a `tuple` instead. for `dict`, use a `NamedTuple` instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.
  module._c._create_method_from_trace(
/home/root/.local/lib/python3.8/site-packages/torch/jit/_trace.py:154: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:480.)
  if a.grad is not None:
/home/root/.local/lib/python3.8/site-packages/torch/jit/annotations.py:309: UserWarning: TorchScript will treat type annotations of Tensor dtype-specific subtypes as if they are normal Tensors. dtype constraints are not enforced in compilation either.
  warnings.warn("TorchScript will treat type annotations of Tensor "
Traceback (most recent call last):
  File "run_onnx_exporter.py", line 207, in <module>
    main()
  File "run_onnx_exporter.py", line 203, in main
    export_and_validate_model(model, tokenizer, output_name, num_beams, max_length)
  File "run_onnx_exporter.py", line 123, in export_and_validate_model
    torch.onnx.export(
  File "/home/root/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 504, in export
    _export(
  File "/home/root/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 1529, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/root/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 1131, in _model_to_graph
    example_outputs = _get_example_outputs(model, args)
  File "/home/root/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 1017, in _get_example_outputs
    example_outputs = model(*input_args, **input_kwargs)
  File "/home/root/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
RuntimeError: forward() Expected a value of type 'Tensor (inferred)' for argument 'num_beams' but instead found type 'int'.
Inferred 'num_beams' to be of type 'Tensor' because it was not annotated with an explicit type.
Position: 3
Value: 4
Declaration: forward(__torch__.bart_onnx.generation_onnx.BARTBeamSearchGenerator self, Tensor input_ids, Tensor attention_mask, Tensor num_beams, Tensor max_length, Tensor decoder_start_token_id) -> Tensor
Cast error details: Unable to cast 4 to Tensor
```
This is an unmaintained example, it won't work without using the transformers version corresponding to the time it was written.<|||||>@sgugger Thanks for your quick reply.

For some reason, I have to use transformer==4.25.0. I wonder if @fatcat-z has any suggestion on how to adapt the code?<|||||>Hi,

For converting summarization models to ONNX, we now have a lot of classes implemented in [HuggingFace Optimum](https://huggingface.co/docs/optimum/index).

Generation with ONNX models is also implemented there (greedy, beam search, etc.). Check the [guides](https://huggingface.co/docs/optimum/onnxruntime/overview) for more info.<|||||>Hi @NielsRogge 

Optimum does look promising. But my model is a GPT-like decoder, with only a `*ModelForConditionalGeneration` interface. You can find details [here](https://huggingface.co/BAAI/glm-large/blob/main/modeling_glm.py). The signature of the generation interface differs a bit from what Optimum has officially supported. As far as I can tell, even if I managed to convert the model into ONNX following this [guide](https://huggingface.co/docs/transformers/serialization), using Optimum will run into another issue.<|||||>Hi @un-certainty, not sure if I understood your issue well. Would the [`ORTModelForCustomTasks`](https://github.com/huggingface/optimum/blob/e8f5a955bc40eea8c1382ab29be8f8ac99601817/optimum/onnxruntime/modeling_ort.py#L1585) help you to achieve this?
Otherwise, don't hesitate to open an issue in [Optimum](https://github.com/huggingface/optimum/issues) to see how we can improve the current implementation.<|||||>Hi @un-certainty ,

Could you please try to update the code [here](https://github.com/huggingface/transformers/blob/197e7ce911d91d85eb2f91858720957c2d979cd2/examples/research_projects/onnx/summarization/bart_onnx/generation_onnx.py#L709)?

Please set the type explicitly, like "num_beams: int".<|||||>Hi @fatcat-z 

Thanks for your suggestion. 

I found that you created two traced decoders [here](https://github.com/huggingface/transformers/blob/197e7ce911d91d85eb2f91858720957c2d979cd2/examples/research_projects/onnx/summarization/bart_onnx/generation_onnx.py#L222). So will there be two decoders in the converted ONNX graph?<|||||>> Hi @fatcat-z
> 
> Thanks for your suggestion.
> 
> I found that you created two traced decoders [here](https://github.com/huggingface/transformers/blob/197e7ce911d91d85eb2f91858720957c2d979cd2/examples/research_projects/onnx/summarization/bart_onnx/generation_onnx.py#L222). So will there be two decoders in the converted ONNX graph?

In the final ONNX graph, there will be no decoder ONNX op so there won't be 2 decoders there.

The point is: each traced decoder in the model will be converted to a set of ONNX ops in the final ONNX graph.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.