unable to load model 'bert', tensor 'input_ids': the model expects 1 dimensions but the model configuration specified 2 dimensions
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->
[https://github.com/NVIDIA/triton-inference-server/issues/1338](url)
how to support batchsize for bert onnx in triton-inference-server?

    use bert from https://github.com/huggingface/transformers,

    fellow https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/bert#export-a-bert-model-from-pytorch.

    gotten "model.onnx" with batchsize as first dim (netron).

i set dynamic_batching { preferred_batch_size: [ 4, 8, 32 ] max_queue_delay_microseconds: 100 } in "config.pbtxt"

wrong
max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [-1, 128] } ]

wrong max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [1, 128] reshape: { shape: [-1,128 ] } } ]

ok max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [128] } ]

all latest two version.
## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 
I'm assuming you read https://blog.einstein.ai/benchmarking-tensorrt-inference-server/ ?<|||||>> 
> 
> I'm assuming you read https://blog.einstein.ai/benchmarking-tensorrt-inference-server/ ?
ok: torch.jit.trace(bert)
Failed :torch.jit.script
Failed:torch.jit.trace(bert+after)
Failed: to onnx: 
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py", line 686, in forward
    extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype)  # fp16 compatibility
UnboundLocalError: local variable 'extended_attention_mask' referenced before assignment   
（pip transformers==2.2.0， fixed latested source）

solved by remove all none default option argments. minize uncertainty

