Error using inputs_embeds argument in TFXLNetModel
While using the TFXLNetModel:
`xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased')`
according to the docs, `input_ids` and `inputs_embeds` can be optionally used. However, when I tried using:

`xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0]`
it throws: `ValueError: The first argument to Layer.call must always be passed.`

which I thought is an issue with the `inputs` argument which must be a positional one:
`xlnet(inputs=None, inputs_embeds=embeddings, attention_mask=attn_masks)[0]` using this gave me:
`RuntimeError: Attempting to capture an EagerTensor without building a function.`

And finally passing both `inputs` and `inputs_embeds` gave: `ValueError: You cannot specify both input_ids and inputs_embeds at the same time`

Can someone suggest a workaround on this?
P.S. the `embeddings` variable is the `last_hidden_state` from another bert which matches the config for the `inputs_embeds` shape.
Note that `input_ids` parameter won't count as it gave the same error if I didn't use the `inputs` argument.
Hey, @patrickvonplaten  I observed the same with TFBertModel. So pretty much evident that the issue must be with the parent (if at all that helps)<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> 
> 
> While using the TFXLNetModel:
> `xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased')`
> according to the docs, `input_ids` and `inputs_embeds` can be optionally used. However, when I tried using:
> 
> `xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0]`
> it throws: `ValueError: The first argument to Layer.call must always be passed.`

try something like this:
xlnet({'attention_mask':attention_mask, 'token_type_ids':token_type_ids},inputs_embeds=embeddings, training=training)

worked for me when getting the same error for a bert model
<|||||>```python
model_outputs = self.transformer(
    input_ids=None,  # add this line
    inputs_embeds=dense_feature,
    attention_mask=attention_mask
)
```<|||||>Is this still relevant? Gently pinging @gante @Rocketknight1 here <|||||>I got around the issue by changing the code from this
```
config = MobileBertConfig()
mbert = TFMobileBertModel(config)
mbert(inputs={"input_ids":input_ids, "attention_mask":attention_mask})
```
to 
```
config = MobileBertConfig()
mbert = TFMobileBertModel(config)
mbert(input_ids=input_ids, attention_mask=attention_mask)
```

transformers version : 4.22.1