getting AssertionError when using Trainer with `fsdp` and `torch_compile`
### System Info

While trying to train a `GPT2` model using the `Trainer` with `torch_compile` and `fsdp` flags I get the following error:

```bash
AssertionError: Dynamo only supports FSDP with use_orig_params=True
```

I'm using `python==3.10.9`, and initially used `transformers==4.27.X` but after stumbling upon #22279 I updated to `transformers==4.28.1` but the problem persisted.

### Who can help?

I'm guessing this is @sgugger territory and maybe @ani300 could help too. 😊 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

My training file looks like:

```python
def train(cfg: DictConfig):

    # Setup data and dataloader
    data_module = prepare_data_module(**cfg.data)

    train_dataloader = data_module.train_dataloader()
    val_dataloader = data_module.val_dataloader()

    # Extract tokenizer from datamodule
    tokenizer = data_module.tokenizer

    # Setup model and optimizer
    model = GPT(**cfg.model)

    # Setup data collator
    data_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)

    # Setup and running training
    train_args = TrainingArguments(**cfg.train_args)

    trainer = Trainer(
        model=model,
        tokenizer=tokenizer,
        train_args=args,
        data_collator=data_collator,
        train_dataset=train_dataloader,
        eval_dataset=val_dataloader,
    )

    trainer.train()
```

Here's my full `TrainingArguments` configuration in `.yaml` format:

```yaml
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
evaluation_strategy: "steps"
eval_steps: 2000
logging_steps: 5000
gradient_accumulation_steps: 8
num_train_epochs: 300
weight_decay: 0.1
warmup_steps: 1_000
lr_scheduler_type: "cosine"
learning_rate: 5e-4
save_steps: 25000
bf16: True
torch_compile: True
tf32: True
fsdp: "full_shard auto_wrap"
fsdp_transformer_layer_cls_to_wrap: 'GPT2Block'
```

And I'm running the training using `torchrun --nproc_per_node=8 train.py` with 8 NVIDIA A40.

### Expected behavior

It should run the training process without problems.
meet same issue....<|||||>cc @pacman100 <|||||>The FSDP wrapper inside `trainer.py` needs to be initialized with `use_orig_params=True` for FSDP + compile to work well together. As of now, that is not the case and there is no flag in the Trainer to make it do so. I can probably find some time later in the week to make a PR, but in any case, that's the issue.<|||||>Thanks @ani300. I will attempt doing a PR. :blush: <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.