run_speech_recognition_ctc.py throwing error when using own dataset
## Environment info
- `transformers` version: 4.12.0.dev0
- Platform: Linux-5.4.0-1055-aws-x86_64-with-glibc2.10
- Python version: 3.8.12
- PyTorch version (GPU?): 1.10.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@patrickvonplaten 

Models:
- Wav2Vec2, HuBERT, SpeechEncoderDecoder: @patrickvonplaten, @anton-l

## Information

I am trying to use the recently added script run_speech_recognition_ctc.py to finetune facebook/wav2vec2-large-xlsr-53 as per the documentation.

The problem arises when using:
If I use the official example script the training works as expected. However if I change the dataset_name argument so that it uses my own dataset I get a pyarrow.lib.ArrowNotImplementedError error.


The tasks I am working on is:
* ASR using my own dataset: (give details below)
I am trying to fine tune using my own dataset "pete/autonlp-data-tas_pa_model" which is on the model hub. This dataset has been used by AutoNLP in the past successfully.

## To reproduce

Steps to reproduce the behavior:

Run run_speech_recognition_ctc.py as shown below. The only parameter that has been changed from the demo (https://github.com/huggingface/transformers/tree/master/examples/pytorch/speech-recognition) is the dataset name. If I change the dataset_name to --dataset_name="common_voice" the training works.

```python
python run_speech_recognition_ctc.py \
        --dataset_name="pete/autonlp-data-tas_pa_model" \
        --model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
        --dataset_config_name="processed" \
        --output_dir="./models" \
        --overwrite_output_dir \
        --num_train_epochs="15" \
        --per_device_train_batch_size="16" \
        --gradient_accumulation_steps="2" \
        --learning_rate="3e-4" \
        --warmup_steps="500" \
        --evaluation_strategy="steps" \
        --text_column_name="text" \
        --save_steps="400" \
        --eval_steps="100" \
        --layerdrop="0.0" \
        --save_total_limit="3" \
        --freeze_feature_extractor \
        --gradient_checkpointing \
        --chars_to_ignore , ? . ! - \; \: \" “ % ‘ ” � \
        --fp16 \
        --group_by_length \
        --push_to_hub \
        --do_train --do_eval
```

I get the following error

```python
Traceback (most recent call last):
  File "run_speech_recognition_ctc.py", line 615, in <module>
    main()
  File "run_speech_recognition_ctc.py", line 336, in main
    raw_datasets["train"] = load_dataset(
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/datasets/load.py", line 1627, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/datasets/builder.py", line 607, in download_and_prepare
    self._download_and_prepare(
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/datasets/builder.py", line 697, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/datasets/builder.py", line 1159, in _prepare_split
    writer.write_table(table)
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/datasets/arrow_writer.py", line 428, in write_table
    pa_table = pa.Table.from_arrays([pa_table[name] for name in self._schema.names], schema=self._schema)
  File "pyarrow/table.pxi", line 1596, in pyarrow.lib.Table.from_arrays
  File "pyarrow/table.pxi", line 592, in pyarrow.lib._sanitize_arrays
  File "pyarrow/array.pxi", line 329, in pyarrow.lib.asarray
  File "pyarrow/table.pxi", line 277, in pyarrow.lib.ChunkedArray.cast
  File "/home/ubuntu/anaconda3/envs/transformers/lib/python3.8/site-packages/pyarrow/compute.py", line 297, in cast
    return call_function("cast", [arr], options)
  File "pyarrow/_compute.pyx", line 527, in pyarrow._compute.call_function
  File "pyarrow/_compute.pyx", line 337, in pyarrow._compute.Function.call
  File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 120, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from struct<train: struct<name: string, num_bytes: int64, num_examples: int64, dataset_name: null>> to list using function cast_list
```

## Expected behavior

The model should be fine tuned using the test and training data in the pete/autonlp-data-tas_pa_model dataset. Note the dataset is private, I had to download the dataset in order for the script to run.

Hey @peterhanlon,

Thanks for the issue. In your case it seems like you have the dataset already more or less processed, so that the script won't work out of the box. 

You will have to adapt the beginning of the script so that it removes all data preprocessing code and simply loads your processed data. 

Could you try taking a look at this script: https://huggingface.co/ami-wav2vec2/wav2vec2-base-ami_multi-nithin3/blob/main/run_speech_recognition_ctc.py which does more or less the same thing as you intend to do (I believe) and see whether this would work for you? :-)<|||||>Thanks so much for the response @patrickvonplaten, I will give that a go :)
<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.