run_mae.py can not be used directly on own dir
### System Info

ref: https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-pretraining

python run_mae.py \
    --model_type vit_mae \
    --dataset_name nateraw/image-folder \
    --train_dir <path-to-train-root> \
    --output_dir ./outputs/ \
    --remove_unused_columns False \
    --label_names pixel_values \
    --do_train \
    --do_eval


My params:

    --model_name_or_path /home/ana/data4/models/vit-mae-base
    --dataset_name nateraw/image-folder
    --train_dir /home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/
    --validation_dir /home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/
    --output_dir /home/ana/data4/output_models/rvl_mae_pretrain_demo_10k_100
    --remove_unused_columns False
    --label_names pixel_values
    --mask_ratio 0.75
    --norm_pix_loss
    --base_learning_rate 1.5e-4
    --lr_scheduler_type cosine
    --weight_decay 0.05
    --num_train_epochs 800
    --warmup_ratio 0.05
    --per_device_train_batch_size 8
    --per_device_eval_batch_size 8
    --logging_strategy steps
    --logging_steps 10
    --evaluation_strategy epoch
    --save_strategy epoch
    --load_best_model_at_end True
    --save_total_limit 5
    --seed 1337
    --do_train
    --do_eval

output:
Traceback (most recent call last):
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 397, in <module>
    main()
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 222, in main
    ds = load_dataset(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1773, in load_dataset
    builder_instance = load_dataset_builder(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1528, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/builder.py", line 329, in __init__
    data_files = DataFilesDict.from_local_or_remote(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 783, in from_local_or_remote
    DataFilesList.from_local_or_remote(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 751, in from_local_or_remote
    data_files = resolve_patterns_locally_or_by_urls(base_path, patterns, allowed_extensions)
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 349, in resolve_patterns_locally_or_by_urls
    for path in _resolve_single_pattern_locally(base_path, pattern, allowed_extensions):
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 293, in _resolve_single_pattern_locally
    raise FileNotFoundError(error_msg)
FileNotFoundError: Unable to find '/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/' at /

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

build a dir like:

dataset/
      train/
          1.jpg
          2.jpg
      eval/
          1.jpg
          2.jpg

run:
python run_mae.py \
    --model_name_or_path /home/ana/data4/models/vit-mae-base
    --dataset_name nateraw/image-folder
    --train_dir /home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/
    --validation_dir /home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/
    --output_dir /home/ana/data4/output_models/rvl_mae_pretrain_demo_10k_100
    --remove_unused_columns False
    --label_names pixel_values
    --mask_ratio 0.75
    --norm_pix_loss
    --base_learning_rate 1.5e-4
    --lr_scheduler_type cosine
    --weight_decay 0.05
    --num_train_epochs 800
    --warmup_ratio 0.05
    --per_device_train_batch_size 8
    --per_device_eval_batch_size 8
    --logging_strategy steps
    --logging_steps 10
    --evaluation_strategy epoch
    --save_strategy epoch
    --load_best_model_at_end True
    --save_total_limit 5
    --seed 1337
    --do_train
    --do_eval


### Expected behavior

output:
Traceback (most recent call last):
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 397, in <module>
    main()
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 222, in main
    ds = load_dataset(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1773, in load_dataset
    builder_instance = load_dataset_builder(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1528, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/builder.py", line 329, in __init__
    data_files = DataFilesDict.from_local_or_remote(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 783, in from_local_or_remote
    DataFilesList.from_local_or_remote(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 751, in from_local_or_remote
    data_files = resolve_patterns_locally_or_by_urls(base_path, patterns, allowed_extensions)
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 349, in resolve_patterns_locally_or_by_urls
    for path in _resolve_single_pattern_locally(base_path, pattern, allowed_extensions):
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/data_files.py", line 293, in _resolve_single_pattern_locally
    raise FileNotFoundError(error_msg)
FileNotFoundError: Unable to find '/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/' at /
The error

> FileNotFoundError: Unable to find '/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/' at /

shows you don't have local datasets (or there is some issue to locate it). Could you verify this on your own side? Thanks.<|||||>Hi @CheungZeeCn, thanks for raising this issue! 

So that we can best help you, could you:
* make sure code snippets and errors are properly formatted - placed between pairs of three backticks e.g. ` ``` code here ``` `. 
* Add information about the running environment: run `transformers-cli env` in the terminal and copy-paste the output

As @ydshieh mentions, it looks like the issue is coming from the paths being passed in for `train_dir` and `validation_dir`. They should be the names of folders containing the train and validation datasets relative to `dataset_name`. Based on the paths, the arguments should be:

```
--dataset_name /home/ana/data4/datasets/rvl_cdip/data/pretrain_images
--train_dir train
--validation_dir eval
```<|||||>@ydshieh  @amyeroberts  thank's for your replies, 

```
--dataset_name /home/ana/data4/datasets/rvl_cdip/data/pretrain_images
--train_dir train
--validation_dir eval
```
can not solve my problem.

That's how I fix it:

step1: download  dataset python file from: https://huggingface.co/datasets/nateraw/imagefolder/tree/main/ than put it in 
my local diretory: /home/ana/data4/datasets/rvl_cdip/data/pretrain_images 

step2: use the following params:
```
--dataset_name \
/home/ana/data4/datasets/rvl_cdip/data/pretrain_images \
--train_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/*" \
--validation_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/*" 
```
It's not the same as the doc.<|||||>Hi @CheungZeeCn 

Glad that you managed to make it work.

Just to make sure, what is works it with `--dataset_name nateraw/image-folder ` like the following

```bash
--dataset_name nateraw/image-folder 
--train_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/*" \
--validation_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/*" 
```

or the one with `/home/ana/data4/datasets/rvl_cdip/data/pretrain_images \
--train_dir \`?

Thanks in advance!<|||||>Hi, @ydshieh 

That's how my local dataset directory looks like:

```
(torch2) ana@pts-m1:~/data4/datasets/rvl_cdip/data/pretrain_images$ pwd
/home/ana/data4/datasets/rvl_cdip/data/pretrain_images
(torch2) ana@pts-m1:~/data4/datasets/rvl_cdip/data/pretrain_images$ ls
eval  imagefolder.py  train
(torch2) ana@pts-m1:~/data4/datasets/rvl_cdip/data/pretrain_images$ ls eval |head -10
0000298044.jpg
0000553824.jpg
0012197285.jpg
0060128913.jpg
```
and the imagefolder.py  is the same as this one  https://huggingface.co/datasets/nateraw/imagefolder/blob/main/imagefolder.py 

using the following is OK:
```
export WANDB_DISABLED=true
python run_mae.py \
--model_name_or_path \
/home/ana/data4/models/vit-mae-base \
--dataset_name \
/home/ana/data4/datasets/rvl_cdip/data/pretrain_images \
--train_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/*" \
--validation_dir \
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/*" \
--output_dir \
/home/ana/data4/output_models/rvl_mae_pretrain_demo_10k_100 \
--remove_unused_columns \
False \
--label_names \
pixel_values \
--mask_ratio \
0.5 \
--base_learning_rate \
1.5e-4 \
--lr_scheduler_type \
cosine \
--weight_decay \
0.05 \
--num_train_epochs \
800 \
--warmup_ratio \
0.05 \
--per_device_train_batch_size \
32 \
--gradient_accumulation_steps \
8 \
--per_device_eval_batch_size \
8 \
--logging_strategy \
steps \
--logging_steps \
10 \
--evaluation_strategy \
epoch \
--save_strategy \
epoch \
--load_best_model_at_end \
True \
--save_total_limit \
5 \
--seed \
1337 \
--do_train \
--do_eval \
--overwrite_output_dir
```

However, if I tried this:
```
python run_mae.py
--model_name_or_path
/home/ana/data4/models/vit-mae-base
--dataset_name nateraw/image-folder
--train_dir
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/train/*"
--validation_dir
"/home/ana/data4/datasets/rvl_cdip/data/pretrain_images/eval/*"
--output_dir
/home/ana/data4/output_models/rvl_mae_pretrain_demo_10k_100_tmp
--remove_unused_columns
False
--label_names
pixel_values
--mask_ratio
0.5
--base_learning_rate
1.5e-4
--lr_scheduler_type
cosine
--weight_decay
0.05
--num_train_epochs
800
--warmup_ratio
0.05
--per_device_train_batch_size
32
--gradient_accumulation_steps
8
--per_device_eval_batch_size
8
--logging_strategy
steps
--logging_steps
10
--evaluation_strategy
epoch
--save_strategy
epoch
--load_best_model_at_end
True
--save_total_limit
5
--seed
1337
--do_train
--do_eval
```
the output is:
```
Traceback (most recent call last):
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 397, in <module>
    main()
  File "/home/ana/data4/projects/hf_mae/run_mae.py", line 222, in main
    ds = load_dataset(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1773, in load_dataset
    builder_instance = load_dataset_builder(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/load.py", line 1528, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/site-packages/datasets/builder.py", line 350, in __init__
    info.update(self._info())
  File "/home/ana/.cache/huggingface/modules/datasets_modules/datasets/nateraw--image-folder/a2b5eb21064d8bd9b44c3b3fc91ae8205c3002a441852e1b02da78e8025c332e/image-folder.py", line 30, in _info
    classes = sorted([x.name.lower() for x in Path(folder).glob('*/**')])
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/pathlib.py", line 1041, in __new__
    self = cls._from_parts(args, init=False)
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/pathlib.py", line 682, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/home/ana/data1/anaconda3/envs/torch2/lib/python3.8/pathlib.py", line 666, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not DataFilesList

```
<|||||>Thanks a lot, we will take a look and update the doc if necessary!