save_pretrained: mkdir(exist_ok=True)
Old Logic:
if you pass something that is not a pre-existing directory -> Error

New Logic:
if you pass a file that exists -> Error.
if you pass a path that doesn't exist -> we call `mkdir path`, no error.
if you pass an existing directory -> no error.

This is not a breaking change, since no calls that previously succeeded produce different results.


Costs:
- you might occasionally make a directory called `pytorch_model.bin` for a confused user.
- a little bit of error checking code

Benefits:
- fewer late failures during training because you forgot to mkdir. This happens to me a lot.
- It feels like the spirit of the lib is to make NLP easier for the user, and I think this change is a small step in that direction.

Feedback much appreciated, I want to see if people like this before I add tests.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=h1) Report
> Merging [#5258](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/efae6645e223f29cf05eeafe95105a9f869b66dd&el=desc) will **decrease** coverage by `0.51%`.
> The diff coverage is `50.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5258/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5258      +/-   ##
==========================================
- Coverage   77.69%   77.17%   -0.52%     
==========================================
  Files         138      138              
  Lines       24291    24300       +9     
==========================================
- Hits        18872    18754     -118     
- Misses       5419     5546     +127     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `75.13% <0.00%> (-0.32%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.39% <50.00%> (-0.10%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.93% <50.00%> (-0.21%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `92.39% <66.66%> (-1.32%)` | :arrow_down: |
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `95.68% <100.00%> (+0.03%)` | :arrow_up: |
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `16.26% <0.00%> (-66.67%)` | :arrow_down: |
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `33.56% <0.00%> (-8.93%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `79.26% <0.00%> (-0.34%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/huggingface/transformers/pull/5258/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=footer). Last update [1af58c0...ac579a6](https://codecov.io/gh/huggingface/transformers/pull/5258?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>I'm favorable to this (I think I wanted to do it some time ago)

If we end up merging this, we should also clean up a lot of `os.makedir` calls "upstream" to this (example scripts, etc.)<|||||>Awesome, I'll grep for makedirs and mkdir and see what I can delete tomorrow.<|||||>Great change! <|||||>@thomwolf  why did you guys put `logger.error` instead of the raising normal exceptions in the python tokenizer file?  Should I raise a `NotADirectoryException` if a save path is mis-specified as a file or keep the `logger.error`, `return None` logic?

This will be for calls like `tokenizer.save_pretrained("tokenizer.json")`<|||||>You should also clean up some calls to `os.makedir` in the Trainer, I think?