new model: IDEFICS via HuggingFaceM4
**important: The following notes are for my team mates and they won't work for anybody else as the data isn't ready for the public yet. should be ready early in August**

status: the modeling code integration is ready - awaiting the final review 

Meanwhile to try it out:

```
$ git clone https://github.com/huggingface/transformers -b add-model-idefics
$ cd transformers

$ cat generate.py
import torch
from transformers import IdeficsForVisionText2Text, AutoProcessor

device = "cuda" if torch.cuda.is_available() else "cpu"

checkpoint = "HuggingFaceM4/idefics-9b"
#checkpoint = "HuggingFaceM4/tiny-random-idefics"

model = IdeficsForVisionText2Text.from_pretrained(checkpoint, torch_dtype=torch.bfloat16).to(device)
processor = AutoProcessor.from_pretrained(checkpoint)

prompts = [
    [
        "User:",
        "https://hips.hearstapps.com/hmg-prod/images/cute-photos-of-cats-in-grass-1593184777.jpg",
        "Describe this image."
        
        "Assistant: An image of two kittens in grass.",
        
        "User:",
        "https://hips.hearstapps.com/hmg-prod/images/dog-puns-1581708208.jpg",
        "Describe this image".
        
        "Assistant:",
    ],
    [
        "User:",
        "https://hips.hearstapps.com/hmg-prod/images/dog-puns-1581708208.jpg",
        "Describe this image."
        
        "Assistant: An image of a dog wearing funny glasses.",

        "User:",
        "https://hips.hearstapps.com/hmg-prod/images/cute-photos-of-cats-in-grass-1593184777.jpg",
        "Describe this image".

        "Assistant:",
    ],
]

# batched mode
inputs = processor(prompts, return_tensors="pt").to(device)
# single sample mode
#inputs = processor(prompts[0], return_tensors="pt").to(device)

generated_ids = model.generate(**inputs, max_length=100)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)
for i,t in enumerate(generated_text):
    print(f"{i}:\n{t}\n")
```

and then run:
```
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=src python generate.py
```

# Demos 

A PR with examples/demos, including finetuning, is here: https://github.com/huggingface/notebooks/pull/418

# TODOs before merging
- [ ] make the models public - which coincides with the announcement/release

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24796). All of your documentation changes will be reflected on that endpoint.<|||||>Is it possible to be a private repo ? ;-) The m4 repo from huggingface organisation does not exist<|||||>Thank you for your interest, @flozi00 - please give us some time. It says WIP because it's not ready for a public consumption. I edited the OP to clarify that.<|||||>I'm not able to rebase as this recently merged PR https://github.com/huggingface/transformers/pull/25174 breaks `tests/models/idefics/test_image_processing_idefics.py::IdeficsImageProcessingTest::test_torchvision_numpy_transforms_equivalency`

cc: @amyeroberts, if I need to adapt our image processing code please let me know - the function in question is called here:

https://github.com/huggingface/transformers/pull/24796/files#diff-e1b90eb52340b91c2471bac7c6fd34c67c7cd530050c607852fd426f397b3b3fR162<|||||>@sgugger, I addressed your feedback and this PR is ready for a detailed review. 

Thank you!<|||||>Thank you, @sgugger, @HugoLaurencon and @leot13 for your reviews - I have addressed everything you have raised.