Is there any plan to add kosmos-2 to the transformers. 
### Model description

Kosmos-2 is a grounded multimodal large language model, which integrates grounding and referring capabilities compared with Kosmos-1. The model can accept image regions selected by the user using bounding boxes as input, provide visual answers (i.e., bounding boxes), and ground the text output to the visual world.

**Is there any plan to add this model to the transformers.**

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Code: https://github.com/microsoft/unilm/tree/master/kosmos-2
Paper: https://arxiv.org/abs/2306.14824
Weight: the checkpoint can be downloaded from [here](https://conversationhub.blob.core.windows.net/beit-share-public/kosmos-2/kosmos-2.pt?sv=2021-10-04&st=2023-06-08T11%3A16%3A02Z&se=2033-06-09T11%3A16%3A00Z&sr=c&sp=r&sig=N4pfCVmSeq4L4tS8QbrFVsX6f6q844eft8xSuXdxU48%3D)  
VQA demo: [here](https://github.com/BIGBALLON/kosmos-2-gd)
Thank you for mentioning this :-). There is some early discussion within the team. I will come back to you once we have some decision.<|||||>This is tracked in PR #24709. (so far empty, but I will try to 🚀 )<|||||>@ydshieh  I'm very excited to hear this news. I sincerely appreciate your efforts.<|||||>any updates?<|||||>Still on it (slowly) 🤗 <|||||>Sure. Thank you. Appreciate those efforts.
<|||||>I just want to say a big thank you for your effort @ydshieh! Looking forward to it.<|||||>@Rajmehta123 @yolandalalala @vanpelt 

This [project](https://github.com/BIGBALLON/kosmos-2-gd) can be provided for everyone to try, I hope it can help everyone<|||||>Very nice! @BIGBALLON Thanks a lot!<|||||>@ydshieh Thank you again for your great contribution!<|||||>Amazing! @BIGBALLON Thanks a lot!<|||||>Just want to give a update: I am almost done the coding - just need to put everything together to finalize.

(The model might ends up as a custom code on the Hub instead of directly available in `transformers` - I am not sure)