RWKV-WORLD-4
### Model description

BlinkDL/rwkv-4-world is a repo present on Huggingface i want the model's tokenizer and the model to be added to the Transformers Lib. 

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_
Hi @CosmoLM, thanks for opening this model request! 

The RWKV-4 model already exists in transformers -- [PR](https://github.com/huggingface/transformers/pull/22797), [docs](https://huggingface.co/docs/transformers/v4.31.0/en/model_doc/rwkv#rwkv-attention-and-the-recurrent-formulas). To enable loading the model through `Rwkv.from_pretrained`, the checkpoints would need to be converted and model configs push to the hub using [the conversion script.](https://github.com/huggingface/transformers/blob/8021c684ec3023295513be36bdc30e27e6f28cfc/src/transformers/models/rwkv/convert_rwkv_checkpoint_to_hf.py#L4)  

I'd suggest opening a discussion on the hub to see if the repo owners would be interested in doing this. 
 <|||||>The RWKV-pile models are available but not the RWKV-world models because
its tokenizer is not in the json format it is in txt format.

On Wed, 2 Aug, 2023, 4:24 pm amyeroberts, ***@***.***> wrote:

> Hi @CosmoLM <https://github.com/CosmoLM>, thanks for opening this model
> request!
>
> The RWKV-4 model already exists in transformers -- PR
> <https://github.com/huggingface/transformers/pull/22797>, docs
> <https://huggingface.co/docs/transformers/v4.31.0/en/model_doc/rwkv#rwkv-attention-and-the-recurrent-formulas>.
> To enable loading the model through Rwkv.from_pretrained, the checkpoints
> would need to be converted and model configs push to the hub using the
> conversion script.
> <https://github.com/huggingface/transformers/blob/8021c684ec3023295513be36bdc30e27e6f28cfc/src/transformers/models/rwkv/convert_rwkv_checkpoint_to_hf.py#L4>
>
> I'd suggest opening a discussion on the hub to see if the repo owners
> would be interested in doing this.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/25253#issuecomment-1661993346>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/BA7FALGYW7ERQ3LODEA6NADXTIWVPANCNFSM6AAAAAA3A3B6CY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
