anton-l

/

gpt-j-tiny-random

like 1

Beside, is this model licensed under the Apache 2.0 like gpt-j-6b from EleutherAI ?

@anton-l Just pinging you as I am not sure whether you get notifications when a PR is opened (If you do, sorry for the spam !)

@lerouxrgd maybe it makes sense to just set up another repo for the rust weights? Because the model here is literally just a dummy one, with a minimally working config and randomly initialized weights 😅 It was intended for unit-testing the transformers implementation

Yes I am well aware that the weights are literally random :D But this is fine, as it would serve the same purpose that is unit testing but for the Rust implementation. It's actually quite useful to have small weights for such tests, and I think that having the Python/Rust weights together makes it clear that the unit tests are correct (I intend to compare the final logits in Rust with the final logits in Python, when using the same model weights).

For some other models the Rust weights are being stored in the same repo:
https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased/discussions/1
I intend to add the Rust weights to EleutherAI's gpt-j-6b repo too once the unit tests are complete.