You said it is open source, however you are requiring a proprietary model from Meta, LLama.
If you say it is open source, then why are you requiring proprietary models? They are not open source. It is asking for tokenizer from a meta. That is proprietary model.
Can you maybe use some other model for tokenizer so that it is fully free software and not like this mixed with proprietary?
Edit: Apologies misunderstood!
Is there going to be any change on this subject? Can we get fully legal model?
The LLama 3.1 license isn't commercial restrictive, it simply has a scale clause that if you have more than 700 million active users utilizing the model or outputs of the model, you need to request a license. So unless folks here are running businesses with almost a billion people using them, you should not be concerned about the Llama 3.1 commercial restrictions...