metadata
license: apache-2.0
language:
- en
Model Card for SuperSparse-Mixtral
The SuperSparse-Mixtral Large Language Model (LLM) is an sparsified version of the Mixtral.
Inference
Our code for accelerate SuperSparse-Mixtral is under refining, stay tuned.
Allow Finetuning
As we merged the predictors for FFN neurons in models, you can finetune SuperSparse-Mixtral with any framework and algorithm.
License
The model is licensed under Apache-2.0, while model weights are fully open for academic research and also allow free commercial usage.