---
license: apache-2.0
language:
- en
---

# Model Card for SuperSparse-Mixtral

The SuperSparse-Mixtral Large Language Model (LLM) is an sparsified version of the Mixtral.

## Inference

Our code for accelerate SuperSparse-Mixtral is under refining, stay tuned. 

## Allow Finetuning

As we merged the predictors for FFN neurons in models, you can finetune SuperSparse-Mixtral with any framework and algorithm.

## License

The model is licensed under Apache-2.0, while model weights are fully open for academic research and also allow **free** commercial usage.