Edit model card

This model is based on the fusion strategy offered by Fanqi Wan(https://github.com/fanqiwan/FuseLLM).

Three models are fused together. 10epochs

Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Blending model 1: HanNayeoniee/LHK_DPO_v1

Blending model 2: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B

This model will be optimized by Laser and DPO later.

This project is to make the on-device sLM. We are doing experiments on the models.

Downloads last month: 74

Safetensors

Model size

1.1B params

Tensor type

BF16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.