--- license: apache-2.0 tags: - merge - mergekit - lazymergekit - Nitral-AI/Nyan-Stunna-7B - Nitral-AI/Kunocchini-7b-128k-test - gguf - Q2_K - Q3_K_L - Q3_K_M - Q3_K_S - Q4_0 - Q4_1 - Q4_K_S - Q4_k_m - Q5_0 - Q5_1 - Q6_K - Q5_K_S - Q5_k_m - Q8_0 - 128k language: - en - ru - th --- # NSK-7B-128k-slerp NSK-7B-128k-slerp is a merge of the following models using [mergekit](https://github.com/cg123/mergekit): * [Nitral-AI/Nyan-Stunna-7B](https://huggingface.co/Nitral-AI/Nyan-Stunna-7B) * [Nitral-AI/Kunocchini-7b-128k-test](https://huggingface.co/Nitral-AI/Kunocchini-7b-128k-test) ## 🧩 Configuration ```yaml slices: - sources: - model: Nitral-AI/Nyan-Stunna-7B layer_range: [0, 32] - model: Nitral-AI/Kunocchini-7b-128k-test layer_range: [0, 32] merge_method: slerp base_model: Nitral-AI/Kunocchini-7b-128k-test parameters: t: - filter: self_attn value: [0, 0.5, 0.3, 0.7, 1] - filter: mlp value: [1, 0.5, 0.7, 0.3, 0] - value: 0.5 dtype: bfloat16 ``` Eval embedding benchmark (with 70 specific quesions): ![inf.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/UbeMfW28pMHSRLsSbEsJB.jpeg) ![md28g.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/6UNV3CaKdofeAUr7C7x9k.jpeg) ![SK.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/uSnHhxDCqo9DP9oSb_l6j.jpeg) ![ks-inf.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/1ekTvK84ZlEsFFOYWOHE4.jpeg) ![command-r.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/5lVz28EK07RmrUe49y4jn.jpeg) ![NSK.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/aNdIdS5MnkwJ9YhprGznw.jpeg) ![NSMv2.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/vk2GpfnJnYS5u1_wA1Nhr.jpeg) ![aura.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/A3m0DC5E2x7V7UCbS1iCf.jpeg) ![ivanDrogo.jpg](https://cdn-uploads.huggingface.co/production/uploads/6404a7eaad54665351d89135/DaQIw6z8c-SupynTm9qos.jpeg)