content.raw: I use mergekit regularly, and often enough get acceptable results without performing fine-tuning afterward. My current thinking is that DARE-TIES should be avoided when merging dense models, as the process of thinning inherently punches holes in models.
content.type: text
content.value: I use mergekit regularly, and often enough get acceptable results without performing fine-tuning afterward. My current thinking is that DARE-TIES should be avoided when merging dense models, as the process of thinning inherently punches holes in models.
content.type: new_line
content.type: new_line
content.raw: I've had success using SLERP merges to graft Mistral v0.1 models with Mistral v0.2 models to obtain the context length benefits of the latter, and am looking forward to experimenting with Mistral v0.3, which recently dropped.
content.type: text
content.value: I've had success using SLERP merges to graft Mistral v0.1 models with Mistral v0.2 models to obtain the context length benefits of the latter, and am looking forward to experimenting with Mistral v0.3, which recently dropped.
rawContent: I use mergekit regularly, and often enough get acceptable results without performing fine-tuning afterward. My current thinking is that DARE-TIES should be avoided when merging dense models, as the process of thinning inherently punches holes in models.
  
  I've had success using SLERP merges to graft Mistral v0.1 models with Mistral v0.2 models to obtain the context length benefits of the latter, and am looking forward to experimenting with Mistral v0.3, which recently dropped.
reactions.reaction: 🔥
reactions.users: Mars2050 Meggido osanseviero Lewdiculous TravelingMan
reactions.reaction: 🚀
reactions.users: maldv Lewdiculous
reactions.reaction: 👍
reactions.users: SerialKicked Lewdiculous
reactions.reaction: ❤️
reactions.users: Lewdiculous Ardvark123
commentators._id: 65d4cf2693a0a3744a27536c
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/HOCJv7WE2C4eqqqppDFpA.png
commentators.fullname: Lewdiculous (AetherArchitectural)
commentators.name: Lewdiculous
commentators.type: user