Cassiopeia-70B

Cassiopeia-70B is the result of an experimental multi-step SLERP merge of Llama-3.3-70B-Instruct, Shimamura-70B, and Anubis-70B-v1.1. It is a coherent, unaligned model intended to be used for creative tasks such as storywriting, brainstorming, interactive roleplay, etc.

Merge composition

Intermediate model

Stay close to Anubis v1.1, but step 20% closer to stock L3.3 70B.

models:
  - model: /opt/workspace/hf/Anubis-70B-v1.1
  - model: /opt/workspace/hf/Llama-3.3-70B-Instruct
merge_method: slerp
base_model: /opt/workspace/hf/Anubis-70B-v1.1
parameters:
  t: 0.2
dtype: bfloat16

Final model

Pull the middle of the intermediate model towards Shimamura, but leave the ends as they are.

models:
  - model: /opt/workspace/hf/Anubis-70B-v1.1-0.8x
  - model: /opt/workspace/hf/Shimamura-70B
merge_method: slerp
base_model: /opt/workspace/hf/Anubis-70B-v1.1-0.8x
parameters:
  t: [0.0, 0.5, 1.0, 0.5, 0.0]
dtype: bfloat16

Feedback

If you like this model, please support one of the original model creators:

• Delta-Vector on ko-fi
• TheDrummer on Patreon

Feedback on this merge is very welcome, good or bad! Please leave a comment in this discussion with your thoughts: Cassiopeia-70B/discussions/1