Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Paper
•
2311.03099
•
Published
•
30
A reasonably logical model with a few datasets thrown in to increase RP abilities. This is a good candidate for a balanced 7B model that can provide assistant functionality alongside roleplaying or romantic endeavors.
This is a merge of pre-trained language models created using mergekit.
This model was merged using the DARE TIES merge method using jeiku/SpaghettiOs_7B as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
merge_method: dare_ties
base_model: jeiku/SpaghettiOs_7B
parameters:
normalize: true
models:
- model: jeiku/SpaghettiOs_7B
parameters:
weight: 1
- model: jeiku/Rainbow_69_7B
parameters:
weight: 1
dtype: float16