YOYO-AI/Qwen2.5-14B-it-restore · Please add a documentation on how you made this "restore" model

marcuscedricridia

Mar 21

•

edited Mar 21

I'm really interested in making "restore" models. Seems like your technique does have benefits and I want to learn.

YOYO-AI

Owner Mar 21

Details of the Merge

models:
  - model: Qwen/Qwen2.5-14B-instruct
    parameters:
      density: 1 
      weight: 1
      lambda: 0.9
merge_method: della
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  lambda: 0.9
  normalize: true
  int8_mask: true
dtype: bfloat16
name: Qwen2.5-14B-della

models:
  - model: Qwen/Qwen2.5-14B-instruct
    parameters:
      density: 1 
      weight: 1
merge_method: ties
base_model: Qwen/Qwen2.5-14B
parameters:
  density: 1
  weight: 1
  normalize: true
  int8_mask: true
dtype: bfloat16
name: Qwen2.5-14B-ties

merge_method: model_stock
base_model: Qwen/Qwen2.5-14B-instruct
models:
  - model: Qwen2.5-14B-della
  - model: Qwen2.5-14B-ties
dtype: bfloat16
tokenizer_source: base
int8_mask: true
normalize: true
name: Qwen2.5-14B-it-restore

marcuscedricridia

Mar 21

Thank you so much!