about

Original name : Llama_3.x_70b_Athoblumicreacalmtess_v1.0

A creation oriented 3.1 merge based on a 3.0->3.3 "port".

The second L3.1 brick of my future merges after the first brick, led by a Dolphin 2.9.1 L3.0-L3.3 "port".

This time, an Athene port leads this merge (I like the original prose of this model), Nemotron is quite present in the other models used which helps "structure and smarts", Tess is used again as a PPL dropper, and the other components are creation oriented (Calme is quite good for the descriptions, and I used this model as a main for a long time, so).

It is also vastly uncensored, and can be used as a standalone as well.

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using Nexesenex/Llama_3.x_70b_L3.3_Athene_128K_v1.02 as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

merge_method: model_stock
models:
  - model: Nexesenex/Llama_3.x_70b_L3.3_Athene_128K_v1.02
    parameters:
      weight: 1.0
  - model: invisietch/L3.1-70Blivion-v0.1-rc1-70B
    parameters:
      weight: 1.0
  - model: nonetrix/llama-3.1-70B-lumitron-lorablated
    parameters:
      weight: 1.0
  - model: NobodySpecial/Llama-3.1-70B-Instruct-Lorablated-Creative-Writer
    parameters:
      weight: 1.0
  - model: MaziyarPanahi/calme-2.3-llama3.1-70b
    parameters:
      weight: 1.0
  - model: migtissera/Tess-3-Llama-3.1-70B
    parameters:
      weight: 1.0
base_model: Nexesenex/Llama_3.x_70b_L3.3_Athene_128K_v1.02
dtype: bfloat16
out_dtype: bfloat16
parameters:
  int8_mask: true
  normalize: true
  rescale: false
  filter_wise: false
  smooth: false
  allow_negative_weights: false
chat_template: auto
tokenizer:
  source: union