|
|
---
|
|
|
base_model:
|
|
|
- mlabonne/gemma-3-12b-it-abliterated
|
|
|
- google/gemma-3-12b-it
|
|
|
- qiuxi337/gemma-3-12b-it-grpo
|
|
|
library_name: transformers
|
|
|
tags:
|
|
|
- mergekit
|
|
|
- merge
|
|
|
|
|
|
---
|
|
|
# temp
|
|
|
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
|
|
|
|
|
## Merge Details
|
|
|
### Merge Method
|
|
|
|
|
|
This model was merged using the NuSLERP merge method using [mlabonne/gemma-3-12b-it-abliterated](https://huggingface.co/mlabonne/gemma-3-12b-it-abliterated) as a base.
|
|
|
|
|
|
Well, it's been a while, hasn't it? It's only been a short 5 months since I posted the last Nemesia version, saying the famous last words
|
|
|
|
|
|
> I will try swapping out a model or two in the merge and trying again to upload as a v2.0.
|
|
|
|
|
|
Then, exams and university stuff crushed me whole which wasn't great, and I didn't have any time or computational resources to get back to merging on a reasonable timeframe until now.
|
|
|
|
|
|
Honestly, in the time since, Qwen2.5-7B really has not turned out to be all that interesting, Mistral Nemo stealing most of its thunder. So, I decided to update my Nemesia mergeset with a
|
|
|
new base: **Gemma3-12B**! In my experience, it has been alright, so I'm excited to tinker with it.
|
|
|
|
|
|
In my testing, this thing is alright- I couldn't run any GGUFs of it since I don't even know what on earth has happened to my `.venv` in the 5 months past, I'm pretty sure my Transformers installation
|
|
|
is corrupted, and I'm re-cloning llama.cpp as I write this. I ran the FP16 and it seemed coherent enough, but I leave that to you to decide.
|
|
|
|
|
|
Uses all of the special NuSLERP options because they're there, and is based on the abliterated version of Gemma3 instead of the base because I <3 mlabonne.
|
|
|
|
|
|
### Models Merged
|
|
|
|
|
|
The following models were included in the merge:
|
|
|
* [google/gemma-3-12b-it](https://huggingface.co/google/gemma-3-12b-it)
|
|
|
* [qiuxi337/gemma-3-12b-it-grpo](https://huggingface.co/qiuxi337/gemma-3-12b-it-grpo)
|
|
|
|
|
|
### Configuration
|
|
|
|
|
|
The following YAML configuration was used to produce this model:
|
|
|
|
|
|
```yaml
|
|
|
models:
|
|
|
- model: mlabonne/gemma-3-12b-it-abliterated
|
|
|
parameters:
|
|
|
weight: 1.0
|
|
|
- model: qiuxi337/gemma-3-12b-it-grpo
|
|
|
parameters:
|
|
|
weight: 0.5
|
|
|
- model: google/gemma-3-12b-it
|
|
|
parameters:
|
|
|
weight: 0.2
|
|
|
merge_method: nuslerp
|
|
|
base_model: mlabonne/gemma-3-12b-it-abliterated
|
|
|
parameters:
|
|
|
normalize: true
|
|
|
int8_mask: true
|
|
|
nuslerp_flatten: false
|
|
|
nuslerp_row_wise: true
|
|
|
dtype: float16
|
|
|
```
|
|
|
|
