---
base_model:
  - Menlo/Jan-nano-128k
  - POLARIS-Project/Polaris-4B-Preview
  - mlabonne/Qwen3-4B-abliterated
  - ValiantLabs/Qwen3-4B-Esper3
library_name: transformers
tags:
  - mergekit
  - merge
license: apache-2.0
---

# Qwen3-4B-Merge-Variant-01

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details

### Merge Method

This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using [mlabonne/Qwen3-4B-abliterated](https://huggingface.co/mlabonne/Qwen3-4B-abliterated) as a base.

### Models Merged

The following models were included in the merge:

- [Menlo/Jan-nano-128k](https://huggingface.co/Menlo/Jan-nano-128k)
- [POLARIS-Project/Polaris-4B-Preview](https://huggingface.co/POLARIS-Project/Polaris-4B-Preview)
- [ValiantLabs/Qwen3-4B-Esper3](https://huggingface.co/ValiantLabs/Qwen3-4B-Esper3)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: della_linear
parameters:
  normalize: true
base_model: "mlabonne/Qwen3-4B-abliterated"
models:
  - model: "POLARIS-Project/Polaris-4B-Preview"
    parameters:
      weight: 1
  - model: "Menlo/Jan-nano-128k"
    parameters:
      weight: 1
  - model: "ValiantLabs/Qwen3-4B-Esper3"
    parameters:
      weight: 1
```