README.md

---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- qwen
- qwen2.5
- finetune
- dpo
- orpo
- qwen2
- chat
- conversational
- instruct
- storywriting
- roleplay
- novelwriting
base_model:
- Qwen/Qwen2.5-14B-Instruct
- v000000/Qwen2.5-14B-Gutenberg-1e-Delta
- tanliboy/lambda-qwen2.5-14b-dpo-test
datasets:
- jondurbin/gutenberg-dpo-v0.1
- Qwen/Qwen2.5-14B-Instruct
- HuggingFaceH4/ultrafeedback_binarized
pipeline_tag: text-generation
model-index:
- name: Qwen2.5-Lumen-14B
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 80.64
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 48.51
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 0.0
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 10.4
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 10.29
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 43.36
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/Qwen2.5-Lumen-14B
      name: Open LLM Leaderboard
---

# Qwen2.5-Lumen-14B

* *Qwen direct preference optimization finetuned for ~3 epochs.*

![wCcJkdrVDUH6m0AN9Lv3B~2.png](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/OzGcIaEhVXZiSLDY3JZ-H.png)

<b>A qwen2.5 preference finetune, targeting prompt adherence, storywriting and roleplay.</b>

-------------------------------------------------------------------------------

* *Llama.cpp*

# (GGUF) Thanks QuantFactory
* static - [GGUF](https://huggingface.co/QuantFactory/Qwen2.5-Lumen-14B-GGUF)

# (GGUF) Thanks mradermacher
* static - [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Lumen-14B-GGUF)
* imatrix - [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Lumen-14B-i1-GGUF)

# (GGUF) Thanks Triangle104
* static - [Q8_0](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q8_0-GGUF) - [Q6_K](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q6_K-GGUF) - [Q5_K_M](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q5_K_M-GGUF) - [Q5_K_S](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q5_K_S-GGUF) - [Q5_0](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q5_0-GGUF) - [Q4_K_M](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q4_K_M-GGUF) - [Q4_K_S](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q4_K_S-GGUF) - [Q4_0](https://huggingface.co/Triangle104/Qwen2.5-Lumen-14B-Q4_0-GGUF)

*Other quant repositories also exist on huggingface and can be searched for.*

## Training Notes

Trained [Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) for 2 epochs on NVidia A100, and on dataset [jondurbin/gutenberg-dpo-v0.1](https://huggingface.co/datasets/jondurbin/gutenberg-dpo-v0.1), saving different checkpoints along the way (completely different runs at varying epochs and learning rates).

[Tanliboy](https://huggingface.co/tanliboy) trained [Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) for 1 epoch on [HuggingFaceH4/ultrafeedback_binarized](HuggingFaceH4/ultrafeedback_binarized), (Credit to Tanliboy! *Check out the model [here](https://huggingface.co/tanliboy/lambda-qwen2.5-14b-dpo-test)*)

*Mass checkpoint merged, Based on Qwen2.5-14B-Instruct (Base Model).*

## Merge

* Merged with a sophosympatheia's <b>SLERP</b> gradient *"Ultrafeedback-Binarized DPO"* and *"Gutenberg DPO"*

* Merged with a sophosympatheia's <b>SLERP</b> gradient *"Qwen2.5-14B-Instruct"* and *"Gutenberg DPO"*

* Merged all <b>DPO checkpoints</b> and <b>SLERP</b> variations with <b>MODEL_STOCK</b> to analyze geometric properties and get the most *performant* aspects of all runs/merges. *Model Stock* was chosen due to the similarity between the merged models.

* This was chosen due to the fact that evaluation for *ORPO* is unclear, so it's hard to know which runs are the best.

## One-Attempt generated example:

* *Temp 1.3 [1], Min_P 0.012 [4], TFS 0.97 [3], Smooth_Factor 0.3 [2], Smoothing_Curve 1.1, Rep 1.1, Rep Range 1000*

<details>
  <summary> >[CLICK ME] Write the opening chapter of a romance novel set in the late 1800s, where the protagonist, a war-weary music star returning to England after an injury in Afghanistan service...</summary>

**Title: A Harp for the Heart**

*Chapter One: Homebound*

England, Winter 1893—
As the steam-powered ferry made its way across the turbulent waters of the Channel, each tremor that reverberated through the vessel felt like a calloused reminder to Edgar Thornton. His mind was already conjuring visions of his beloved London—the sprawling city draped in an ever-present blanket of melancholic grey. Yet within him lingered more pressing memories: the whizzing sound of artillery shells as they streaked across Afghan deserts, leaving behind billowing black smoke and devastating wreckage.

Edgar's injuries were visible, the ghostly scar of a saber cut marring one cheek; less evident, however, was the toll that service took on his spirit. Each warbling note from an opera played by some amateur on the ferry sliced into Edgar's hollow heart, echoing through him with a bitter lament—the song was *Adagio* and it meant *Slow down*. Only now did he dare allow himself to slow—decades since music had flowed through the seams of his body in this way.

For years he had been sought out, from England to New York, as the country's foremost harpist, not just for his fingers nimble on a range of scales but also for how each piece conveyed an insurmountable melancholy—a reflection of his solitary life and unending loneliness. Each recital saw thousands of faces that remembered his smiles or silences; yet today he had no one to return them to.

The ferry's horn blared as they approached Dover, cutting through Edgar’s reverie much the same as cannonade did in Kabul. He thought about London with mixed emotions—longing for the grandeur and splendor of its concert halls, a city vibrant under lamplight but devoid of comfort or real life; a home where he was no less alien than anywhere else.

A soft murmur drew his gaze to a young woman standing near the window, her back to him. Her blonde hair cascaded in waves, shimmering under the lantern light as if lit from within. Despite the chill of the approaching winter night, she appeared oblivious—entirely entranced by something on the far horizon or lost in thought.

His breath caught at a sensation that had been uncommon for years but familiar to his boyhood—a light flutter tickling just behind his ribcage. And as her shoulders drew back and her chin tipped upwards, he saw a hint of the same unspoken story reflected in her eyes: dreams big enough to leave them scarred by distance and time.

And then she spoke without ever turning around—

"Do you hear that?" she asked into the gathering darkness outside their window. "Is that London?"[...]
</details>

-----------------------------------

<details>
  <summary> >[CLICK ME] Write the opening chapter of a dark fantasy book during the darkest age, where the protagonist, a war-weary woman is introduced returning to stand against the orcs and fight in what they falsely  believe is the last battle. Introduce us as she is riding into her army to talk to some friends before they strike... </summary>

---

The night was pitch black with only the thinnest sliver of a moon slicing across the sky, barely enough to paint the horizon a sickly shade of greyish-blue. Below it stretched an expanse of rolling hills and forests that swayed like shadows in an eternal nightmare. The wind carried with it not the scents of living earth but the taint of the bloodied battlegrounds below, where for too long had men, women, and beasts clashed and bled.

Ariath rode amidst this gloomy spectacle atop a pale stallion named Ebon, a beast as weary and war-weary as she was. Her dark cloak billowed around her like an inky cloud, contrasting starkly with the barren landscape through which they traveled. She had not chosen such darkness—nor any choice at all; tonight, every decision from waking to fighting lay in someone else's cold, indifferent hands.

Ebon’s hooves stirred the heavy mud of long trampled fields, sinking silently into the earth as he treaded towards her rallying army. It was said that this night they would fight, not for honor nor country—no, those noble phantoms had vanished many a moon past—but to stand once more between the realm and its approaching destruction.

Her chestnut mare, a beauty in years past before every bone seemed like it could break from its own weight, whinnied quietly under Ariath's calming voice. Her friends awaited not far ahead; they were an eclectic bunch—half-brother Raen, the steadfast knight who fought by her side for almost all their battles, Elaeril the mage known more as a wielder of shadow spells than life-giving magic, and finally Meryn the roguish, whose sword was as quick as his wit.[...]
</details>

-----------------------------------

* *Temp 1.3, Min_P 0.012, Rep 1.1*

<details>
  <summary> >[CLICK ME (<b>Disclaimer: highly developed prompt from dataset - best case scenario)</b>] Summary of previous portion of the novel: In the previous chapter, Anna Pavlovna is overwhelmed by a deep sense of fear and dread after her partner, Count Vronsky, has left. Despite trying to reach out to him with a note pleading for his return, she is informed that the count had driven off, and her message did not reach him. Struggling with her thoughts and barely able to find solace, Anna decides to send a telegram and visit Darya Alexandrovna to avoid succumbing to her overwhelming feelings. Her distress is palpable as she attempts to take action to distract herself from her anxiety and looks for any form of support or distraction to keep her from unraveling. Write the next chapter of a story in novel format that focuses on the internal and external conflicts of a woman named Anna. Detail her poignant reflections on her past, her struggle with her current sense of self-worth, and her interactions with family and society. Include flashbacks to her younger days, comparisons between her past and present, and vivid descriptions of her environment and society's norms that have led to her feelings of alienation. This chapter should showcase Anna's complex emotional state as she contemplates her next steps in life, hinting at recent life-altering decisions and their repercussions. It should contain interactions with characters such as Dolly and Kitty, reflecting the tensions and societal judgments she is grappling with.</summary>
  
  ### Chapter Seven

It was early morning, but the heavy clouds of melancholy still clung tightly to Anna's heart like a shroud over her spirit. The atmosphere outside mirrored this burden; rain fell softly, soaking the flower beds as mournful streaks rolled lazily across the gray sky in muted tones of despair.

Anna could not have taken her steps outwards to face the world until she had come to some kind of resolution within herself, even if it were no more than acceptance. She reflected deeply over how far away everything now seemed from what was once close and familiar. In every leaf trembling on a wet branch, in the low sighs of a wind that wandered lost without a purpose across the gray fields, she felt echoes of her inner turmoil.

She remembered her younger days. Life had then been filled with smiles, warmth and acceptance — her friends called her Annaarina, the spirit they saw in her so vividly, bright-eyed and kind. Their little gathering at Kitty’s had been an annual celebration for all four families: Oblonskys’, Shtcherbats’ys’, Karenins’, and Count Vronsky’s parents — each as familiar and loved a presence in that room as Anna.

But where were their faces now? Where was her happiness from those times, which only seemed to taunt the present?

"Here we are again,” she murmured in self-accusation with a mocking bitterness. The first snow had fallen softly upon them last winter then — and today it reminded her of something too lovely and cruel; something not allowed now in the rigid confines that life had chosen as its own path.

Determined to avoid being weighed down further, Anna prepared herself mentally to face Dolly, her dearest friend amongst women who seemed like a beacon among dim lights. In moments such as these she believed fully that somehow they could ease or even rid this dark fog from her heart — yet Anna felt apprehension nevertheless. Would their kinship and bond stand unchanged in light of recent circumstances? Perhaps now it would reveal cracks that Dolly was oblivious to, since the old Annaarina had been seen so rarely even then.

---

The carriage jolted through wet cobblestones as Anna finally made her way to Darya Alexandrovna’s house. The house stood there as warmly and welcomingly familiar with smoke still rising from chimneys, signaling signs of comfort within. Anna pulled off a shivering hand — dampness and cold seeped in merciless through fine silks.

Sneezing lightly into lace-lined white gloves, Anna steadied herself before rapping at the door. There would be no more room for her to hide; any attempt at self-preservation now lay broken between her heart and truth.

Anna stepped inside, greeted by the sight of Dolly busying about an unswept floor. For a fleeting moment, those familiarly busy hands paused — brows furrowed gently in concern even before their full attention met Anna.

"Why so early? Is there something...?" began Dolly warmly yet tentatively as she approached to take the delicate gloved hand.

Anna bit her lip at Darya's solicitous face. Would she now have to divulge her secrets of how utterly unhinged everything had become, or simply hint enough for suspicion?

"I — I need your advice," Anna whispered shyly, for there was no courage left to handle the world straight-up that day.
Dolly hugged the fragile figure and replied earnestly, "Speak with me here freely. No matter what is troubling you..."

That simple reassurance gave Anna courage. The despairing weight lessened as though carried a little by this soft-hearted soul.

“I have changed more than you would ever understand...” began Anna softly, but the words broke as unshed tears finally welled.
"Your eyes tell their story alone," soothed Dolly, patting gently her friend’s hand. “Do not be ashamed.”

In the wake of support and open hearts, Anna continued her revelation, pouring out all fears and hesitations — she who had been so content to live life sheltered inside, was suddenly exposed in full force by necessity alone; fragile glass cast into a brutal world.

---

At intervals throughout that day, Anna could scarcely believe how swiftly and faithfully Darya’s love proved the eternal rock to stand against crumbling shores. Amidst shared tea and comfort that night though shadows still lingered darkly between them.

When asked about Count Vronsky's presence with the family at gatherings or personal times now, Anna met this inquiry with a breathy sorrowful silence which spoke volumes to both the depth of their new estrangement, as well as how truly beloved the name was yet by Anna still in spite all waning prospects for resolution.

Daria continued to encourage her friend steadfastly; even shared letters and small consolations to penultimate messages destined to Count Vronsky alone. Each offered another grain's worth of hope for peace - both with him, herself, and beyond her personal dilemmas within that cloaked society so mercilessly turning its back against any form of irregularities in married life.

By the end of their meeting late evening shadows were upon them once more, and Anna reluctantly made to depart — a newly found strength (but still sorely tested) rippling across the troubled soul. However, as Anna prepared herself for returning to her solitary chambers, Dolly rose swiftly off the settee with concern laced in her tone.

“But we have barely scratched the surface of what might truly be affecting your heart.” The sympathy etched upon her face almost spoke a plea for understanding.
Anna reached over and took hands of compassionate warmth — eyes wide with gratefulness.

"I think perhaps it is all right to stay here for the night… while I gather myself," she confided quietly into Darya’s caring ears.[...]
</details>

-----------------------------------

<b>*As you can see the model has mostly adapted to the intended response style from Gutenberg dataset.*</b>

## Recipe

```yaml
models:
  - model: v000000/Qwen2.5-14B-Gutenberg-1e-Delta
  - model: v000000/Qwen2.5-14B-Gutenberg-0.6e-Sequential
  - model: v000000/Qwen2.5-14B-Gutenberg-0.25e-Early
  - model: v000000/Qwen2.5-14B-Gutenberg-2e-Sequential
  - model: v000000/Qwen2.5-14B-Gutenberg-0.37e-Early
  - model: v000000/Qwen2.5-14B-Gutenberg-2e-Zeta
  - model: v000000/Qwen2.5-14B-Gutenberg-1e-Theta
  - model: tanliboy/lambda-qwen2.5-14b-dpo-test
  - model: v000000/Qwen2.5-14B-Gutenberg-1e-Delta
  - model: tanliboy/lambda-qwen2.5-14b-dpo-test
  - model: v000000/Qwen2.5-14B-Gutenberg-UltraLambda-Slerpeno
  - model: v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpeno
base_model: v000000/Qwen2.5-14B-Gutenberg-1e-Delta
merge_method: model_stock
dtype: bfloat16
```

*If your use case is character-based roleplay, please consider using the prompts below for an enhanced experience*
* [For realistic RP/non-RPGs - MarinaraSpaghetti ChatML Customized](https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings/tree/main/Customized/ChatML)
* [For freeform RP/RPGs - MarinaraSpaghetti ChatML Basic](https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings/tree/main/Basic/ChatML) 


### Finetune and merge

This is a merge and finetune of pre-trained language models.

###  Models Merged

[Arxiv 2403.19522](https://arxiv.org/abs/2403.19522)

The following models were included in the merge:
* v000000/Qwen2.5-14B-Gutenberg-1e-Delta
* v000000/Qwen2.5-14B-Gutenberg-0.6e-Sequential
* v000000/Qwen2.5-14B-Gutenberg-0.25e-Early
* v000000/Qwen2.5-14B-Gutenberg-2e-Sequential
* v000000/Qwen2.5-14B-Gutenberg-0.37e-Early
* v000000/Qwen2.5-14B-Gutenberg-2e-Zeta
* v000000/Qwen2.5-14B-Gutenberg-1e-Theta
* v000000/Qwen2.5-14B-Gutenberg-UltraLambda-Slerpeno
* v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpeno
* tanliboy/lambda-qwen2.5-14b-dpo-test

-------------------------------------------------------------------------------

- Context Length: Full 131,072 tokens and generation 8192 tokens
- Qwen2(ChatML) Prompt format
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_v000000__Qwen2.5-Lumen-14B)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |32.20|
|IFEval (0-Shot)    |80.64|
|BBH (3-Shot)       |48.51|
|MATH Lvl 5 (4-Shot)| 0.00|
|GPQA (0-shot)      |10.40|
|MuSR (0-shot)      |10.29|
|MMLU-PRO (5-shot)  |43.36|