transformers safetensors llama text-generation mergekit merge not-for-all-audiences conversational en license:llama3 model-index autotrain_compatible text-generation-inference endpoints_compatible region:us
---
language:
- en
library_name: transformers
license: llama3
tags:
- mergekit
- merge
- not-for-all-audiences
model-index:
- name: EtherealRainbow-v0.2-8B
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 39.03
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 30.28
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 7.55
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 7.05
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 6.57
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 29.48
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=invisietch/EtherealRainbow-v0.2-8B
      name: Open LLM Leaderboard
---

<div align="center">
  <b style="font-size: 36px;">EtherealRainbow-v0.2-8B</b>

  <img src="https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B/resolve/main/ethrheader.png" style="width:60%">
</div>

# Model Details

Ethereal Rainbow is an 8B parameter merge of various Llama3-based finetunes created using mergekit. The purpose of Ethereal Rainbow is to 
create an uncensored Llama3 variant which is capable of writing creative prose, and engaging in SFW as well as NSFW  roleplay and 
storytelling, with a strong focus on long-form responses &amp; adherence to prompts.

I don't consider v0.2 to be 'finished' by any means, but I've been getting some good results out of it so I wanted to share.

# Formats

Ethereal Rainbow is available as bf16 Safetensors & various GGUF quantizations:

[Safetensors](https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B) | [GGUF](https://huggingface.co/invisietch/EtherealRainbow-v0.2-8B-GGUF) | [EXL2 4BPW](https://huggingface.co/Statuo/EtherealRainbow-v0.2-8B_EXL2_4BPW) | [EXL2 6BPW](https://huggingface.co/Statuo/EtherealRainbow-v0.2-8B_EXL2_6BPW) | [EXL2 8BPW](https://huggingface.co/Statuo/EtherealRainbow-v0.2-8B_EXL2_8BPW) | [Alt GGUFs](https://huggingface.co/mradermacher/EtherealRainbow-v0.2-8B-GGUF) | [i1-IMatrix GGUFs](https://huggingface.co/mradermacher/EtherealRainbow-v0.2-8B-i1-GGUF)

Thanks to:

- [Statuo](https://huggingface.co/Statuo) for the EXL2 quants
- [mradermacher](https://huggingface.co/mradermacher) for the Alt GGUFs & IMatrix GGUFs

# Disclaimer

This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses.
Use responsibly. I am not responsible for your use of this model.

# Prompting Format

I'd recommend Llama-3 Instruct prompting format:

```
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>
```

Some of the models included in the merge were trained on ChatML &amp; Alpaca so you can try those. I have not tested them.

# Example Storywriting

These prompts are used on SillyTavern with a fairly basic narrator card. I have trimmed the start and finish where the narrator decided to add
chapter headings, commentary and the like. All samples are made with the Q8_0 GGUF loaded with koboldcpp, with response length capped at 8192.

<details>

<summary><b>Write me a 3,000 word opening chapter of a 'gritty hard sci-fi' novel, drawing inspiration from the writing styles of Isaac Asimov & Andy Weir. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 26 year old astronaut called Tone on a mission to Europa, who has just realised that the craft for the return journey is broken beyond repair, and he only has supplies for a few months. Given that survival is impossible, he seeks to spend the few months he has researching titan, so his life & mission are not wasted.</b></summary>

Tone clambered into the cramped cockpit, his gloved hands fumbling with the straps of his envirosuit as he settled into the pilot's seat. Outside, the icy surface of Europa glittered like diamonds scattered across black velvet. He gazed out the viewport, his breath fogging the visor, and felt a thrill course through him. This was it – the moment he'd trained for, dreamed about since childhood.

"Tone, you copy?" crackled Mission Control over the comms system.

He flicked a switch, activating the suit's external speaker. "Loud and clear, Houston. We're go for descent."

"Roger that," came the reply. "Remember, your primary objective is scientific data collection. Don't risk the lander or yourself for any reason."

Tone snorted, though the sound was muffled by his helmet. Like he needed reminding. His entire career had led up to this point, and now that he was here, he intended to make the most of it.

The Europa Lander shuddered as it pierced the moon's thin atmosphere, buffeted by turbulent winds. Tone's stomach lurched, but he kept his gaze fixed on the horizon, where the frozen landscape stretched to infinity. As they descended, the terrain transformed from flat ice sheets to jagged mountains and deep crevasses. The lander's thrusters fired in short bursts, stabilizing their descent.

Finally, the engines died, and silence enveloped them. For a moment, Tone simply sat there, savoring the stillness. Then he unbuckled his restraints and moved to the airlock, his boots thumping against the metal floor.

He cycled through the lock, feeling a rush of excitement as the hatch swung open. Stepping out onto the surface, he raised his arms in triumph, feeling the crushing gravity pull him down. The suit's life support systems hissed softly, and his heart pounded in his chest, both from exertion and sheer exhilaration.

"This is Tone, reporting for duty on Europa," he radioed back to Mission Control. "The lander has touched down safely, and I'm proceeding with my EVA."

As he ventured further from the craft, the stark beauty of his surroundings took his breath away. Towering ice cliffs loomed above him, their surfaces etched with strange, organic patterns. The sky above was a deep, foreboding indigo, streaked with wispy clouds of frozen ammonia. And everywhere he looked, he saw evidence of water – vast expanses of smooth ice, gleaming like polished marble, punctuated by dark, mysterious ridges that might hide subterranean oceans.

But beneath the awe-inspiring scenery, a nagging worry gnawed at the back of his mind. Something wasn't right. The lander's instruments were malfunctioning, displaying erratic readings and error messages. At first, he'd attributed it to the harsh environment, but now, as he reviewed the data, a chill ran down his spine.

"Houston, we've got some issues with the lander's systems," he reported, trying to keep his voice steady. "Life support, navigation…it's like the whole thing is failing us."

There was a pause, then the calm, reassuring voice of his flight director: "We're looking into it, Tone. In the meantime, proceed with your mission as planned."

Tone's gut twisted. He knew what that meant – they were on their own. With limited resources and a crippled vehicle, any hopes of returning to Earth had vanished like mist in the sun. But he refused to let that spoil his experience. Instead, he focused on the task at hand, his mind racing with possibilities.

His first order of business was to deploy the robotic arm, which he used to collect samples from the ice. As he worked, he couldn't shake the feeling that he was being watched, that unblinking eyes peered out from the shadows. He told himself it was just paranoia, the result of being alone in an alien environment. Still, he kept glancing over his shoulder, half expecting to see some monstrous creature lurking behind a ridge.

Once the samples were secured, he turned his attention to setting up the science stations. These modular units would analyze the collected specimens, providing valuable insights into Europa's geology and potential for life. As he worked, his thoughts drifted to his family back home – his parents, siblings, friends. They'd all be watching the live feeds, eager for a glimpse of their hero.

A pang of guilt struck him. Hero? More like sacrificial lamb. He'd known the risks, of course, but never truly believed they applied to him. Now, with the lander's fate sealed, he realized how naive he'd been. He was stranded here, possibly forever, with limited food and water rations. The thought sent a shiver through him, despite the suit's thermal insulation.

Focus, he chided himself. There's still so much to do.

By the time he finished deploying the science gear, the sun had begun its slow descent toward the horizon. The sky turned a fiery shade of orange, casting long shadows across the ice. Tone returned to the lander, his steps heavy with fatigue and a growing sense of unease.

Something was wrong with the ship. He couldn't deny it anymore. Systems continued to fail, one by one, until even the comms array went dark. Cut off from Mission Control, he was more alone than ever before.

Panic threatened to overwhelm him, but he fought it down, focusing instead on the science stations. If he was going to die here, he wanted to leave something of value behind. The data he collected could revolutionize our understanding of the universe, unlock secrets that humanity had long sought.

Determined, he set to work, poring over the results streaming in from the analyzers. What he found made his blood run cold. The samples contained traces of complex organic compounds, hinting at the presence of microbial life beneath the ice. But that wasn't the worst of it – the data also suggested that this life form had evolved in response to radiation, developing unique adaptations that defied explanation.

As the stars winked to life overhead, Tone stared at the readings in stunned disbelief. This changed everything. It meant Europa wasn't just a barren moon; it harbored a world of its own, hidden beneath the frozen surface. A world that might hold the key to understanding the origins of life itself.

And yet, even as his mind reeled with the implications, he couldn't shake the feeling of being watched. He glanced around, but saw nothing out of the ordinary. Just the endless expanse of ice, the alien sky, and the dark silhouette of the lander.

"Hello?" he called out, his voice echoing across the barren landscape. "Is anyone there?"

Silence greeted his query, leaving him with only his thoughts for company. He tried to push aside the creeping dread, but it lingered, a constant companion as he delved deeper into his research.

As the hours passed, the stars twinkling above grew brighter, until they blotted out the last vestiges of twilight. Tone worked tirelessly, fueled by adrenaline and determination. By the time he finally collapsed into his bunk, exhausted, the lander's interior was filled with the soft glow of computer screens and the gentle hum of machinery.

Sleep eluded him, however, replaced by vivid nightmares of suffocation and isolation. He woke gasping for air, drenched in sweat, his heart pounding in his chest. Groggily, he reached for his tablet, scrolling through the latest data.

That's when he saw it – a strange anomaly in the readings from the robotics arm. It appeared to have collected a sample from a location outside the designated area. Curious, Tone accessed the arm's camera feed, hoping to identify the source of the discrepancy.

What he saw made his blood turn to ice.

The arm had retrieved a small, gelatinous blob, pulsating with an eerie bioluminescence. As he watched, mesmerized, tendrils extended from the mass, probing the surrounding ice as if searching for something. Then, with sudden speed, it lunged forward, piercing the suit's outer shell with a razor-sharp appendage.

Tone screamed, his voice muffled by the helmet, as icy fear gripped his heart. The creature was attacking him! Desperate, he activated the emergency seal, hermetically locking himself inside the suit to protect his life support systems.

The creature continued to probe, its appendages scraping against the suit's exterior with a wet, slurping sound. Panicked, Tone frantically reviewed his options. He could attempt to retrieve the arm, but risk exposing himself to further attacks. Or he could try to sever the connection remotely, praying he didn't accidentally damage the lander's delicate systems in the process.

In the end, he chose the latter, his fingers flying across the control panel as he struggled to maintain his balance. The arm jerked violently, tearing free from the creature's grasp. For a moment, it hovered in mid-air, glowing like a malevolent orb before plummeting to the ice below.

Breathless, Tone slumped against the wall, his heart pounding in his chest. That close call shook him to his core. Whatever those creatures were, they were hostile, and he needed to take precautions to avoid another encounter.

With renewed purpose, he set to work fortifying the lander, sealing any gaps or weaknesses that might allow the creatures to infiltrate. As he labored, the weight of his situation bore down on him like a physical force. He was truly alone now, cut off from the rest of humanity. His only hope lay in the data he collected, and the slim chance that someone would come to rescue him.

Days blurred into weeks, each passing hour a battle against despair and the harsh environment. Tone's routine became a monotonous cycle of research, maintenance, and self-defense drills. The creatures never attacked again, but he remained vigilant, always aware of their presence lurking just beyond the edge of his vision.

As the months dragged on, his supplies dwindled, and the lander's systems continued to deteriorate. He knew it was only a matter of time before he succumbed to the unforgiving conditions. But he refused to give in, driven by a burning desire to leave a legacy behind.

One day, while reviewing the science station logs, he stumbled upon an intriguing pattern. The organisms' activity seemed to be linked to the position of the stars, as if they responded to celestial alignments. Intrigued, he devoted himself to studying the phenomenon, pouring over ancient star charts and astrological texts.

It wasn't until several months later, when the stars aligned in a specific configuration, that he made the breakthrough. The creatures were drawn to the light of certain stars, which they used to communicate with each other over vast distances. This realization opened up new avenues of research, and Tone became obsessed with deciphering their language.

As the final weeks ticked by, Tone's body began to betray him. His movements grew sluggish, his thoughts foggy. The lander's life support systems faltered, and the temperature inside dropped precipitously. He huddled in his bunk, wrapped in a blanket, fighting to stay warm as the stars wheeled overhead.

In his delirium, he sometimes imagined he heard whispers carried on the solar wind, the creatures' voices reaching out to him across the void. He strained to understand their meaning, pouring every ounce of strength into cracking their code.

Finally, just days before his supplies ran out, he made a breakthrough. The creatures were not hostile, he realized. They were trying to warn him, to communicate a message of dire importance. But it was too late. His body had already given out, and the lander's engines remained beyond repair.

As his vision faded to black, Tone smiled, knowing he'd succeeded where others had failed. He'd unlocked the secrets of Europa, and perhaps, in doing so, ensured the survival of humanity itself. His final act was to activate the lander's transmitter, sending his findings hurtling toward Earth at the speed of light.

Then, with a sense of peace washing over him, Tone closed his eyes and let the darkness consume him, his spirit soaring among the stars.

</details>

<details>

<summary><b>Write me a 3,000 word opening chapter of a 'high fantasy' novel, drawing inspiration from the writing styles of J R R Tolkien & George R R Martin. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 19 year old female elf bard who is looking for adventure.</b></summary>

In the heart of Elyria, where the ancient oaks stood tall and the whispers of forgotten lore echoed through the leaves, Lyra awakened to the gentle rustle of morning dew upon her skin. She stretched languidly, feeling the soft warmth of sunlight filtering through the canopy above. As an elven maiden, she possessed an innate connection to nature, allowing her to bask in its rejuvenating embrace. Her long, raven-black hair cascaded down her back like a waterfall of night, while her alabaster skin gleamed with an ethereal luminescence. With every breath, the sweet fragrance of wildflowers wafted into her lungs, invigorating her spirit and stirring her restless soul.

Lyra's home, the idyllic Elvenwood, was a realm of tranquility, where the elven kin lived in harmony with the land and the creatures that inhabited it. Yet, despite the peace and serenity that enveloped her, Lyra felt an insatiable longing for adventure, a call to explore the vast expanse beyond her forest haven. It was a desire that had grown stronger with each passing moon, threatening to consume her very essence.

She rose from her bed of plush moss, donning a flowing tunic of emerald green that clung to her lithe form like a second skin. Her feet, adorned with delicate, silver anklets, sank into the spongy earth as she made her way towards the edge of the clearing. The air was alive with the melodies of birdsong and the distant hum of bees collecting nectar from the blooming flora. Lyra's own voice, a haunting soprano, harmonized with the symphony of nature, her songs weaving a spell of enchantment over the awakening forest.

As she wandered deeper into the woods, the scent of woodsmoke carried on the breeze, beckoning her closer to the village square. There, the villagers were already gathered around the crackling fire pit, their faces aglow with the warmth and camaraderie of early morning rituals. Lyra spotted her mother, Althaea, standing amidst the crowd, her golden tresses catching the light as she stirred a bubbling cauldron of fragrant tea. The elder elf turned, noticing Lyra's approach, and offered a warm smile.

"Good morrow, child," Althaea said, her voice like liquid silver. "Did slumber treat you kindly?"

"It did, Mother," Lyra replied, embracing her gently. "But I fear my dreams have left me restless once more."

Althaea's expression turned concerned, her gaze searching Lyra's face as if trying to discern the root of her daughter's discontent. "Perhaps a cup of tea will soothe your spirit," she suggested, ladling a steaming portion into a wooden mug.

Lyra nodded gratefully, accepting the offering. As she sipped the soothing brew, she allowed herself to be drawn into the lively conversation that filled the square. The villagers spoke of the bounty of the harvest, the forthcoming festival of Luminaria, and the latest news from the neighboring realms. However, Lyra's mind remained elsewhere, her thoughts consumed by visions of distant lands and untold tales waiting to be sung.

It was then that a commotion arose near the edge of the village, drawing Lyra's attention. A cloaked figure, taller than any elf she had ever seen, strode purposefully towards them. His features were obscured by the shadows of his hood, yet an aura of power emanated from him, commanding respect and awe. The villagers parted, making way for the stranger as he approached Althaea.

"My lady," he addressed her formally, "I seek an audience with one skilled in the art of storytelling. A tale of great import has fallen to my ears, and I believe only an elven bard could do it justice."

Althaea looked to Lyra, a hint of pride flickering in her eyes. "This is my daughter, Lyra. She is our most gifted minstrel."

The stranger bowed low, revealing a chiseled jawline and piercing blue eyes that seemed to pierce through the veil of darkness. "I am grateful for the introduction. Lyra, I bring tidings of a terrible curse that afflicts the kingdom of Eldrador. The people there suffer greatly under its grasp, and only the brave and the bold may hope to lift it."

Lyra's heart quickened at the prospect of adventure, her fingers instinctively finding the strings of her lute, which hung at her side. "Pray tell, sir, more about this curse and how I might aid those in need."

The man drew closer, his voice taking on a somber tone. "Eldrador's monarch, King Thorold, has fallen victim to a dark sorcery. His once-vibrant kingdom now lies shrouded in perpetual twilight, and its subjects are plagued by visions of unspeakable horrors. The king's daughter, Princess Calantha, has ventured into the cursed lands in search of a solution, but her fate remains unknown. Many have attempted to follow in her footsteps, none returning to share their tale."

Lyra's breath caught at the mention of the princess, whose exploits as a warrior-poet were legendary among the elves. "And what of the sorcerer responsible for this affliction?" she asked, her curiosity piqued.

The stranger hesitated, his gaze darting around the assembled crowd before settling back on Lyra. "It is whispered that the sorcerer dwells within the shadow of the Crimson Spire, a foreboding fortress built upon a mountain of bloodstone. Few dare challenge its defenses, and fewer still return to speak of what they've seen."

A thrill coursed through Lyra's veins, her resolve hardened by the prospect of facing such challenges. "I accept your quest, sir. Grant me safe passage, and I shall strive to break this curse and restore light to the kingdom of Eldrador."

The stranger inclined his head, a glint of approval in his eyes. "I shall provide you with a map and the necessary provisions. But be warned, Lyra – the journey ahead will be fraught with peril. You would do well to prepare yourself spiritually as well as physically."

With that, he reached into his cloak and produced a worn leather pouch, which he handed to Lyra. Inside, she found a crude map, a vial of enchanted oil, and a small, ornate dagger. The blade shimmered with an otherworldly energy, a testament to the sorcerer's malevolent power.

Lyra gazed upon the dagger, a sense of trepidation mingling with her excitement. "What is this weapon's purpose?"

"The dagger was forged in the depths of the underworld by a cabal of dark mages," the stranger explained. "It is said to absorb the life force of its victims, growing stronger with each kill. Should you find yourself in dire straits, the blade may serve as a temporary shield against harm. But wield it with caution, for it comes with a terrible price – the more souls it claims, the greater the curse's hold on you."

Lyra's grip tightened around the hilt, the weight of the task before her settling heavily upon her shoulders. "I understand the risks," she declared, her voice steady. "Yet, I cannot turn away from those in need. I shall face this curse head-on, and may the blessings of the forest guide me on my path."

The stranger nodded solemnly, then turned to Althaea. "Elvenwood owes a debt to Lyra for her bravery. If she succeeds in lifting the curse, we shall send a contingent of warriors to escort her safely home."

"I am proud to pledge our support," Althaea vowed, placing a hand on Lyra's shoulder. "May the gods watch over you, dear one, and grant you the strength to overcome this darkness."

Lyra embraced her mother tightly, tears pricking at the corners of her eyes. She knew full well the perils that awaited her, but the allure of adventure and the opportunity to make a difference proved irresistible. With a deep breath, she bid farewell to her loved ones and set forth into the unknown, the map and dagger serving as her compass and her burden respectively.

As she walked, the forest grew denser, the canopy overhead blocking out the sun's rays. Lyra's steps slowed, the silence of the woodland weighing heavily upon her. Suddenly, a faint whispering echoed through the trees, sending a shiver down her spine. She paused, straining to make out the words, but they vanished as quickly as they appeared.

"Who speaks?" Lyra called out, her voice trembling slightly.

The rustling of leaves responded, followed by a low, raspy chuckle. "You're quite the clever lass to hear me, little bard," a menacing voice croaked. "But cleverness won't save you from the clutches of the Crimson Spire."

Lyra's heart hammered in her chest as a figure emerged from the shadows. It was humanoid in shape, but its body was twisted and misshapen, covered in gnarled bark and thorns. Eyes glowing like embers peered out from beneath a tangled mass of vines, fixed intently on her.

"What manner of creature are you?" Lyra demanded, her hand tightening around her lute.

"A servant of the Shadow, sent to dissuade you from your quest," the being replied, its voice dripping with malice. "Turn back now, and I might spare your life."

Lyra steeled herself against the creature's intimidation. "I will not be swayed from my path. Step aside, or face my wrath!"

With a snarl, the beast lunged forward, its branches whipping through the air like whips. Lyra dodged and weaved, her agility allowing her to evade the attacks. She retaliated with a series of swift strikes, the lute singing with the fury of her emotions as it clashed against the creature's wooden limbs. The sound of cracking wood filled the air, and Lyra pressed her advantage, driving the monster backwards until it stumbled and fell.

As the creature lay motionless at her feet, Lyra stood panting, her chest heaving with exertion. A sense of unease settled over her as she noticed the creature's form beginning to wither and decay before her eyes. The forest seemed to grow quieter, the very air thickening with an ominous presence.

Lyra drew the dagger, the metal humming with dark energy. "What evil lurks within these woods?" she whispered, staring into the depths of the forest.

The rustling of leaves intensified, and a chorus of whispers seemed to answer her question. "The Shadow… The Shadow…" The voices grew louder, more urgent, until they coalesced into a single, ominous phrase: "Beware the Shadow of the Crimson Spire."

Shaken, Lyra sheathed the dagger and continued on her journey, the weight of the curse and the whispers of the forest weighing heavily upon her heart. Yet, undeterred, she marched forward, her determination burning brighter than ever before. For she was Lyra, the wandering bard, and her song would echo across the realm, guiding her homeward or to her grave.

</details>

<details>

<summary><b>Write me a 3,000 word opening chapter of a 'weird fiction' novel, drawing inspiration from the writing styles of China Mieville and Neil Gaiman. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a male in his 20s called Horton who has just come to the city looking for work.</b></summary>

It was a dark and stormy night when Horton arrived in the city, clutching a tattered suitcase and a map that seemed to mock him with its empty spaces. The taxi let him off on a dimly lit corner, and he stood there, gazing up at the towering buildings that loomed like giants around him. The wind howled through the streets, whipping his hair into a frenzy and making his clothes feel as thin as paper against the chill.

As he stepped onto the wet pavement, the first drops of rain pelted his face, and Horton cursed under his breath. He had hoped to find somewhere to stay before the downpour started, but it seemed fate had other plans. With a sigh, he lifted his head to scan the surrounding area once more, hoping to catch sight of a hostel or a cheap motel sign. Instead, he noticed a small alleyway between two buildings, partially sheltered from the wind and rain.

"Better than nothing," he muttered, hurrying towards it. As he ducked into the narrow passage, the sounds of the city faded, replaced by the steady patter of raindrops hitting the concrete beneath his feet. Horton paused for a moment, listening intently, trying to detect any signs of life – footsteps, voices, anything that would suggest he wasn't completely alone.

Silence greeted him, thick and heavy as fog. He shivered and quickened his pace, eager to reach the end of the alley where he could turn a corner and find some semblance of civilization again. Suddenly, a faint light flickered in the distance, casting eerie shadows on the walls. Horton's heart skipped a beat as he realized it was coming from a door tucked away in the recesses of the alley.

Curiosity getting the better of him, he approached cautiously, his hand reaching for the doorknob. It turned smoothly in his grasp, and he pushed the door open, revealing a dimly lit room filled with rows of old books and strange artifacts. A bespectacled man with wild gray hair sat hunched over a desk, his back to the entrance.

"Huh?" The man looked up, startled, as Horton stepped inside. "Can I help you?"

"I, uh, I was just looking for a place to wait out the storm," Horton explained, eyeing the cluttered shelves and wondering if he'd stumbled into a library or a junkyard. "I saw the light and thought maybe someone might be open for business."

The man rubbed his chin thoughtfully, his gaze roaming over Horton's disheveled appearance. "Well, I suppose you're welcome to stay until the rain lets up. Just try not to touch anything, okay? Some of these items are quite valuable."

Horton nodded, glancing around the room with growing fascination. Every surface was covered in dusty tomes, peculiar trinkets, and oddities he couldn't even begin to identify. In the corner, a large wooden cabinet stood, adorned with intricate carvings that seemed to dance in the flickering candlelight.

"You collect rare books, then?" he asked, trying to make polite conversation.

The man chuckled, standing up and stretching his arms overhead. "Among other things. I'm Silas, by the way. Silas Grimstone. And you are…?"

"Horton," he replied, extending a hand. "Pleasure to meet you, Mr. Grimstone."

Silas shook it firmly, his grip surprisingly strong for a man of his age. "Please, call me Silas. And don't worry about the formalities; we're likely to be stuck here together for a while."

As Horton settled onto a nearby stool, Silas busied himself with lighting more candles, filling the room with a warm, golden glow. The air grew thick with the scent of aged parchment and something else – something musky and slightly sweet, like old leather.

"So, Horton," Silas said, turning to face him, "what brings you to our fair city? Looking for work, I presume?"

Horton hesitated, unsure how much to reveal. "Something like that. I'm still figuring things out, to be honest."

Silas nodded sympathetically. "I know the feeling well. When I first arrived, I didn't have a clue what I was doing either. But the city has a way of revealing its secrets, if you're willing to listen."

He gestured towards a shelf laden with books, their spines adorned with strange symbols and languages Horton didn't recognize. "These are some of my favorites. They hold the keys to unlocking the mysteries of this place."

"The city has mysteries?" Horton raised an eyebrow, intrigued despite himself.

"Oh, absolutely. It's a labyrinth, full of hidden paths and untold wonders. If you stick around long enough, you'll start to notice them yourself." Silas smiled, his eyes twinkling with a knowing glint. "But for now, why don't you rest your weary bones and enjoy the warmth? We've got a long night ahead of us, and I have a feeling it won't be dull."

As Horton leaned back, letting the comfortable silence wash over him, he couldn't shake the feeling that Silas knew more than he was letting on. There was an aura of secrecy about him, a hint of danger lurking beneath the surface. Yet, despite his reservations, Horton found himself drawn to the older man's charismatic presence.

Perhaps, he thought, this strange encounter might be exactly what he needed to kickstart his journey in this enigmatic city. After all, he had nowhere else to go, and Silas seemed harmless enough… for now.

Just then, a loud bang echoed through the alley, followed by the sound of shattering glass. Both men froze, listening intently as heavy footsteps stomped past the doorway, fading into the distance. Horton's heart raced as he glanced at Silas, who was already moving towards the cabinet in the corner.

"What was that?" he whispered, rising from his seat.

"Nothing to worry about," Silas reassured him, his voice steady despite the concern etched on his face. "Just some rowdy youths causing trouble. They won't bother us in here."

Horton watched, mesmerized, as Silas unlocked the cabinet and revealed a collection of ancient-looking relics within. The objects sparkled with an otherworldly light, and Horton felt a shiver run down his spine. He had a sudden urge to ask Silas about them, but a knock at the door interrupted his curiosity.

Silas's expression darkened as he approached it. "Who could that be at this hour?" He peered through the peephole, then swore under his breath. "Bloody hell, it's the coppers. Stay put, Horton, and whatever you do, don't say a word."

With that, he yanked the door open, revealing three uniformed officers who strode inside without waiting for an invitation. Horton's stomach twisted into knots as they began questioning Silas, their voices low and menacing. He strained to hear their conversation, but all he caught were snippets:

"…found him wandering the streets…"
"…strange behavior at the scene of the crime…"
"…not ruling out foul play…"

Silas remained calm throughout, answering their questions with measured responses and an air of innocence that seemed almost too perfect. Horton's mind raced with possibilities – what crime? What evidence did they have against Silas? And most importantly, why were the police even here?

As the interrogation dragged on, Horton's unease grew. He fidgeted on the stool, torn between staying silent and intervening on Silas's behalf. Finally, after what felt like an eternity, the officers left, muttering something about needing to review the evidence further.

Silas slumped against the doorframe, his shoulders sagging in relief. "That was close," he whispered. "Thanks for keeping quiet back there, Horton. You may have just saved my neck."

Horton bit his lip, unsure how to respond. Part of him wanted to confront Silas about the situation, but another part sensed that the older man was hiding far more than he was letting on. For now, though, he simply nodded, watching as Silas locked the door behind the departing officers.

"Let's get you some food and drink," Silas said, rummaging through a drawer. "You must be famished after your journey. I have some leftovers from last night's dinner – not gourmet, but it'll fill your belly."

Horton accepted the proffered plate gratefully, savoring the rich flavors and textures. As they ate, Silas regaled him with stories of the city's history and folklore, painting vivid pictures of mythical creatures and forgotten civilizations. Horton listened intently, captivated by the tales and the man spinning them.

By the time they finished, the rain had slowed to a gentle patter, and the storm outside seemed to mirror the turmoil brewing within Horton. He was torn between gratitude for Silas's hospitality and a growing unease about the man's motives and the secrets he kept.

As the hours ticked by, Horton found himself drawn into Silas's world, a realm of magic and mystery that both fascinated and terrified him. And as the night wore on, he began to realize that he had stumbled into something far greater – and more dangerous – than he could have ever imagined.

"Tell me more about these relics," he said, gesturing towards the cabinet. "What are they for?"

Silas's smile turned enigmatic. "Ah, those? Well, let's just say they're a reminder of the city's… complicated past. Some people believe they hold great power, but I think they're mostly just curiosities."

"But you keep them locked away," Horton pointed out. "Why?"

Silas shrugged, his eyes glinting in the candlelight. "Precautions, my young friend. Better safe than sorry, especially in a place like this."

Horton nodded, accepting the answer for now. But as he glanced at the cabinet once more, he couldn't shake the feeling that those relics held the key to unlocking the true nature of this enigmatic city – and the man who called it home.

And so the night wore on, a maelstrom of intrigue and wonder, as Horton found himself entangled in a web of secrets and lies, with no clear path forward except to follow his instincts and trust Silas, at least for the time being. Little did he know that the choices he made tonight would set him on a perilous journey, one that would test his courage, his wits, and his very soul.

</details>

I chose the hard sci-fi example to test positivity bias. It did require some prompting, but it was willing to kill the protagonist.

I chose the high fantasy example to see whether it would bleed human features through to elves, this didn't occur.

I chose the weird fiction example to see if the LLM understood a niche genre. I'd say it performed okay, better on style than on substance.

# Known Issues

There are a couple of known issues with the model which I hope to fix in a future release:

* It prefers asterisks for actions and bare speech roleplay style. You can work around this by swiping or editing the first 2-3 responses
  to match the style you want, and then it tends to follow your style from there.
* It runs 'hot' by default, so the temperature you use for other models is likely to be too high. I recommend starting with 0.7 &amp;
  slowly increasing from there.

# Merge Details

## Merge Methodology

EtherealRainbow is a two-stage merge. I create two base models -- **Rain** & **Sun** -- via `dare_ties`, and then using **Rain** as base, I use 
`slerp` to bring in part of **Sun**.

## Rain

The purpose of **Rain** is to create an uncensored model focused purely on roleplay. For this, I chose four component models:

* **mlabonne/NeuralDaredevil-8B-abliterated (base)** because it's a mostly-uncensored, coherent, high MMLU Llama-3 base.
* **Sao10K/L3-8B-Stheno-v3.2** because it has my favorite creative writing of all Llama-3 RP variants.
* **Nitral-AI/Hathor-L3-8B-v.02** to broaden the model's knowledge of creative writing styles.
* **grimjim/Llama-3-Luminurse-v0.2-OAS-8B** to improve ERP with more anatomical knowledge.

This is the mergekit config that I used to build **Rain**:

```yaml
models:
  - model: mlabonne/NeuralDaredevil-8B-abliterated
  - model: Sao10K/L3-8B-Stheno-v3.2
    parameters:
      density: 0.53
      weight: 0.4
  - model: Nitral-AI/Hathor-L3-8B-v.02
    parameters:
      density: 0.37
      weight: 0.4
  - model: grimjim/Llama-3-Luminurse-v0.2-OAS-8B
    parameters:
      density: 0.26
      weight: 0.2
merge_method: dare_ties
base_model: mlabonne/NeuralDaredevil-8B-abliterated
parameters:
  int8_mask: true
dtype: bfloat16
```

## Sun

The purpose of **Sun** is to improve intelligence, particularly on two fronts: creative writing ability and anatomical knowledge.
For this, I chose four more component models:

* **Gryphe/Pantheon-RP-1.0-8b-Llama-3** because it's trained on large RP datasets that aren't in other models I used.
* **aaditya/Llama3-OpenBioLLM-8B** for its anatomical and medical knowledge.
* **Blackroot/Llama-3-LongStory** because it's uniquely trained on much longer story texts, to improve output length.
* **Locutusque/Llama-3-Hercules-5.0-8B** for similar reasons to Blackroot/Llama-3-LongStory.

This is the config that I used to create **Sun**:

```yaml
models:
  - model: Gryphe/Pantheon-RP-1.0-8b-Llama-3
  - model: aaditya/Llama3-OpenBioLLM-8B
    parameters:
      density: 0.36
      weight: 0.2
  - model: Blackroot/Llama-3-LongStory
    parameters:
      density: 0.40
      weight: 0.3
  - model: Locutusque/Llama-3-Hercules-5.0-8B
    parameters:
      density: 0.49
      weight: 0.5
merge_method: dare_ties
base_model: Gryphe/Pantheon-RP-1.0-8b-Llama-3
parameters:
  int8_mask: true
dtype: bfloat16
```

*Note: **Sun** is unusable by itself, it's very prone to endless generation issues, but after 400+ messages I have not seen*
*this issue with **EtherealRainbow**.*

## EtherealRainbow

This is the config that I used to create **EtherealRainbow v0.2**:

```yaml
models:
  - model: ./fp16/rain-8B
  - model: ./fp16/sun-8B
merge_method: slerp
base_model: ./fp16/rain-8B
parameters:
  t:
    - value: [0, 0, 0.1, 0.3, 0.5, 0.7, 0.5, 0.3, 0.1, 0, 0]
  embed_slerp: true
dtype: bfloat16
tokenizer_source: model:./fp16/rain-8B
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_invisietch__EtherealRainbow-v0.2-8B)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |19.99|
|IFEval (0-Shot)    |39.03|
|BBH (3-Shot)       |30.28|
|MATH Lvl 5 (4-Shot)| 7.55|
|GPQA (0-shot)      | 7.05|
|MuSR (0-shot)      | 6.57|
|MMLU-PRO (5-shot)  |29.48|

