--- base_model: - TareksLab/Erudite-V1-Unleashed-LLaMA-70B - TareksLab/RolePlayer-V4-LLaMa-70B - TareksLab/Anathema-V2-LLaMA-70B - TareksLab/Wordsmith-V2.0-LLaMa-70B - TareksLab/Scrivener-Base-V4-LLaMA-70B library_name: transformers tags: - mergekit - merge --- # merge This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [TareksLab/Erudite-V1-Unleashed-LLaMA-70B](https://huggingface.co/TareksLab/Erudite-V1-Unleashed-LLaMA-70B) as a base. ### Models Merged The following models were included in the merge: * [TareksLab/RolePlayer-V4-LLaMa-70B](https://huggingface.co/TareksLab/RolePlayer-V4-LLaMa-70B) * [TareksLab/Anathema-V2-LLaMA-70B](https://huggingface.co/TareksLab/Anathema-V2-LLaMA-70B) * [TareksLab/Wordsmith-V2.0-LLaMa-70B](https://huggingface.co/TareksLab/Wordsmith-V2.0-LLaMa-70B) * [TareksLab/Scrivener-Base-V4-LLaMA-70B](https://huggingface.co/TareksLab/Scrivener-Base-V4-LLaMA-70B) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: TareksLab/Wordsmith-V2.0-LLaMa-70B - model: TareksLab/Anathema-V2-LLaMA-70B - model: TareksLab/Scrivener-Base-V4-LLaMA-70B - model: TareksLab/RolePlayer-V4-LLaMa-70B merge_method: sce base_model: TareksLab/Erudite-V1-Unleashed-LLaMA-70B parameters: select_topk: 0.16 dtype: bfloat16 tokenizer: source: TareksLab/Scrivener-Base-V4-LLaMA-70B ```