README.md

metadata

license: apache-2.0
tags:
  - merge
  - mergekit
  - lazymergekit
  - shenzhi-wang/Llama3-70B-Chinese-Chat
  - nvidia/Llama3-ChatQA-1.5-70B

Llama3-70B-Fusion-ChatQA-Chinese

Llama3-70B-Fusion-ChatQA-Chinese is a merge of the following models using mergekit:

• shenzhi-wang/Llama3-70B-Chinese-Chat
• nvidia/Llama3-ChatQA-1.5-70B

Configuration

```yaml models:

• model: meta-llama/Meta-Llama-3-70B
• model: shenzhi-wang/Llama3-70B-Chinese-Chat parameters: density: 0.5 weight: 0.6
• model: nvidia/Llama3-ChatQA-1.5-70B parameters: density: 0.5 weight: 0.4 merge_method: dare_ties base_model: meta-llama/Meta-Llama-3-70B parameters: int8_mask: true dtype: bfloat16 ```