gguf arxiv:2309.09530 endpoints_compatible region:us
---
{}
---
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


medicine-LLM - GGUF
- Model creator: https://huggingface.co/AdaptLLM/
- Original model: https://huggingface.co/AdaptLLM/medicine-LLM/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [medicine-LLM.Q2_K.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q2_K.gguf) | Q2_K | 2.36GB |
| [medicine-LLM.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.IQ3_XS.gguf) | IQ3_XS | 2.6GB |
| [medicine-LLM.IQ3_S.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.IQ3_S.gguf) | IQ3_S | 2.75GB |
| [medicine-LLM.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q3_K_S.gguf) | Q3_K_S | 2.75GB |
| [medicine-LLM.IQ3_M.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.IQ3_M.gguf) | IQ3_M | 2.9GB |
| [medicine-LLM.Q3_K.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q3_K.gguf) | Q3_K | 3.07GB |
| [medicine-LLM.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q3_K_M.gguf) | Q3_K_M | 3.07GB |
| [medicine-LLM.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q3_K_L.gguf) | Q3_K_L | 3.35GB |
| [medicine-LLM.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.IQ4_XS.gguf) | IQ4_XS | 3.4GB |
| [medicine-LLM.Q4_0.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q4_0.gguf) | Q4_0 | 3.56GB |
| [medicine-LLM.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.IQ4_NL.gguf) | IQ4_NL | 3.58GB |
| [medicine-LLM.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q4_K_S.gguf) | Q4_K_S | 3.59GB |
| [medicine-LLM.Q4_K.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q4_K.gguf) | Q4_K | 3.8GB |
| [medicine-LLM.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q4_K_M.gguf) | Q4_K_M | 3.8GB |
| [medicine-LLM.Q4_1.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q4_1.gguf) | Q4_1 | 3.95GB |
| [medicine-LLM.Q5_0.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q5_0.gguf) | Q5_0 | 4.33GB |
| [medicine-LLM.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q5_K_S.gguf) | Q5_K_S | 4.33GB |
| [medicine-LLM.Q5_K.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q5_K.gguf) | Q5_K | 4.45GB |
| [medicine-LLM.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q5_K_M.gguf) | Q5_K_M | 4.45GB |
| [medicine-LLM.Q5_1.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q5_1.gguf) | Q5_1 | 4.72GB |
| [medicine-LLM.Q6_K.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q6_K.gguf) | Q6_K | 5.15GB |
| [medicine-LLM.Q8_0.gguf](https://huggingface.co/RichardErkhov/AdaptLLM_-_medicine-LLM-gguf/blob/main/medicine-LLM.Q8_0.gguf) | Q8_0 | 6.67GB |




Original model description:
---
language:
- en
datasets:
- Open-Orca/OpenOrca
- GAIR/lima
- WizardLM/WizardLM_evol_instruct_V2_196k
- EleutherAI/pile
metrics:
- accuracy
pipeline_tag: text-generation
tags:
- biology
- medical
---

# Adapting Large Language Models to Domains via Continual Pre-Training
This repo contains the domain-specific base model developed from **LLaMA-1-7B**, using the method in our **ICLR 2024** paper [Adapting Large Language Models via Reading Comprehension](https://huggingface.co/papers/2309.09530).

We explore **continued pre-training on domain-specific corpora** for large language models. While this approach enriches LLMs with domain knowledge, it significantly hurts their prompting ability for question answering. Inspired by human learning via reading comprehension, we propose a simple method to **transform large-scale pre-training corpora into reading comprehension texts**, consistently improving prompting performance across tasks in biomedicine, finance, and law domains. **Our 7B model competes with much larger domain-specific models like BloombergGPT-50B**. 

### 🤗 [2024/6/21] We release the 2nd version of AdaptLLM at [Instruction-Pretrain](https://huggingface.co/instruction-pretrain), effective for both general pre-training from scratch and domain-adaptive continual pre-training!!! 🤗

**************************** **Updates** ****************************
* 2024/6/22: Released the [benchmarking code](https://github.com/microsoft/LMOps/tree/main/adaptllm).
* 2024/6/21: 👏🏻 Released the 2nd version of AdaptLLM at [Instruction-Pretrain](https://huggingface.co/instruction-pretrain) 👏🏻
* 2024/1/16: 🎉 Our [research paper](https://huggingface.co/papers/2309.09530) has been accepted by ICLR 2024!!!🎉
* 2023/12/19: Released our [13B base models](https://huggingface.co/AdaptLLM/law-LLM-13B) developed from LLaMA-1-13B.
* 2023/12/8: Released our [chat models](https://huggingface.co/AdaptLLM/law-chat) developed from LLaMA-2-Chat-7B.
* 2023/9/18: Released our [paper](https://huggingface.co/papers/2309.09530), [code](https://github.com/microsoft/LMOps), [data](https://huggingface.co/datasets/AdaptLLM/law-tasks), and [base models](https://huggingface.co/AdaptLLM/law-LLM) developed from LLaMA-1-7B.


## Domain-Specific LLaMA-1
### LLaMA-1-7B
In our paper, we develop three domain-specific models from LLaMA-1-7B, which are also available in Huggingface: [Biomedicine-LLM](https://huggingface.co/AdaptLLM/medicine-LLM), [Finance-LLM](https://huggingface.co/AdaptLLM/finance-LLM) and [Law-LLM](https://huggingface.co/AdaptLLM/law-LLM), the performances of our AdaptLLM compared to other domain-specific LLMs are:

<p align='center'>
    <img src="https://cdn-uploads.huggingface.co/production/uploads/650801ced5578ef7e20b33d4/6efPwitFgy-pLTzvccdcP.png" width="700">
</p>

### LLaMA-1-13B
Moreover, we scale up our base model to LLaMA-1-13B to see if **our method is similarly effective for larger-scale models**, and the results are consistently positive too: [Biomedicine-LLM-13B](https://huggingface.co/AdaptLLM/medicine-LLM-13B), [Finance-LLM-13B](https://huggingface.co/AdaptLLM/finance-LLM-13B) and [Law-LLM-13B](https://huggingface.co/AdaptLLM/law-LLM-13B).

## Domain-Specific LLaMA-2-Chat
Our method is also effective for aligned models! LLaMA-2-Chat requires a [specific data format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2), and our **reading comprehension can perfectly fit the data format** by transforming the reading comprehension into a multi-turn conversation. We have also open-sourced chat models in different domains: [Biomedicine-Chat](https://huggingface.co/AdaptLLM/medicine-chat), [Finance-Chat](https://huggingface.co/AdaptLLM/finance-chat) and [Law-Chat](https://huggingface.co/AdaptLLM/law-chat)

For example, to chat with the biomedicine base model (**🤗we highly recommend switching to the [chat model](https://huggingface.co/AdaptLLM/medicine-chat) for better response quality!**):
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AdaptLLM/medicine-LLM")
tokenizer = AutoTokenizer.from_pretrained("AdaptLLM/medicine-LLM", use_fast=False)

# Put your input here:
user_input = '''Question: Which of the following is an example of monosomy?
Options:
- 46,XX
- 47,XXX
- 69,XYY
- 45,X

Please provide your choice first and then provide explanations if possible.'''

# Simply use your input as the prompt for base models
prompt = user_input

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
outputs = model.generate(input_ids=inputs, max_length=2048)[0]

answer_start = int(inputs.shape[-1])
pred = tokenizer.decode(outputs[answer_start:], skip_special_tokens=True)

print(f'### User Input:\n{user_input}\n\n### Assistant Output:\n{pred}')
```

## Domain-Specific Tasks
To easily reproduce our results, we have uploaded the filled-in zero/few-shot input instructions and output completions of each domain-specific task: [biomedicine-tasks](https://huggingface.co/datasets/AdaptLLM/medicine-tasks), [finance-tasks](https://huggingface.co/datasets/AdaptLLM/finance-tasks), and [law-tasks](https://huggingface.co/datasets/AdaptLLM/law-tasks).

**Note:** those filled-in instructions are specifically tailored for models before alignment and do NOT fit for the specific data format required for chat models.

## Citation
If you find our work helpful, please cite us:
```bibtex
@inproceedings{
cheng2024adapting,
title={Adapting Large Language Models via Reading Comprehension},
author={Daixuan Cheng and Shaohan Huang and Furu Wei},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=y886UXPEZ0}
}
```

