Update README.md
Browse files
README.md
CHANGED
@@ -2,10 +2,6 @@
|
|
2 |
license: mit
|
3 |
datasets:
|
4 |
- ncbi/pubmed
|
5 |
-
- starmpcc/Asclepius-Synthetic-Clinical-Notes
|
6 |
-
- akemiH/NoteChat
|
7 |
-
- zhengyun21/PMC-Patients
|
8 |
-
- jpcorb20/medical_wikipedia
|
9 |
language:
|
10 |
- en
|
11 |
base_model:
|
@@ -26,7 +22,7 @@ The MediPhi Model Collection comprises 7 small language models of 3.8B parameter
|
|
26 |
## Model Details
|
27 |
### Model Description
|
28 |
|
29 |
-
This model is `MediPhi` obtained by merging
|
30 |
|
31 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
32 |
- **Model type:** Phi3
|
@@ -90,7 +86,7 @@ Researchers should apply responsible AI best practices, including mapping, measu
|
|
90 |
|
91 |
torch.random.manual_seed(0)
|
92 |
|
93 |
-
model_name = "microsoft/MediPhi"
|
94 |
model = AutoModelForCausalLM.from_pretrained(
|
95 |
model_name,
|
96 |
device_map="cuda",
|
@@ -135,12 +131,6 @@ Check `microsoft/Phi-3.5-mini-instruct` for details about the tokenizer, require
|
|
135 |
|
136 |
Continual Pre-training:
|
137 |
- PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
|
138 |
-
- Medical Guideline `epfl-llm/guidelines`.
|
139 |
-
- Medical Wikipedia `jpcorb20/medical_wikipedia`.
|
140 |
-
- Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
|
141 |
-
- Clinical documents:
|
142 |
-
- `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
|
143 |
-
- mtsamples
|
144 |
|
145 |
See paper for details.
|
146 |
|
|
|
2 |
license: mit
|
3 |
datasets:
|
4 |
- ncbi/pubmed
|
|
|
|
|
|
|
|
|
5 |
language:
|
6 |
- en
|
7 |
base_model:
|
|
|
22 |
## Model Details
|
23 |
### Model Description
|
24 |
|
25 |
+
This model is `MediPhi-PubMed` obtained by merging the fine-tuned PubMed expert with the SLERP technique into the base model at 10%.
|
26 |
|
27 |
- **Developed by:** Microsoft Healthcare \& Life Sciences
|
28 |
- **Model type:** Phi3
|
|
|
86 |
|
87 |
torch.random.manual_seed(0)
|
88 |
|
89 |
+
model_name = "microsoft/MediPhi-PubMed"
|
90 |
model = AutoModelForCausalLM.from_pretrained(
|
91 |
model_name,
|
92 |
device_map="cuda",
|
|
|
131 |
|
132 |
Continual Pre-training:
|
133 |
- PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
|
|
|
|
|
|
|
|
|
|
|
|
|
134 |
|
135 |
See paper for details.
|
136 |
|