jpcorb20 commited on
Commit
a9aeef0
·
verified ·
1 Parent(s): a802445

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -14
README.md CHANGED
@@ -1,11 +1,5 @@
1
  ---
2
  license: mit
3
- datasets:
4
- - ncbi/pubmed
5
- - starmpcc/Asclepius-Synthetic-Clinical-Notes
6
- - akemiH/NoteChat
7
- - zhengyun21/PMC-Patients
8
- - jpcorb20/medical_wikipedia
9
  language:
10
  - en
11
  base_model:
@@ -26,7 +20,7 @@ The MediPhi Model Collection comprises 7 small language models of 3.8B parameter
26
  ## Model Details
27
  ### Model Description
28
 
29
- This model is `MediPhi` obtained by merging all 5 experts with the BreadCrumbs technique into this unified expert.
30
 
31
  - **Developed by:** Microsoft Healthcare \& Life Sciences
32
  - **Model type:** Phi3
@@ -90,7 +84,7 @@ Researchers should apply responsible AI best practices, including mapping, measu
90
 
91
  torch.random.manual_seed(0)
92
 
93
- model_name = "microsoft/MediPhi"
94
  model = AutoModelForCausalLM.from_pretrained(
95
  model_name,
96
  device_map="cuda",
@@ -134,13 +128,7 @@ Check `microsoft/Phi-3.5-mini-instruct` for details about the tokenizer, require
134
  ### Training Data
135
 
136
  Continual Pre-training:
137
- - PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
138
- - Medical Guideline `epfl-llm/guidelines`.
139
- - Medical Wikipedia `jpcorb20/medical_wikipedia`.
140
  - Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
141
- - Clinical documents:
142
- - `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
143
- - mtsamples
144
 
145
  See paper for details.
146
 
 
1
  ---
2
  license: mit
 
 
 
 
 
 
3
  language:
4
  - en
5
  base_model:
 
20
  ## Model Details
21
  ### Model Description
22
 
23
+ This model is `MediPhi-MedCode` obtained by merging the fine-tuned MedCode expert with the SLERP technique into its base model at 50%.
24
 
25
  - **Developed by:** Microsoft Healthcare \& Life Sciences
26
  - **Model type:** Phi3
 
84
 
85
  torch.random.manual_seed(0)
86
 
87
+ model_name = "microsoft/MediPhi-MedCode"
88
  model = AutoModelForCausalLM.from_pretrained(
89
  model_name,
90
  device_map="cuda",
 
128
  ### Training Data
129
 
130
  Continual Pre-training:
 
 
 
131
  - Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
 
 
 
132
 
133
  See paper for details.
134