jpcorb20 commited on
Commit
e077816
·
verified ·
1 Parent(s): 5d0de8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -12
README.md CHANGED
@@ -2,10 +2,6 @@
2
  license: mit
3
  datasets:
4
  - ncbi/pubmed
5
- - starmpcc/Asclepius-Synthetic-Clinical-Notes
6
- - akemiH/NoteChat
7
- - zhengyun21/PMC-Patients
8
- - jpcorb20/medical_wikipedia
9
  language:
10
  - en
11
  base_model:
@@ -26,7 +22,7 @@ The MediPhi Model Collection comprises 7 small language models of 3.8B parameter
26
  ## Model Details
27
  ### Model Description
28
 
29
- This model is `MediPhi` obtained by merging all 5 experts with the BreadCrumbs technique into this unified expert.
30
 
31
  - **Developed by:** Microsoft Healthcare \& Life Sciences
32
  - **Model type:** Phi3
@@ -90,7 +86,7 @@ Researchers should apply responsible AI best practices, including mapping, measu
90
 
91
  torch.random.manual_seed(0)
92
 
93
- model_name = "microsoft/MediPhi"
94
  model = AutoModelForCausalLM.from_pretrained(
95
  model_name,
96
  device_map="cuda",
@@ -135,12 +131,6 @@ Check `microsoft/Phi-3.5-mini-instruct` for details about the tokenizer, require
135
 
136
  Continual Pre-training:
137
  - PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
138
- - Medical Guideline `epfl-llm/guidelines`.
139
- - Medical Wikipedia `jpcorb20/medical_wikipedia`.
140
- - Medical Coding: ICD10CM, ICD10PROC, ICD9CM, ICD9PROC, and ATC.
141
- - Clinical documents:
142
- - `zhengyun21/PMC-Patients`, `akemiH/NoteChat`, and `starmpcc/Asclepius-Synthetic-Clinical-Notes` (only commercial-friendly licenses across all three datasets)
143
- - mtsamples
144
 
145
  See paper for details.
146
 
 
2
  license: mit
3
  datasets:
4
  - ncbi/pubmed
 
 
 
 
5
  language:
6
  - en
7
  base_model:
 
22
  ## Model Details
23
  ### Model Description
24
 
25
+ This model is `MediPhi-PubMed` obtained by merging the fine-tuned PubMed expert with the SLERP technique into the base model at 10%.
26
 
27
  - **Developed by:** Microsoft Healthcare \& Life Sciences
28
  - **Model type:** Phi3
 
86
 
87
  torch.random.manual_seed(0)
88
 
89
+ model_name = "microsoft/MediPhi-PubMed"
90
  model = AutoModelForCausalLM.from_pretrained(
91
  model_name,
92
  device_map="cuda",
 
131
 
132
  Continual Pre-training:
133
  - PubMed (commercial subset) and abstracts from `ncbi/pubmed`.
 
 
 
 
 
 
134
 
135
  See paper for details.
136