Phi-3 Technical Report: A Highly Capable Language Model Locally on Your
  Phone
We introduce phi-3-mini, a 3.8 billion parameter language model trained on
3.3 trillion tokens, whose overall performance, as measured by both academic
benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and
GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite
being small enough to be deployed on a phone. The innovation lies entirely in
our dataset for training, a scaled-up version of the one used for phi-2,
composed of heavily filtered web data and synthetic data. The model is also
further aligned for robustness, safety, and chat format. We also provide some
initial parameter-scaling results with a 7B and 14B models trained for 4.8T
tokens, called phi-3-small and phi-3-medium, both significantly more capable
than phi-3-mini (e.g., respectively 75% and 78% on MMLU, and 8.7 and 8.9 on
MT-bench).
66271d7a322746c7bfca83a8
Marah Abdin
claimed_verified
2024-04-23T13:00:35.357Z
649af1e3ca20306aeef590b8
https://cdn-avatars.huggingface.co/v1/production/uploads/649af1e3ca20306aeef590b8/Tvy8yf7zDO2FoNs03tWj6.jpeg
Marah Abdin
user
marah-abdin
66271d7a322746c7bfca83a9
Sam Ade Jacobs
admin_assigned
2024-04-23T07:17:35.386Z
651736dd20b18e99b44dd5d5
/avatars/c054eb2567e027408340a03a7ca0b29d.svg
Sam Ade Jacobs
user
samadejacobs
66271d7a322746c7bfca83aa
Ammar Ahmad Awan
admin_assigned
2024-04-23T07:17:44.962Z
6324b8417d6d0cbbe2c65019
/avatars/7c1a468daaa984203a600a3b3526298f.svg
Ammar Awan
user
ammarawan
66271d7a322746c7bfca83ab
Jyoti Aneja
admin_assigned
2024-04-23T07:17:51.473Z
623e17038972a8c030af23fa
https://cdn-avatars.huggingface.co/v1/production/uploads/623e17038972a8c030af23fa/2ERyhCNAdV2wt4uRy3Wq5.jpeg
Jyoti Aneja
user
jyotiA
66271d7a322746c7bfca83ac
Ahmed Awadallah
admin_assigned
2024-04-23T07:18:04.741Z
6557a41ee74d1065f7d05c51
/avatars/b29e57fc61d1d3012637f84b82c14818.svg
Ahmed Awadallah
user
AhmedAwadallah
66271d7a322746c7bfca83ad
Hany Awadalla
66271d7a322746c7bfca83ae
Nguyen Bach
admin_assigned
2024-04-23T07:18:30.800Z
5f3ec133a4dd343b63a632dd
https://cdn-avatars.huggingface.co/v1/production/uploads/1597948137099-noauth.jpeg
Nguyen Bach
user
nguyenbh
66271d7a322746c7bfca83af
Amit Bahree
admin_assigned
2024-04-23T07:18:40.847Z
604cfaaad8a4193fd3a928ef
https://cdn-avatars.huggingface.co/v1/production/uploads/1615657716895-604cfaaad8a4193fd3a928ef.jpeg
Amit Bahree
user
bahree
66271d7a322746c7bfca83b0
Arash Bakhtiari
claimed_verified
2024-04-24T11:11:57.013Z
6553cda0c81411e2aaf930b1
/avatars/291e4e9cef092d649ab734a3f09a3af1.svg
Arash Bakhtiari
user
arashb
66271d7a322746c7bfca83b1
Harkirat Behl
claimed_verified
2024-04-23T07:13:54.219Z
63c6aa9e656e7822e2359d9c
https://cdn-avatars.huggingface.co/v1/production/uploads/63c6aa9e656e7822e2359d9c/4rn_faJ23m_a6bevXEDG_.jpeg
Harkirat Behl
user
Harkirat
66271d7a322746c7bfca83b2
Alon Benhaim
admin_assigned
2024-04-23T07:19:30.493Z
65b9b627e7c838136275a681
/avatars/22423f3d9a6c4ee34cad3b0894d27d23.svg
Alon Benhaim
user
alonbenhaim
66271d7a322746c7bfca83b3
Misha Bilenko
66271d7a322746c7bfca83b4
Johan Bjorck
admin_assigned
2024-04-23T07:19:54.353Z
65fdfbbbc91ba4c08ae76db9
/avatars/6a44dcf01a531966806f7de9df94178b.svg
johan bjorck
user
njb-ms
66271d7a322746c7bfca83b5
Sébastien Bubeck
admin_assigned
2024-04-23T07:20:04.354Z
64557408c9c0dcc8c24a7a92
/avatars/911ba8fc3c224578d5946dddd94bf4c0.svg
Sebastien Bubeck
user
sebubeck
66271d7a322746c7bfca83b6
Martin Cai
admin_assigned
2024-04-23T07:20:23.447Z
63599df91a0dca1aa2497470
https://cdn-avatars.huggingface.co/v1/production/uploads/1666817522152-noauth.png
Martin Cai
user
cailang
66271d7a322746c7bfca83b7
Caio César Teodoro Mendes
admin_assigned
2024-04-23T07:20:59.498Z
66269c058f7573e6a6488532
/avatars/5d47ce0eaba09b385596ad5fd97f8ef7.svg
Caio Mendes
user
caiomms
66271d7a322746c7bfca83b8
Weizhu Chen
admin_assigned
2024-04-23T07:21:07.482Z
64da876370446182be5b608d
/avatars/e412fdc71404ecdf638e416846e3ebfb.svg
Weizhu Chen
user
chenweizhu
66271d7a322746c7bfca83b9
Vishrav Chaudhary
admin_assigned
2024-04-23T07:21:31.901Z
659c7ac977ac6f1bf5e63d7e
/avatars/86a6efde0d483564a67ed5f344d479a0.svg
Vishrav Chaudhary
user
vishravmsft
66271d7a322746c7bfca83ba
Parul Chopra
admin_assigned
2024-04-23T07:21:38.415Z
65fe033542e4b66cd59bbc33
/avatars/71a5d01b83f36d2926aef9aed624b953.svg
Parul Chopra
user
Parul09
66271d7a322746c7bfca83bb
Allie Del Giorno
admin_assigned
2024-04-23T07:21:53.089Z
63f5562471a5d395c721cd8e
/avatars/ac296b6017e86ea04c73803fe2c44433.svg
Allie Del Giorno
user
microallie
66271d7a322746c7bfca83bc
Gustavo de Rosa
admin_assigned
2024-04-23T07:22:00.600Z
6157454831624da88210e627
https://cdn-avatars.huggingface.co/v1/production/uploads/1666203761402-6157454831624da88210e627.jpeg
Gustavo de Rosa
user
gugarosa
66271d7a322746c7bfca83bd
Matthew Dixon
admin_assigned
2024-04-23T07:22:06.151Z
6316cbde29411a6864b9a15c
/avatars/649e713b17871a18f55ea8f167860ae3.svg
Matthew Dixon
user
mmdixon
66271d7a322746c7bfca83be
Ronen Eldan
admin_assigned
2024-04-23T07:22:13.165Z
633723a80267ebcf0264c06b
/avatars/22bb971597e9f3abfa343280a9d0f65f.svg
Ronen Eldan
user
roneneldan
66271d7a322746c7bfca83bf
Dan Iter
admin_assigned
2024-04-23T07:22:22.938Z
6604959a5186899ca9d62d27
/avatars/32416322dc3fd3b057d2cdd78750cdff.svg
Dan I
user
daniter-msft
66271d7a322746c7bfca83c0
Abhishek Goswami
admin_assigned
2024-04-23T07:22:39.305Z
62cdae333529c21a2283a0a1
/avatars/cafc2821e522bbd06d49830e36a073e3.svg
Abhishek GOSWAMI
user
abgoswam
66271d7a322746c7bfca83c1
Suriya Gunasekar
admin_assigned
2024-04-23T07:22:47.893Z
63e1b4f77fbb6ae4d4f36aa4
/avatars/7febc332d393e0bb94255104527ffe2a.svg
Gunasekar
user
suriyagunasekar
66271d7a322746c7bfca83c2
Emman Haider
66271d7a322746c7bfca83c3
Junheng Hao
admin_assigned
2024-04-23T07:22:59.089Z
5f04c4394ec31d33a72116d6
/avatars/75d4b9020070e73604b12e5adc1c8201.svg
Junheng Hao
user
jeffhao
66271d7a322746c7bfca83c4
Russell J. Hewett
66271d7a322746c7bfca83c5
Jamie Huynh
66271d7a322746c7bfca83c6
Mojan Javaheripi
admin_assigned
2024-04-23T07:23:17.966Z
643dbfa31e5be78c66440fcf
https://cdn-avatars.huggingface.co/v1/production/uploads/643dbfa31e5be78c66440fcf/VtLrAlPY_4ncVrM4eL7V-.jpeg
Mojan Javaheripi
user
mojanjp
66271d7a322746c7bfca83c7
Xin Jin
66271d7a322746c7bfca83c8
Piero Kauffmann
66271d7a322746c7bfca83c9
Nikos Karampatziakis
admin_assigned
2024-04-23T07:23:34.411Z
64d6785b089bc502cea059be
/avatars/79a504d2b66aa256327c8d11243d41b0.svg
Nikos Karampatziakis
user
n17s
66271d7a322746c7bfca83ca
Dongwoo Kim
admin_assigned
2024-04-23T07:23:48.587Z
662476aec8920ec351b8d3d8
/avatars/791e40f53073563680ef18f75b3ea95e.svg
Dongwoo Kim
user
dongwookim-ms
66271d7a322746c7bfca83cb
Mahoud Khademi
66271d7a322746c7bfca83cc
Lev Kurilenko
admin_assigned
2024-04-23T07:24:00.911Z
635852141d66b442317cbe26
/avatars/bcf9164f6ac6393ffecc5f4383d12cbe.svg
Lev Kurilenko
user
lekurile
66271d7a322746c7bfca83cd
James R. Lee
66271d7a322746c7bfca83ce
Yin Tat Lee
admin_assigned
2024-04-23T07:24:21.143Z
6303297604c75db08b8972e2
/avatars/693b2dada2244ba5d97df9518f473ccb.svg
Yin Tat Lee
user
yintat
66271d7a322746c7bfca83cf
Yuanzhi Li
admin_assigned
2024-04-23T07:24:45.250Z
641d5a313f04f9bf2da17e8c
/avatars/4035f73c69ddb045573ad96db4f13f04.svg
Yuanzhi Li
user
Uushizhu1234
66271d7a322746c7bfca83d0
Chen Liang
claimed_verified
2024-06-02T18:48:02.336Z
63e6b5e22d2c508de9001afd
/avatars/43cec9e8b8d490bd259e383954846a1e.svg
Chen Liang
user
cliang1453
66271d7a322746c7bfca83d1
Weishung Liu
admin_assigned
2024-04-23T07:24:59.941Z
637c3ac41c13d2970bf67625
https://cdn-avatars.huggingface.co/v1/production/uploads/637c3ac41c13d2970bf67625/K_KEPoWjj4k7EndHeU2Jt.png
Weishung Liu 
user
weish
66271d7a322746c7bfca83d2
Eric Lin
66271d7a322746c7bfca83d3
Zeqi Lin
66271d7a322746c7bfca83d4
Piyush Madan
admin_assigned
2024-04-23T07:25:40.367Z
66269a329014ef4d10f55d9d
/avatars/d4866c32419a7dd07e9aa0660f4bafa9.svg
Piyush Madan
user
PiyushMadan
66271d7a322746c7bfca83d5
Arindam Mitra
66271d7a322746c7bfca83d6
Hardik Modi
admin_assigned
2024-04-23T07:26:10.428Z
617c70112ee1adbd7739e7d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1635545103491-noauth.jpeg
Hardik Modi
user
hardikmodi
66271d7a322746c7bfca83d7
Anh Nguyen
admin_assigned
2024-04-23T07:26:19.012Z
649bc84833486cdd77c01c66
/avatars/36f4e4bb15c337c4391bfbd234051f4c.svg
Nguyen Anh
user
Anhnguyen
66271d7a322746c7bfca83d8
Brandon Norick
admin_assigned
2024-04-23T07:26:45.406Z
63200bb0b45367a05fef3049
/avatars/13e7a5c84d253c74112b3a6b058ae260.svg
Brandon Norick
user
bnorick
66271d7a322746c7bfca83d9
Barun Patra
66271d7a322746c7bfca83da
Daniel Perez-Becker
66271d7a322746c7bfca83db
Thomas Portet
admin_assigned
2024-04-23T07:34:57.983Z
65c52dad286bf45e79491697
/avatars/01ebc7979273df6e53971ae9835b503f.svg
Thomas Portet
user
thopo
66271d7a322746c7bfca83dc
Reid Pryzant
admin_assigned
2024-04-23T07:34:50.855Z
64e3d38c618cd90997ee0138
/avatars/847210a609b156fdb9d824cc815bde27.svg
Reid Pryzant
user
rpryzant
66271d7a322746c7bfca83dd
Heyang Qin
admin_assigned
2024-04-23T07:34:43.485Z
63bc6a611374e3ef9134a24e
/avatars/9df037b9cf7d7952a10d919e24a2ffa2.svg
Heyang Qin
user
heyangqin
66271d7a322746c7bfca83de
Marko Radmilac
66271d7a322746c7bfca83df
Corby Rosset
admin_assigned
2024-04-23T07:34:30.505Z
64ec43147e2ec711a761c594
/avatars/e4fc3de141ff026eee6eaa0b91d34ff1.svg
corby
user
corbyrosset
66271d7a322746c7bfca83e0
Sambudha Roy
66271d7a322746c7bfca83e1
Olli Saarikivi
admin_assigned
2024-04-23T07:34:13.500Z
6477a7108ab7e732b6d86701
/avatars/3319db398ce8142bd7d61de1d2cd1b4b.svg
Olli Saarikivi
user
olsaarik
66271d7a322746c7bfca83e2
Amin Saied
66271d7a322746c7bfca83e3
Adil Salim
66271d7a322746c7bfca83e4
Michael Santacroce
admin_assigned
2024-04-23T07:33:54.298Z
6626a4371ee17eac05a311ee
/avatars/0e0af517b715af3a7a4113855448702a.svg
Michael Santacroce
user
misantac
66271d7a322746c7bfca83e5
Shital Shah
admin_assigned
2024-04-23T07:33:36.790Z
5fea740dbb3114fa14c82934
/avatars/5f9fe9aaf74a3b930b6fcebcf8a4e4cc.svg
Shital Shah
user
sytelus
66271d7a322746c7bfca83e6
Ning Shang
66271d7a322746c7bfca83e7
Hiteshi Sharma
66271d7a322746c7bfca83e8
Xia Song
admin_assigned
2024-04-23T07:32:42.218Z
630540f999870e13d3ddb997
/avatars/7dc53475142c0c5c3a04d7476682269d.svg
Xia Song
user
ssaaxx
66271d7a322746c7bfca83e9
Olatunji Ruwase
admin_assigned
2024-04-23T07:32:31.128Z
6499d685ea2cdac80992e742
/avatars/c32741e7fc57c9a08722fab3877a7b81.svg
Olatunji Ruwase
user
tjruwase
66271d7a322746c7bfca83ea
Xin Wang
66271d7a322746c7bfca83eb
Rachel Ward
admin_assigned
2024-04-23T07:32:20.827Z
6626ab758a19c27042ed5e73
/avatars/757f1fda0ce88a344d85bde05e468d9c.svg
Rachel Ward
user
rward314
66271d7a322746c7bfca83ec
Guanhua Wang
66271d7a322746c7bfca83ed
Philipp Witte
admin_assigned
2024-04-23T07:30:48.497Z
64fcf02401aedd0e86e4b933
/avatars/602e8081d396101f537f773d873eb41b.svg
Philipp Witte
user
philippwitte
66271d7a322746c7bfca83ee
Michael Wyatt
admin_assigned
2024-04-23T07:30:36.125Z
62910c52d1be2630c0d38a72
/avatars/40a72a10e3ea17809a09bde77360c194.svg
Michael Wyatt
user
mwyatt
66271d7a322746c7bfca83ef
Can Xu
66271d7a322746c7bfca83f0
Jiahang Xu
admin_assigned
2024-04-23T07:30:06.292Z
62abdf657b037eafffc48808
https://cdn-avatars.huggingface.co/v1/production/uploads/1655430982462-noauth.jpeg
Jiahang Xu
user
Jiahang
66271d7a322746c7bfca83f1
Sonali Yadav
66271d7a322746c7bfca83f2
Fan Yang
66271d7a322746c7bfca83f3
Ziyi Yang
claimed_verified
2024-07-03T07:41:36.627Z
646be72031968a60a021ddb3
/avatars/79f3766cf404fd180274dbc86ea4dac6.svg
Ziyi Yang
user
Ziyi-Yang
66271d7a322746c7bfca83f4
Donghan Yu
admin_assigned
2024-04-23T07:28:53.970Z
65b01b8a29ae836e9ed5af24
/avatars/a8b78a4b54d3f10858c5925521357001.svg
Donghan Yu
user
donghanyu
66271d7a322746c7bfca83f5
Chengruidong Zhang
admin_assigned
2024-04-23T07:28:36.041Z
64646896884f2e3e1ced3cd5
https://cdn-avatars.huggingface.co/v1/production/uploads/64646896884f2e3e1ced3cd5/86-t8V8LGMNaPQRXnADiD.png
Zhang
user
Chengruidong
66271d7a322746c7bfca83f6
Cyril Zhang
admin_assigned
2024-04-23T07:28:23.642Z
63a6a3a93d8cc9b1183c6593
https://cdn-avatars.huggingface.co/v1/production/uploads/63a6a3a93d8cc9b1183c6593/UF-9ostqBHiAGDOuEtIw_.png
Cyril Zhang
user
cyrilzhang
66271d7a322746c7bfca83f7
Jianwen Zhang
admin_assigned
2024-04-23T07:28:15.724Z
63601ee38fb9c2420ffbe45d
/avatars/56af091aaff1b42dcfbae84a6ee1e7f7.svg
Jianwen Zhang
user
Jianwen
66271d7a322746c7bfca83f8
Li Lyna Zhang
admin_assigned
2024-04-23T07:28:03.688Z
62b0009c72043b05d29492b2
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0009c72043b05d29492b2/NqRkX2YLhlfOLvYysa7dD.png
Li Lyna Zhang
user
lynazhang
66271d7a322746c7bfca83f9
Yi Zhang
66271d7a322746c7bfca83fa
Yunan Zhang
claimed_verified
2024-04-24T11:11:55.084Z
6367f0be856a92b278915bb0
/avatars/6b36f53c74f0fa559aca5d4a8e3ed585.svg
yunan
user
yunan
66271d7a322746c7bfca83fb
Xiren Zhou
claimed_verified
2024-08-28T08:01:22.806Z
66ce4c9f864befb39cfc74e9
/avatars/ef66398466c470fc1d384c6817d9e461.svg
Xiren Zhou
user
XirenZhou
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your
  Phone
We introduce phi-3-mini, a 3.8 billion parameter language model trained on
3.3 trillion tokens, whose overall performance, as measured by both academic
benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and
GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite
being small enough to be deployed on a phone. The innovation lies entirely in
our dataset for training, a scaled-up version of the one used for phi-2,
composed of heavily filtered web data and synthetic data. The model is also
further aligned for robustness, safety, and chat format. We also provide some
initial parameter-scaling results with a 7B and 14B models trained for 4.8T
tokens, called phi-3-small and phi-3-medium, both significantly more capable
than phi-3-mini (e.g., respectively 75% and 78% on MMLU, and 8.7 and 8.9 on
MT-bench).
Phi-3-mini, a compact 3.8 billion parameter language model, achieves competitive performance with larger models through an enhanced training dataset and alignment.
language model MMLU MT-bench robustness safety chat format parameter-scaling