Microscaling Data Formats for Deep Learning
Narrow bit-width data formats are key to reducing the computational and
storage costs of modern deep learning applications. This paper evaluates
Microscaling (MX) data formats that combine a per-block scaling factor with
narrow floating-point and integer types for individual elements.MX formats
balance the competing needs of hardware efficiency, model accuracy, and user
friction. Empirical results on over two dozen benchmarks demonstrate
practicality of MX data formats as a drop-in replacement for baseline FP32 for
AI inference and training with low user friction. We also show the first
instance of training generative language models at sub-8-bit weights,
activations, and gradients with minimal accuracy loss and no modifications to
the training recipe.
652df53842d54348c294cc6f
Bita Darvish Rouhani
652df53842d54348c294cc70
Ritchie Zhao
admin_assigned
2023-10-17T08:30:39.468Z
65038b7a1801ba3867c4b924
/avatars/b9499df4974df46fb403a639b801c9b8.svg
Ritchie Zhao
user
rizhao
652df53842d54348c294cc71
Ankit More
652df53842d54348c294cc72
Mathew Hall
652df53842d54348c294cc73
Alireza Khodamoradi
admin_assigned
2023-10-17T08:31:35.965Z
62841d4adcfdf66c168610ca
/avatars/42e0144acb64969eac94e123c4f5fdbe.svg
Alireza Khodamoradi
user
arkhodamoradi
652df53842d54348c294cc74
Summer Deng
admin_assigned
2023-10-17T08:31:42.766Z
6463d76038083255f6756434
/avatars/5dfcee4812963dbff953b962a4d785fc.svg
Summer Deng
user
SummerDeng
652df53842d54348c294cc75
Dhruv Choudhary
652df53842d54348c294cc76
Marius Cornea
652df53842d54348c294cc77
Eric Dellinger
admin_assigned
2023-10-17T08:32:05.486Z
64bfd086623ce87c701c4851
/avatars/a2baa56c0e6b19ef48de39d2c2216925.svg
Eric Dellinger
user
ericdellinger
652df53842d54348c294cc78
Kristof Denolf
652df53842d54348c294cc79
Stosic Dusan
652df53842d54348c294cc7a
Venmugil Elango
652df53842d54348c294cc7b
Maximilian Golub
claimed_verified
2024-02-22T22:28:34.941Z
64dfcf93f6c2311e7ed48a74
/avatars/61326c72f64c1195084a2613035d9b94.svg
Golub
user
mtg01
652df53842d54348c294cc7c
Alexander Heinecke
652df53842d54348c294cc7d
Phil James-Roxby
652df53842d54348c294cc7e
Dharmesh Jani
652df53842d54348c294cc7f
Gaurav Kolhe
652df53842d54348c294cc80
Martin Langhammer
652df53842d54348c294cc81
Ada Li
652df53842d54348c294cc82
Levi Melnick
652df53842d54348c294cc83
Maral Mesmakhosroshahi
admin_assigned
2023-10-17T08:34:44.294Z
64dfd286ead049c7b88822ba
/avatars/f4dff23cb59de052ad48e8c2326c9b7c.svg
Maral Mesmakhosroshahi
user
mamesmak
652df53842d54348c294cc84
Andres Rodriguez
652df53842d54348c294cc85
Michael Schulte
652df53842d54348c294cc86
Rasoul Shafipour
652df53842d54348c294cc87
Lei Shao
652df53842d54348c294cc88
Michael Siu
652df53842d54348c294cc89
Pradeep Dubey
652df53842d54348c294cc8a
Paulius Micikevicius
652df53842d54348c294cc8b
Maxim Naumov
652df53842d54348c294cc8c
Colin Verilli
652df53842d54348c294cc8d
Ralph Wittig
652df53842d54348c294cc8e
Eric Chung
Microscaling Data Formats for Deep Learning
Narrow bit-width data formats are key to reducing the computational and
storage costs of modern deep learning applications. This paper evaluates
Microscaling (MX) data formats that combine a per-block scaling factor with
narrow floating-point and integer types for individual elements.MX formats
balance the competing needs of hardware efficiency, model accuracy, and user
friction. Empirical results on over two dozen benchmarks demonstrate
practicality of MX data formats as a drop-in replacement for baseline FP32 for
AI inference and training with low user friction. We also show the first
instance of training generative language models at sub-8-bit weights,
activations, and gradients with minimal accuracy loss and no modifications to
the training recipe.
Microscaling data formats reduce computational and storage costs while maintaining model accuracy and enabling sub-8-bit training for generative language models.
narrow bit-width data formats Microscaling (MX) data formats per-block scaling factor narrow floating-point narrow integer types hardware efficiency user friction AI inference training generative language models