Prithvi WxC: Foundation Model for Weather and Climate
Triggered by the realization that AI emulators can rival the performance of
traditional numerical weather prediction models running on HPC systems, there
is now an increasing number of large AI models that address use cases such as
forecasting, downscaling, or nowcasting. While the parallel developments in the
AI literature focus on foundation models -- models that can be effectively
tuned to address multiple, different use cases -- the developments on the
weather and climate side largely focus on single-use cases with particular
emphasis on mid-range forecasting. We close this gap by introducing Prithvi
WxC, a 2.3 billion parameter foundation model developed using 160 variables
from the Modern-Era Retrospective Analysis for Research and Applications,
Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture,
incorporating concepts from various recent transformer models to effectively
capture both regional and global dependencies in the input data. The model has
been designed to accommodate large token counts to model weather phenomena in
different topologies at fine resolutions. Furthermore, it is trained with a
mixed objective that combines the paradigms of masked reconstruction with
forecasting. We test the model on a set of challenging downstream tasks namely:
Autoregressive rollout forecasting, Downscaling, Gravity wave flux
parameterization, and Extreme events estimation. The pretrained model with 2.3
billion parameters, along with the associated fine-tuning workflows, has been
publicly released as an open-source contribution via Hugging Face.
66f0c7e20c00b14702737322
Johannes Schmude
admin_assigned
2024-09-23T07:38:33.054Z
66e714ffe1e454e8b8cb929c
/avatars/1f7acae68e43cfd50a00fa39ba2aa344.svg
Johannes Schmude
user
johannesschmude
66f0c7e20c00b14702737323
Sujit Roy
claimed_verified
2024-09-23T16:29:38.744Z
6488f1d3e22a0081a561ec8f
https://cdn-avatars.huggingface.co/v1/production/uploads/6488f1d3e22a0081a561ec8f/RFk5nPfSOvHsJj1DWOJ0R.png
Sujit Roy
user
thesujitroy
66f0c7e20c00b14702737324
Will Trojak
admin_assigned
2024-09-23T07:39:01.731Z
66abb2ad80170b585a774248
/avatars/d3ec6e41e3467b8bbba600505d098c9e.svg
Will Trojak
user
WillTrojak
66f0c7e20c00b14702737325
Johannes Jakubik
admin_assigned
2024-09-23T07:39:07.496Z
64b84a418b53fb5dbde793fa
https://cdn-avatars.huggingface.co/v1/production/uploads/64b84a418b53fb5dbde793fa/Xb35aMd-EUAG1IDlwD_aF.jpeg
Johannes Jakubik
user
jhnnsjkbk
66f0c7e20c00b14702737326
Daniel Salles Civitarese
admin_assigned
2024-09-23T07:39:12.922Z
630395877b50dd9d0a33a6c2
https://cdn-avatars.huggingface.co/v1/production/uploads/630395877b50dd9d0a33a6c2/PvsZUOjlJ0MbdAvk0uaCw.jpeg
Daniel Salles Civitarese
user
ds6574
66f0c7e20c00b14702737327
Shraddha Singh
66f0c7e20c00b14702737328
Julian Kuehnert
admin_assigned
2024-09-23T07:39:51.079Z
64c8fd55b0e61287787739a1
https://cdn-avatars.huggingface.co/v1/production/uploads/64c8fd55b0e61287787739a1/aJ73octpRoU-2TcjNZl8L.jpeg
Julian Kuehnert
user
jubeku
66f0c7e20c00b14702737329
Kumar Ankur
claimed_verified
2024-09-23T13:48:37.940Z
64c92ddc0b2ba05b2f66aee1
https://cdn-avatars.huggingface.co/v1/production/uploads/64c92ddc0b2ba05b2f66aee1/7-7NFmk5DJPYOmqt4sOPL.jpeg
Ankur Kumar
user
ankurk017
66f0c7e20c00b1470273732a
Aman Gupta
66f0c7e20c00b1470273732b
Christopher E Phillips
admin_assigned
2024-09-23T07:40:59.955Z
64429c9dc82791fbe15ffa8b
/avatars/cdaddb91652ac020e32a6af3e4e01a22.svg
Christopher Phillips
user
CEPhillips
66f0c7e20c00b1470273732c
Romeo Kienzler
admin_assigned
2024-09-23T07:41:07.135Z
66559f32ccdee62544fe08da
https://cdn-avatars.huggingface.co/v1/production/uploads/66559f32ccdee62544fe08da/Gc8P9qDJFxgJ1J0gM1nQp.jpeg
Romeo Kienzler
user
romeokienzler
66f0c7e20c00b1470273732d
Daniela Szwarcman
admin_assigned
2024-09-23T07:41:12.851Z
64c949eae761f4706136148d
/avatars/4f9ffff84838f34c4cac534c368a483d.svg
Daniela Szwarcman
user
daniszw
66f0c7e20c00b1470273732e
Vishal Gaur
admin_assigned
2024-09-23T07:41:37.252Z
65119331b6bdfa5be9b9fa67
/avatars/3b37828c433e03c518e9563bacf83ba8.svg
Vishal Gaur
user
swordsaintlancelot
66f0c7e20c00b1470273732f
Rajat Shinde
admin_assigned
2024-09-23T07:41:46.374Z
651acaa0551c9a100b084a2e
/avatars/6a5f264cc7c9fe34bf9064ceed1d090d.svg
Rajat Shinde
user
omshinde
66f0c7e20c00b14702737330
Rohit Lal
claimed_verified
2024-09-24T07:53:20.213Z
63f9043ee45925b1dc0f2142
https://cdn-avatars.huggingface.co/v1/production/uploads/63f9043ee45925b1dc0f2142/ewzdBeTLjg6PCaPD4WJuu.jpeg
Rohit Lal
user
rlal011
66f0c7e20c00b14702737331
Arlindo Da Silva
66f0c7e20c00b14702737332
Jorge Luis Guevara Diaz
66f0c7e20c00b14702737333
Anne Jones
admin_assigned
2024-09-23T07:42:16.857Z
66e19458a533df3d82955b0c
/avatars/3352538065eff47963081f48570afe1f.svg
Anne Jones
user
annezj
66f0c7e20c00b14702737334
Simon Pfreundschuh
admin_assigned
2024-09-23T07:42:22.817Z
652f01139a99dd6bf69af838
/avatars/3d0c514da97ad50e52d39d2824f175e2.svg
Simon Pfreundschuh
user
simonpf
66f0c7e20c00b14702737335
Amy Lin
66f0c7e20c00b14702737336
Aditi Sheshadri
66f0c7e20c00b14702737337
Udaysankar Nair
66f0c7e20c00b14702737338
Valentine Anantharaj
admin_assigned
2024-09-23T07:42:59.237Z
62794873bba7c62cbab2456e
/avatars/ab3b503a340ad47a8c7224c4add464e2.svg
Valentine Anantharaj
user
anantharajvg
66f0c7e20c00b14702737339
Hendrik Hamann
admin_assigned
2024-09-23T07:43:06.095Z
6661a9965ebb442b5040ae27
/avatars/66830cc5e56516b403e3e9191ba80129.svg
Hendrik Hamann
user
HHamann
66f0c7e20c00b1470273733a
Campbell Watson
admin_assigned
2024-09-23T07:43:13.152Z
64766671d6c79c9d0e16ab3e
https://cdn-avatars.huggingface.co/v1/production/uploads/64766671d6c79c9d0e16ab3e/nTgk6_e3ctuhwiG4E7-Go.jpeg
Campbell Watson
user
camelstation
66f0c7e20c00b1470273733b
Manil Maskey
admin_assigned
2024-09-23T07:43:19.668Z
64ba94321e53e10faf0c14ca
https://cdn-avatars.huggingface.co/v1/production/uploads/64ba94321e53e10faf0c14ca/leAHIQoy8nk74Mdut3MJ7.jpeg
Manil Maskey
user
manilmaskey
66f0c7e20c00b1470273733c
Tsengdar J Lee
66f0c7e20c00b1470273733d
Juan Bernabe Moreno
66f0c7e20c00b1470273733e
Rahul Ramachandran
Prithvi WxC: Foundation Model for Weather and Climate
Triggered by the realization that AI emulators can rival the performance of
traditional numerical weather prediction models running on HPC systems, there
is now an increasing number of large AI models that address use cases such as
forecasting, downscaling, or nowcasting. While the parallel developments in the
AI literature focus on foundation models -- models that can be effectively
tuned to address multiple, different use cases -- the developments on the
weather and climate side largely focus on single-use cases with particular
emphasis on mid-range forecasting. We close this gap by introducing Prithvi
WxC, a 2.3 billion parameter foundation model developed using 160 variables
from the Modern-Era Retrospective Analysis for Research and Applications,
Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture,
incorporating concepts from various recent transformer models to effectively
capture both regional and global dependencies in the input data. The model has
been designed to accommodate large token counts to model weather phenomena in
different topologies at fine resolutions. Furthermore, it is trained with a
mixed objective that combines the paradigms of masked reconstruction with
forecasting. We test the model on a set of challenging downstream tasks namely:
Autoregressive rollout forecasting, Downscaling, Gravity wave flux
parameterization, and Extreme events estimation. The pretrained model with 2.3
billion parameters, along with the associated fine-tuning workflows, has been
publicly released as an open-source contribution via Hugging Face.
Prithvi WxC, a 2.3 billion parameter foundation model using an encoder-decoder architecture with transformer concepts, addresses weather forecasting, downscaling, and extreme events estimation with a mixed masked reconstruction and forecasting objective.
encoder-decoder-based architecture transformer models regional dependencies global dependencies token counts masked reconstruction forecasting autoregressive rollout forecasting downscaling gravity wave flux parameterization extreme events estimation