title: Prithvi WxC: Foundation Model for Weather and Climate
summary: Triggered by the realization that AI emulators can rival the performance of
  traditional numerical weather prediction models running on HPC systems, there
  is now an increasing number of large AI models that address use cases such as
  forecasting, downscaling, or nowcasting. While the parallel developments in the
  AI literature focus on foundation models -- models that can be effectively
  tuned to address multiple, different use cases -- the developments on the
  weather and climate side largely focus on single-use cases with particular
  emphasis on mid-range forecasting. We close this gap by introducing Prithvi
  WxC, a 2.3 billion parameter foundation model developed using 160 variables
  from the Modern-Era Retrospective Analysis for Research and Applications,
  Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture,
  incorporating concepts from various recent transformer models to effectively
  capture both regional and global dependencies in the input data. The model has
  been designed to accommodate large token counts to model weather phenomena in
  different topologies at fine resolutions. Furthermore, it is trained with a
  mixed objective that combines the paradigms of masked reconstruction with
  forecasting. We test the model on a set of challenging downstream tasks namely:
  Autoregressive rollout forecasting, Downscaling, Gravity wave flux
  parameterization, and Extreme events estimation. The pretrained model with 2.3
  billion parameters, along with the associated fine-tuning workflows, has been
  publicly released as an open-source contribution via Hugging Face.
paper_authors._id: 66f0c7e20c00b14702737322
paper_authors.name: Johannes Schmude
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:38:33.054Z
paper_authors._id: 66e714ffe1e454e8b8cb929c
paper_authors.avatarUrl: /avatars/1f7acae68e43cfd50a00fa39ba2aa344.svg
paper_authors.fullname: Johannes Schmude
paper_authors.type: user
paper_authors.user: johannesschmude
paper_authors._id: 66f0c7e20c00b14702737323
paper_authors.name: Sujit Roy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T16:29:38.744Z
paper_authors._id: 6488f1d3e22a0081a561ec8f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6488f1d3e22a0081a561ec8f/RFk5nPfSOvHsJj1DWOJ0R.png
paper_authors.fullname: Sujit Roy
paper_authors.type: user
paper_authors.user: thesujitroy
paper_authors._id: 66f0c7e20c00b14702737324
paper_authors.name: Will Trojak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:39:01.731Z
paper_authors._id: 66abb2ad80170b585a774248
paper_authors.avatarUrl: /avatars/d3ec6e41e3467b8bbba600505d098c9e.svg
paper_authors.fullname: Will Trojak
paper_authors.type: user
paper_authors.user: WillTrojak
paper_authors._id: 66f0c7e20c00b14702737325
paper_authors.name: Johannes Jakubik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:39:07.496Z
paper_authors._id: 64b84a418b53fb5dbde793fa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b84a418b53fb5dbde793fa/Xb35aMd-EUAG1IDlwD_aF.jpeg
paper_authors.fullname: Johannes Jakubik
paper_authors.type: user
paper_authors.user: jhnnsjkbk
paper_authors._id: 66f0c7e20c00b14702737326
paper_authors.name: Daniel Salles Civitarese
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:39:12.922Z
paper_authors._id: 630395877b50dd9d0a33a6c2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630395877b50dd9d0a33a6c2/PvsZUOjlJ0MbdAvk0uaCw.jpeg
paper_authors.fullname: Daniel Salles Civitarese
paper_authors.type: user
paper_authors.user: ds6574
paper_authors._id: 66f0c7e20c00b14702737327
paper_authors.name: Shraddha Singh
paper_authors._id: 66f0c7e20c00b14702737328
paper_authors.name: Julian Kuehnert
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:39:51.079Z
paper_authors._id: 64c8fd55b0e61287787739a1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c8fd55b0e61287787739a1/aJ73octpRoU-2TcjNZl8L.jpeg
paper_authors.fullname: Julian Kuehnert
paper_authors.type: user
paper_authors.user: jubeku
paper_authors._id: 66f0c7e20c00b14702737329
paper_authors.name: Kumar Ankur
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T13:48:37.940Z
paper_authors._id: 64c92ddc0b2ba05b2f66aee1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64c92ddc0b2ba05b2f66aee1/7-7NFmk5DJPYOmqt4sOPL.jpeg
paper_authors.fullname: Ankur Kumar
paper_authors.type: user
paper_authors.user: ankurk017
paper_authors._id: 66f0c7e20c00b1470273732a
paper_authors.name: Aman Gupta
paper_authors._id: 66f0c7e20c00b1470273732b
paper_authors.name: Christopher E Phillips
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:40:59.955Z
paper_authors._id: 64429c9dc82791fbe15ffa8b
paper_authors.avatarUrl: /avatars/cdaddb91652ac020e32a6af3e4e01a22.svg
paper_authors.fullname: Christopher Phillips
paper_authors.type: user
paper_authors.user: CEPhillips
paper_authors._id: 66f0c7e20c00b1470273732c
paper_authors.name: Romeo Kienzler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:41:07.135Z
paper_authors._id: 66559f32ccdee62544fe08da
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66559f32ccdee62544fe08da/Gc8P9qDJFxgJ1J0gM1nQp.jpeg
paper_authors.fullname: Romeo Kienzler
paper_authors.type: user
paper_authors.user: romeokienzler
paper_authors._id: 66f0c7e20c00b1470273732d
paper_authors.name: Daniela Szwarcman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:41:12.851Z
paper_authors._id: 64c949eae761f4706136148d
paper_authors.avatarUrl: /avatars/4f9ffff84838f34c4cac534c368a483d.svg
paper_authors.fullname: Daniela Szwarcman
paper_authors.type: user
paper_authors.user: daniszw
paper_authors._id: 66f0c7e20c00b1470273732e
paper_authors.name: Vishal Gaur
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:41:37.252Z
paper_authors._id: 65119331b6bdfa5be9b9fa67
paper_authors.avatarUrl: /avatars/3b37828c433e03c518e9563bacf83ba8.svg
paper_authors.fullname: Vishal Gaur
paper_authors.type: user
paper_authors.user: swordsaintlancelot
paper_authors._id: 66f0c7e20c00b1470273732f
paper_authors.name: Rajat Shinde
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:41:46.374Z
paper_authors._id: 651acaa0551c9a100b084a2e
paper_authors.avatarUrl: /avatars/6a5f264cc7c9fe34bf9064ceed1d090d.svg
paper_authors.fullname: Rajat Shinde
paper_authors.type: user
paper_authors.user: omshinde
paper_authors._id: 66f0c7e20c00b14702737330
paper_authors.name: Rohit Lal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-24T07:53:20.213Z
paper_authors._id: 63f9043ee45925b1dc0f2142
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f9043ee45925b1dc0f2142/ewzdBeTLjg6PCaPD4WJuu.jpeg
paper_authors.fullname: Rohit Lal
paper_authors.type: user
paper_authors.user: rlal011
paper_authors._id: 66f0c7e20c00b14702737331
paper_authors.name: Arlindo Da Silva
paper_authors._id: 66f0c7e20c00b14702737332
paper_authors.name: Jorge Luis Guevara Diaz
paper_authors._id: 66f0c7e20c00b14702737333
paper_authors.name: Anne Jones
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:42:16.857Z
paper_authors._id: 66e19458a533df3d82955b0c
paper_authors.avatarUrl: /avatars/3352538065eff47963081f48570afe1f.svg
paper_authors.fullname: Anne Jones
paper_authors.type: user
paper_authors.user: annezj
paper_authors._id: 66f0c7e20c00b14702737334
paper_authors.name: Simon Pfreundschuh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:42:22.817Z
paper_authors._id: 652f01139a99dd6bf69af838
paper_authors.avatarUrl: /avatars/3d0c514da97ad50e52d39d2824f175e2.svg
paper_authors.fullname: Simon Pfreundschuh
paper_authors.type: user
paper_authors.user: simonpf
paper_authors._id: 66f0c7e20c00b14702737335
paper_authors.name: Amy Lin
paper_authors._id: 66f0c7e20c00b14702737336
paper_authors.name: Aditi Sheshadri
paper_authors._id: 66f0c7e20c00b14702737337
paper_authors.name: Udaysankar Nair
paper_authors._id: 66f0c7e20c00b14702737338
paper_authors.name: Valentine Anantharaj
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:42:59.237Z
paper_authors._id: 62794873bba7c62cbab2456e
paper_authors.avatarUrl: /avatars/ab3b503a340ad47a8c7224c4add464e2.svg
paper_authors.fullname: Valentine Anantharaj
paper_authors.type: user
paper_authors.user: anantharajvg
paper_authors._id: 66f0c7e20c00b14702737339
paper_authors.name: Hendrik Hamann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:43:06.095Z
paper_authors._id: 6661a9965ebb442b5040ae27
paper_authors.avatarUrl: /avatars/66830cc5e56516b403e3e9191ba80129.svg
paper_authors.fullname: Hendrik Hamann
paper_authors.type: user
paper_authors.user: HHamann
paper_authors._id: 66f0c7e20c00b1470273733a
paper_authors.name: Campbell Watson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:43:13.152Z
paper_authors._id: 64766671d6c79c9d0e16ab3e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64766671d6c79c9d0e16ab3e/nTgk6_e3ctuhwiG4E7-Go.jpeg
paper_authors.fullname: Campbell Watson
paper_authors.type: user
paper_authors.user: camelstation
paper_authors._id: 66f0c7e20c00b1470273733b
paper_authors.name: Manil Maskey
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-23T07:43:19.668Z
paper_authors._id: 64ba94321e53e10faf0c14ca
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ba94321e53e10faf0c14ca/leAHIQoy8nk74Mdut3MJ7.jpeg
paper_authors.fullname: Manil Maskey
paper_authors.type: user
paper_authors.user: manilmaskey
paper_authors._id: 66f0c7e20c00b1470273733c
paper_authors.name: Tsengdar J Lee
paper_authors._id: 66f0c7e20c00b1470273733d
paper_authors.name: Juan Bernabe Moreno
paper_authors._id: 66f0c7e20c00b1470273733e
paper_authors.name: Rahul Ramachandran
paper_title: Prithvi WxC: Foundation Model for Weather and Climate
paper_summary: Triggered by the realization that AI emulators can rival the performance of
  traditional numerical weather prediction models running on HPC systems, there
  is now an increasing number of large AI models that address use cases such as
  forecasting, downscaling, or nowcasting. While the parallel developments in the
  AI literature focus on foundation models -- models that can be effectively
  tuned to address multiple, different use cases -- the developments on the
  weather and climate side largely focus on single-use cases with particular
  emphasis on mid-range forecasting. We close this gap by introducing Prithvi
  WxC, a 2.3 billion parameter foundation model developed using 160 variables
  from the Modern-Era Retrospective Analysis for Research and Applications,
  Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture,
  incorporating concepts from various recent transformer models to effectively
  capture both regional and global dependencies in the input data. The model has
  been designed to accommodate large token counts to model weather phenomena in
  different topologies at fine resolutions. Furthermore, it is trained with a
  mixed objective that combines the paradigms of masked reconstruction with
  forecasting. We test the model on a set of challenging downstream tasks namely:
  Autoregressive rollout forecasting, Downscaling, Gravity wave flux
  parameterization, and Extreme events estimation. The pretrained model with 2.3
  billion parameters, along with the associated fine-tuning workflows, has been
  publicly released as an open-source contribution via Hugging Face.
paper_ai_keywords: foundation models encoder-decoder-based architecture transformer models regional dependencies global dependencies token counts masked reconstruction forecasting autoregressive rollout forecasting downscaling gravity wave flux parameterization extreme events estimation pretrained model open-source contribution Hugging Face