README.md

---
license: apache-2.0
---
![image/png](images/banner-github-simpler.png)

# [CVPRW 2025] 🌍 COP-GEN-Beta: Unified Generative Modelling of COPernicus Imagery Thumbnails

<div align="center" style="line-height: 1;">
  <a href="https://huggingface.co/spaces/mikonvergence/COP-GEN-Beta" style="margin: 2px;">
    <img src="https://img.shields.io/badge/%F0%9F%A4%97-Demo-yellow" alt="HF Demo" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://github.com/miquel-espinosa/COP-GEN-Beta" style="margin: 2px;">
    <img src="https://img.shields.io/badge/%E2%80%8B-COP--GEN--Beta-black?logo=github" alt="GitHub" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://miquel-espinosa.github.io/cop-gen-beta/" style="margin: 2px;">
    <img src="https://img.shields.io/badge/🌐-Website-grey" alt="Website" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://huggingface.co/mespinosami/COP-GEN-Beta" style="margin: 2px;">
    <img src="https://img.shields.io/badge/%F0%9F%A4%97-Model-yellow" alt="HF Model" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://www.arxiv.org/abs/2504.08548" style="margin: 2px;">
    <img src="https://img.shields.io/badge/arXiv-2402.12095-D12424" alt="arXiv" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://colab.research.google.com/github/miquel-espinosa/COP-GEN-Beta/blob/main/COP_GEN_Beta_inference.ipynb" target="_parent" style="margin: 2px;">
    <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" style="display: inline-block; vertical-align: middle;"/>
  </a>
</div>

## Abstract
> _In remote sensing, multi-modal data from various sensors capturing the same scene_
_offers rich opportunities, but learning a unified representation across these modalities remains a significant challenge._
_Traditional methods have often been limited to single or dual-modality approaches._
_In this paper, we introduce COP-GEN-Beta, a generative diffusion model trained on optical, radar, and elevation data from the Major TOM dataset._
_What sets COP-GEN-Beta apart is its ability to map any subset of modalities to any other, enabling zero-shot modality translation after training._
_This is achieved through a sequence-based diffusion transformer, where each modality is controlled by its own timestep embedding._
_We extensively evaluate COP-GEN-Beta on thumbnail images from the Major TOM dataset, demonstrating its effectiveness in generating high-quality samples._
_Qualitative and quantitative evaluations validate the model's performance, highlighting its potential as a powerful pre-trained model for future remote sensing tasks._

## COP-GEN-Beta: Architecture Overview

COP-GEN-Beta is a diffusion model designed to handle multiple remote sensing modalities, specifically: Digital Elevation Model (DEM), Sentinel-1 Radar Terrain Corrected (S1 RTC), Sentinel-2 Level 1C (S2 L1C), and Sentinel-2 Level 2A (S2 L2A). The model learns joint, conditional, and marginal distributions within a unified framework.

![COP-GEN-Beta Architecture](images/cop-gen-beta-architecture.png)

## COP-GEN-Beta: Results

COP-GEN-Beta's flexible sampling capabilities enable a wide range of downstream applications through various modality translation combinations. By allowing generation of any subset of modalities conditioned on any other subset, our model unlocks numerous practical use cases in remote sensing, from atmospheric correction and DEM generation to dataset expansion.

![COP-GEN-Beta Results](images/use-case-horizontal.png)

## Getting Started

For detailed instructions on installation, training, and inference, please visit our [GitHub repository](https://github.com/miquel-espinosa/COP-GEN-Beta).

COP-GEN-Beta offers great versatility for generating images, including:
- **Unconditional generation:** Generates tuples of 4 modalities without any condition.
- **Conditional generation:**
  - Single modality conditioning
  - 2 modality conditioning
  - 3 modality conditioning

## Citation

If you find this work useful, please cite it as follows:

```bibtex
@inproceedings{espinosa2025copgenbeta,
  title={COP-GEN-Beta: Unified Generative Modelling of COPernicus Imagery Thumbnails},
  author={Espinosa, Miguel and Marsocci, Valerio and Jia, Yuru and Crowley, Elliot J. and Czerkawski, Mikolaj},
  booktitle={CVPRW},
  year={2025}
}
```