readme: ---
  language:
  - en
  tags:
  - Climate Change
  - GAN
  - Domain Adaptation
  license: gpl-3.0
  title: ClimateGAN
  emoji: 🌎
  colorFrom: blue
  colorTo: green
  sdk: gradio
  sdk_version: 3.6
  app_file: app.py
  inference: true
  pinned: true
  ---
  
  # ClimateGAN: Raising Awareness about Climate Change by Generating Images of Floods
  
  This repository contains the code used to train the model presented in our **[paper](https://openreview.net/forum?id=EZNOb_uNpJk)**.
  
  It is not simply a presentation repository but the code we have used over the past 30 months to come to our final architecture. As such, you will find many scripts, classes, blocks and options which we actively use for our own development purposes but are not directly relevant to reproduce results or use pretrained weights.
  
  ![flood processing](images/flood.png)
  
  If you use this code, data or pre-trained weights, please cite our ICLR 2022 paper:
  
  ```
  @inproceedings{schmidt2022climategan,
    title     = {Climate{GAN}: Raising Climate Change Awareness by Generating Images of Floods},
    author    = {Victor Schmidt and Alexandra Luccioni and M{\'e}lisande Teng and Tianyu Zhang and Alexia Reynaud and Sunand Raghupathi and Gautier Cosne and Adrien Juraver and Vahe Vardanyan and Alex Hern{\'a}ndez-Garc{\'\i}a and Yoshua Bengio},
    booktitle = {International Conference on Learning Representations},
    year      = {2022},
    url       = {https://openreview.net/forum?id=EZNOb_uNpJk}
  }
  ```
  
  ## Using pre-trained weights from this Huggingface Space and Stable Diffusion In-painting
  
  <p align="center">
    <strong>Huggingface ClimateGAN Space:</strong>
    <a href="https://huggingface.co/spaces/vict0rsch/climateGAN" target="_blank">
      <img src="https://huggingface.co/vict0rsch/climateGAN/resolve/main/images/hf-cg.png">
    </a>
  </p>
  
  1. Download code and model
    ```bash
    git lfs install
    git clone https://huggingface.co/vict0rsch/climateGAN
    git lfs pull # optional if you don't have the weights
    ```
  2. Install requirements
    ```
    pip install requirements.txt
    ```
  3. **Enable Stable Diffusion Inpainting** by visiting the model's card: https://huggingface.co/runwayml/stable-diffusion-inpainting **and** running `$ huggingface-cli login`
  4. Run `$ python climategan_wrapper.py help` for usage instructions on how to infer on a folder's images.
  5. Run `$ python app.py` to see the Gradio app.
     1. To use Google Street View you'll need an API key and set the `GMAPS_API_KEY` environment variable.
     2. To use Stable Diffusion if you can't run `$ huggingface-cli login` (on a Huggingface Space for instance) set the `HF_AUTH_TOKEN` env variable to a [Huggingface authorization token](https://huggingface.co/settings/tokens)
     3. To change the UI without model overhead, set the `CG_DEV_MODE` environment variable to `true`.
  
  For a more fine-grained control on ClimateGAN's inferences, refer to `apply_events.py` (does not support Stable Diffusion painter)
  
  **Note:** you don't have control on the prompt by design because I disabled the safety checker. Fork this space/repo and do it yourself if you really need to change the prompt. At least [open a discussion](https://huggingface.co/spaces/vict0rsch/climateGAN/discussions).
  
  ## Using pre-trained weights from source
  
  In the paper, we present ClimateGAN as a solution to produce images of floods. It can actually do **more**:
  
  * reusing the segmentation map, we are able to isolate the sky, turn it red and in a few more steps create an image resembling the consequences of a wildfire on a neighboring area, similarly to the [California wildfires](https://www.google.com/search?q=california+wildfires+red+sky&source=lnms&tbm=isch&sa=X&ved=2ahUKEwisws-hx7zxAhXxyYUKHQyKBUwQ_AUoAXoECAEQBA&biw=1680&bih=917&dpr=2).
  * reusing the depth map, we can simulate the consequences of a smog event on an image, scaling the intensity of the filter by the distance of an object to the camera, as per [HazeRD](http://www2.ece.rochester.edu/~gsharma/papers/Zhang_ICIP2017_HazeRD.pdf)
  
  ![image of wildfire processing](images/wildfire.png)
  ![image of smog processing](images/smog.png)
  
  In this section we'll explain how to produce the `Painted Input` along with the Smog and Wildfire outputs of a pre-trained ClimateGAN model.
  
  ### Installation
  
  This repository and associated model have been developed using Python 3.8.2 and **Pytorch 1.7.0**.
  
  ```bash
  $ git clone git@github.com:cc-ai/climategan.git
  $ cd climategan
  $ pip install -r requirements-3.8.2.txt # or `requirements-any.txt` for other Python versions (not tested but expected to be fine)
  ```
  
  Our pipeline uses [comet.ml](https://comet.ml) to log images. You don't *have* to use their services but we recommend you do as images can be uploaded on your workspace instead of being written to disk.
  
  If you want to use Comet, make sure you have the [appropriate configuration in place (API key and workspace at least)](https://www.comet.ml/docs/python-sdk/advanced/#non-interactive-setup)
  
  ### Inference
  
  1. Download and unzip the weights [from this link