aswerdlow/unidisc_non_interleaved

Unified Multimodal Discrete Diffusion

Alexander Swerdlow^1* Mihir Prabhudesai^1* Siddharth Gandhi¹ Deepak Pathak¹ Katerina Fragkiadaki¹

¹ Carnegie Mellon University

Hugging Face models

The UniDisc checkpoints are available on Hugging Face:

Getting Started

Code can be found here: https://github.com/alexanderswerdlow/unidisc/tree/main

To install the dependencies, run:

git submodule update --init --recursive
uv sync --no-group dev
uv sync

For a more detailed installation guide, please refer to INSTALL.md.

Data

See DATA.md for details on how to download and preprocess the datasets. We provide processing scripts and instructions for all of the used datasets. Additionally, we release a synthetic dataset available here and the corresponding generation scripts as well as the raw data.

Training

See TRAIN.md for training commands.

Inference

Interactive demo:

python demo/server.py
python demo/client_simple_fasthtml.py

Training

See TRAINING.md for details.

Evaluation

See EVAL.md for details.

Citation

To cite our work, please use the following:

@article{TODO,
  title={TODO},
  author={TODO},
  journal={arXiv preprint arXiv:TODO},
  year={TODO}
}

Credits

This repository is built on top of the following repositories: