---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: depth-estimation
tags:
  - applied-intuition
  - autonomous-driving
  - depth-estimation
  - semantic-segmentation
  - motion-prediction
  - self-supervised
  - cvpr-2026
  - arxiv:2602.22091
paper: 2602.22091
---

<p align="center">
  <img src="https://huggingface.co/AppliedIntuitionResearch/LFG/resolve/main/assets/icon.svg" width="120"/>
</p>

# LFG: Learning to Drive is a Free Gift — Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos

<p align="center">
  <a href="https://arxiv.org/abs/2602.22091"><img src="https://img.shields.io/badge/arXiv-2602.22091-b31b1b.svg" alt="arXiv"></a>
  <a href="https://lfg-ai.github.io/"><img src="https://img.shields.io/badge/Project-Website-blue" alt="Project Page"></a>
  <a href="https://github.com/Applied-Intuition-Open-Source/LFG"><img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="Code"></a>
  <a href="https://creativecommons.org/licenses/by-nc/4.0/"><img src="https://img.shields.io/badge/License-CC%20BY--NC%204.0-lightgrey" alt="License"></a>
</p>

LFG turns ordinary driving video into a unified pseudo-4D representation — 3D point maps, camera
poses, semantic layouts, confidence and motion masks — learned from unposed, unlabelled dashcam
footage with no human annotation. Given three observed frames it predicts all of these for those
frames *and* for three frames into the future.

> ⚠️ **Research release.** This model is released for research purposes under the terms below. It is not a production system and does not include any Applied Intuition proprietary data, product code, or production checkpoints.

## Model Details

| | |
|---|---|
| **Developed by** | Applied Intuition — AI Research |
| **Model type** | Autoregressive transformer over a DINOv2 encoder, with future-frame prediction |
| **Paper** | [Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos](https://arxiv.org/abs/2602.22091) |
| **Conference / venue** | CVPR 2026 |
| **Code** | https://github.com/Applied-Intuition-Open-Source/LFG |
| **Project page** | https://lfg-ai.github.io/ |
| **License (weights)** | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
| **Contact** | GitHub Issues on the code repo |

## Checkpoints

| Checkpoint | Description | Size | Link |
|---|---|---|---|
| `lfg_seg_motion_m3n3.pt` | 1.22B params. 3 observed frames in, 3 observed + 3 future out. Heads: depth/points, camera pose, confidence, segmentation (7 classes), motion. | 4.87 GB | [lfg_seg_motion_m3n3.pt](lfg_seg_motion_m3n3.pt) |

The file holds `model_state_dict` (inference weights), `config` (architecture settings) and
`global_step`, and loads with `torch.load(..., weights_only=True)`.

## Intended Use & Limitations

**Intended use:** research on label-free pretraining for driving perception, reproducing the
paper's KITTI-360 and Waymo benchmarks, and using the representation as a frozen backbone for
downstream tasks.

**Out of scope / limitations:**

- Point maps are predicted up to one unknown scale and shift. Align them against a metric
  reference before reporting metric error.
- Trained on forward-facing dashcam video; other camera placements, viewpoints and sensor rigs
  are untested.
- Segmentation covers seven coarse classes, not a full urban taxonomy.

**Commercial use is not permitted** under CC BY-NC 4.0.

## How to Use

```python
import numpy as np
from PIL import Image
from huggingface_hub import hf_hub_download

from lfg.checkpoint import load_model_from_checkpoint
from lfg.inference import predict_window
from lfg.io import Frame

path = hf_hub_download("AppliedIntuitionResearch/LFG", "lfg_seg_motion_m3n3.pt")
model, config, _, _ = load_model_from_checkpoint(path, device="cuda")

# three consecutive RGB frames from a forward-facing camera
frames = [Frame(rgb=np.asarray(Image.open(p).convert("RGB")), source=p, frame_index=i)
          for i, p in enumerate(["000.png", "001.png", "002.png"])]

out = predict_window(model, frames, config, device="cuda",
                     target_size=518, resize_mode="crop", keep_ratio=False)

print(out["local_points"].shape)   # (1, 6, H, W, 3) — 3 observed + 3 predicted
print(out["segmentation"].shape)   # (1, 6, H, W, 7)
```

Full installation and usage instructions: see the [GitHub repository](https://github.com/Applied-Intuition-Open-Source/LFG).

## Citation

```bibtex
@inproceedings{strong2026lfg,
  title     = {Learning to Drive is a Free Gift: Large-Scale Label-Free Autonomy Pretraining from Unposed In-The-Wild Videos},
  author    = {Strong, Matthew and Chang, Wei-Jer and Herau, Quentin and Yang, Jiezhi and Hu, Yihan and Peng, Chensheng and Zhan, Wei},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}
```

## Acknowledgments

This model builds on [Pi3](https://github.com/yyfz/Pi3) (BSD-3-Clause), whose model code is
bundled in the inference repo, and which in turn builds on
[DINOv2](https://github.com/facebookresearch/dinov2) (Meta Platforms, Apache-2.0). Evaluation
baselines use [VGGT](https://github.com/facebookresearch/vggt),
[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3),
[SegFormer](https://huggingface.co/nvidia/segformer-b5-finetuned-cityscapes-1024-1024) and
[MaskFormer](https://huggingface.co/facebook/maskformer-resnet101-cityscapes), and the benchmarks
use [KITTI-360](https://www.cvlibs.net/datasets/kitti-360/) and the
[Waymo Open Dataset](https://waymo.com/open/). We thank the authors for open-sourcing their work.