Heeding the Inner Voice: Aligning ControlNet Training via Intermediate
  Features Feedback
Despite significant progress in text-to-image diffusion models, achieving
precise spatial control over generated outputs remains challenging. ControlNet
addresses this by introducing an auxiliary conditioning module, while
ControlNet++ further refines alignment through a cycle consistency loss applied
only to the final denoising steps. However, this approach neglects intermediate
generation stages, limiting its effectiveness. We propose InnerControl, a
training strategy that enforces spatial consistency across all diffusion steps.
Our method trains lightweight convolutional probes to reconstruct input control
signals (e.g., edges, depth) from intermediate UNet features at every denoising
step. These probes efficiently extract signals even from highly noisy latents,
enabling pseudo ground truth controls for training. By minimizing the
discrepancy between predicted and target conditions throughout the entire
diffusion process, our alignment loss improves both control fidelity and
generation quality. Combined with established techniques like ControlNet++,
InnerControl achieves state-of-the-art performance across diverse conditioning
methods (e.g., edges, depth).
68679bf1213f123a1f88b8bd
Nina Konovalova
68679bf1213f123a1f88b8be
Maxim Nikolaev
68679bf1213f123a1f88b8bf
Andrey Kuznetsov
claimed_verified
2025-07-11T13:13:07.396Z
643984dceb7c5616ef3f5d54
https://cdn-avatars.huggingface.co/v1/production/uploads/643984dceb7c5616ef3f5d54/10JRkblrRIEVci6UJwvPz.jpeg
Andrey Kuznetsov
user
kuznetsoffandrey
68679bf1213f123a1f88b8c0
Aibek Alanov
claimed_verified
2025-07-11T08:08:52.677Z
66680c6451545a8b46c6fd21
/avatars/03707f5ea4e2aa8dc825a9782b00ed85.svg
Aibek Alanov
user
ai-alanov
Heeding the Inner Voice: Aligning ControlNet Training via Intermediate
  Features Feedback
Despite significant progress in text-to-image diffusion models, achieving
precise spatial control over generated outputs remains challenging. ControlNet
addresses this by introducing an auxiliary conditioning module, while
ControlNet++ further refines alignment through a cycle consistency loss applied
only to the final denoising steps. However, this approach neglects intermediate
generation stages, limiting its effectiveness. We propose InnerControl, a
training strategy that enforces spatial consistency across all diffusion steps.
Our method trains lightweight convolutional probes to reconstruct input control
signals (e.g., edges, depth) from intermediate UNet features at every denoising
step. These probes efficiently extract signals even from highly noisy latents,
enabling pseudo ground truth controls for training. By minimizing the
discrepancy between predicted and target conditions throughout the entire
diffusion process, our alignment loss improves both control fidelity and
generation quality. Combined with established techniques like ControlNet++,
InnerControl achieves state-of-the-art performance across diverse conditioning
methods (e.g., edges, depth).
InnerControl enhances text-to-image diffusion models by enforcing spatial consistency across all diffusion steps using lightweight convolutional probes, improving control fidelity and generation quality.
ControlNet ControlNet++ diffusion models spatial control auxiliary conditioning module cycle consistency loss denoising steps UNet convolutional probes pseudo ground truth controls alignment loss