  Masked language models have recently been interpreted as energy-based
sequence models that can be generated from using a Metropolis--Hastings
sampler. This short paper demonstrates how this can be instrumentalized for
constrained composition and explores the poetics implied by such a usage. Our
focus on constraints makes it especially apt to understand the generated text
through the poetics of the OuLiPo movement.

Computation and Language