  In this work, we present TGLS, a novel framework to unsupervised Text
Generation by Learning from Search. We start by applying a strong search
algorithm (in particular, simulated annealing) towards a heuristically defined
objective that (roughly) estimates the quality of sentences. Then, a
conditional generative model learns from the search results, and meanwhile
smooth out the noise of search. The alternation between search and learning can
be repeated for performance bootstrapping. We demonstrate the effectiveness of
TGLS on two real-world natural language generation tasks, paraphrase generation
and text formalization. Our model significantly outperforms unsupervised
baseline methods in both tasks. Especially, it achieves comparable performance
with the state-of-the-art supervised methods in paraphrase generation.

Computation and Language;Artificial Intelligence;Information Retrieval;Machine Learning;Machine Learning