title: CaptainGAN: Navigate Through Embedding Space For Better Text Generation
abstract: Score-function-based text generation approaches such as REINFORCE, in general, suffer from high computational complexity and training instability problems. This is mainly due to the non-differentiable nature of the discrete space sampling and thus these methods have to treat the discriminator as a reward function and ignore the gradient information. In this paper, we propose a novel approach, CaptainGAN, which adopts the straight-through gradient estimator and introduces a ”re-centered” gradient estimation technique to steer the generator toward better text tokens through the embedding space. Our method is stable to train and converges quickly without maximum likelihood pre-training. On multiple metrics of text quality and diversity, our method outperforms existing GAN-based methods on natural language generation.
authors: Chun-Hsing Lin Alvin Chiang Chi-Liang Liu Chien-Fu Lin Po-Hsien Chu Siang-Ruei Wu Yi-En Tsai Chung-Yang (Ric) Huang
hIndex: Chi-Liang Liu 5