title: Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided
    Document Generation
summary: This paper introduces a novel approach for topic modeling utilizing latent
  codebooks from Vector-Quantized Variational Auto-Encoder~(VQ-VAE), discretely
  encapsulating the rich information of the pre-trained embeddings such as the
  pre-trained language model. From the novel interpretation of the latent
  codebooks and embeddings as conceptual bag-of-words, we propose a new
  generative topic model called Topic-VQ-VAE~(TVQ-VAE) which inversely generates
  the original documents related to the respective latent codebook. The TVQ-VAE
  can visualize the topics with various generative distributions including the
  traditional BoW distribution and the autoregressive image generation. Our
  experimental results on document analysis and image generation demonstrate that
  TVQ-VAE effectively captures the topic context which reveals the underlying
  structures of the dataset and supports flexible forms of document generation.
  Official implementation of the proposed TVQ-VAE is available at
  https://github.com/clovaai/TVQ-VAE.
paper_authors._id: 65825446cd17364539db4647
paper_authors.name: YoungJoon Yoo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T11:56:57.928Z
paper_authors._id: 6582d1b85def15befa412a88
paper_authors.avatarUrl: /avatars/5104073d752a9ed1a57678dd70753790.svg
paper_authors.fullname: YoungJoon Yoo
paper_authors.type: user
paper_authors.user: yjyoo3312
paper_authors._id: 65825446cd17364539db4648
paper_authors.name: Jongwon Choi
paper_title: Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided
    Document Generation
paper_summary: This paper introduces a novel approach for topic modeling utilizing latent
  codebooks from Vector-Quantized Variational Auto-Encoder~(VQ-VAE), discretely
  encapsulating the rich information of the pre-trained embeddings such as the
  pre-trained language model. From the novel interpretation of the latent
  codebooks and embeddings as conceptual bag-of-words, we propose a new
  generative topic model called Topic-VQ-VAE~(TVQ-VAE) which inversely generates
  the original documents related to the respective latent codebook. The TVQ-VAE
  can visualize the topics with various generative distributions including the
  traditional BoW distribution and the autoregressive image generation. Our
  experimental results on document analysis and image generation demonstrate that
  TVQ-VAE effectively captures the topic context which reveals the underlying
  structures of the dataset and supports flexible forms of document generation.
  Official implementation of the proposed TVQ-VAE is available at
  https://github.com/clovaai/TVQ-VAE.
paper_ai_keywords: Vector-Quantized Variational Auto-Encoder VQ-VAE latent codebooks pre-trained embeddings pre-trained language model conceptual bag-of-words Topic-VQ-VAE TVQ-VAE generative topic model autoregressive image generation BoW distribution