fPLSA: Learning Semantic Structures in Document Collections Using Foundation Models
<|reference_start|>fPLSA: Learning Semantic Structures in Document Collections Using Foundation Models: Humans have the ability to learn new tasks by inferring high-level concepts from existing solution, then manipulating these concepts in lieu of the raw data. Can we automate this process by deriving latent semantic structures in a document collection using foundation models? We introduce fPLSA, a foundation-model-based Probabilistic Latent Semantic Analysis (PLSA) method that iteratively clusters and tags document segments based on document-level contexts. These tags can be used to model the structure of given documents and for hierarchical sampling of new texts. Our experiments on story writing, math, and multi-step reasoning datasets demonstrate that fPLSA tags help reconstruct the original texts better than existing tagging methods. Moreover, when used for hierarchical sampling, fPLSA produces more diverse outputs with a higher likelihood of hitting the correct answer than direct sampling and hierarchical sampling with existing tagging methods.<|reference_end|>
@article{xu2024fplsa:,
    title={fPLSA: Learning Semantic Structures in Document Collections Using
  Foundation Models},
    author={Weijia Xu, Nebojsa Jojic, Nicolas Le Roux},
    journal={arXiv preprint arXiv:2410.05481},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05481},
    primaryClass={cs.LG}
}