Understanding and Modeling Job Marketplace with Pretrained Language Models
<|reference_start|>Understanding and Modeling Job Marketplace with Pretrained Language Models: Job marketplace is a heterogeneous graph composed of interactions among members (job-seekers), companies, and jobs. Understanding and modeling job marketplace can benefit both job seekers and employers, ultimately contributing to the greater good of the society. However, existing graph neural network (GNN)-based methods have shallow understandings of the associated textual features and heterogeneous relations. To address the above challenges, we propose PLM4Job, a job marketplace foundation model that tightly couples pretrained language models (PLM) with job market graph, aiming to fully utilize the pretrained knowledge and reasoning ability to model member/job textual features as well as various member-job relations simultaneously. In the pretraining phase, we propose a heterogeneous ego-graph-based prompting strategy to model and aggregate member/job textual features based on the topological structure around the target member/job node, where entity type embeddings and graph positional embeddings are introduced accordingly to model different entities and their heterogeneous relations. Meanwhile, a proximity-aware attention alignment strategy is designed to dynamically adjust the attention of the PLM on ego-graph node tokens in the prompt, such that the attention can be better aligned with job marketplace semantics. Extensive experiments at LinkedIn demonstrate the effectiveness of PLM4Job.<|reference_end|>
@article{zhu2024understanding,
    title={Understanding and Modeling Job Marketplace with Pretrained Language
  Models},
    author={Yaochen Zhu, Liang Wu, Binchi Zhang, Song Wang, Qi Guo, Liangjie Hong,
  Luke Simon, Jundong Li},
    journal={arXiv preprint arXiv:2408.04381},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.04381},
    primaryClass={cs.IR}
}