content.raw: We (
content.type: text
content.value: We (
content.raw: @osma
content.type: mention
content.user: osma
content.raw: , 
content.type: text
content.value: , 
content.raw: @MonaLehtinen
content.type: mention
content.user: MonaLehtinen
content.raw:  & me, i.e. the Annif team at the National Library of Finland) recently took part in the LLMs4Subjects challenge at the SemEval-2025 workshop. The task was to use large language models (LLMs) to generate good quality subject indexing for bibliographic records, i.e. titles and abstracts.
content.type: text
content.value:  & me, i.e. the Annif team at the National Library of Finland) recently took part in the LLMs4Subjects challenge at the SemEval-2025 workshop. The task was to use large language models (LLMs) to generate good quality subject indexing for bibliographic records, i.e. titles and abstracts.
content.type: new_line
content.type: new_line
content.raw: We are glad to report that our system performed well; it was ranked
content.type: text
content.value: We are glad to report that our system performed well; it was ranked
content.type: new_line
content.type: new_line
content.raw: 🥇 1st in the category where the full vocabulary was used
content.type: text
content.value: 🥇 1st in the category where the full vocabulary was used
content.type: new_line
content.raw: 🥈 2nd in the smaller vocabulary category
content.type: text
content.value: 🥈 2nd in the smaller vocabulary category
content.type: new_line
content.raw: 🏅 4th in the qualitative evaluations.
content.type: text
content.value: 🏅 4th in the qualitative evaluations.
content.type: new_line
content.type: new_line
content.raw: 14 participating teams developed their own solutions for generating subject headings and the output of each system was assessed using both quantitative and qualitative evaluations. Research papers about most of the systems are going to be published around the time of the workshop in late July, and many pre-prints are already available.
content.type: text
content.value: 14 participating teams developed their own solutions for generating subject headings and the output of each system was assessed using both quantitative and qualitative evaluations. Research papers about most of the systems are going to be published around the time of the workshop in late July, and many pre-prints are already available.
content.type: new_line
content.type: new_line
content.raw: We applied Annif together with several LLMs that we used to preprocess the data sets: translated the GND vocabulary terms to English, translated bibliographic records into English and German as required, and generated additional synthetic training data. After the preprocessing, we used the traditional machine learning algorithms in Annif as well as the experimental XTransformer algorithm that is based on language models. We also combined the subject suggestions generated using English and German language records in a novel way.
content.type: text
content.value: We applied Annif together with several LLMs that we used to preprocess the data sets: translated the GND vocabulary terms to English, translated bibliographic records into English and German as required, and generated additional synthetic training data. After the preprocessing, we used the traditional machine learning algorithms in Annif as well as the experimental XTransformer algorithm that is based on language models. We also combined the subject suggestions generated using English and German language records in a novel way.
content.type: new_line
content.type: new_line
content.raw: More information can be found in our system description preprint: 
content.type: text
content.value: More information can be found in our system description preprint: 
content.label: Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs (2504.19675)
content.raw: https://huggingface.co/papers/2504.19675
content.id: 2504.19675
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2504.19675
content.type: new_line
content.type: new_line
content.raw: See also the task description preprint: 
content.type: text
content.value: See also the task description preprint: 
content.label: SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject
    Tagging for a National Technical Library's Open-Access Catalog (2504.07199)
content.raw: https://huggingface.co/papers/2504.07199
content.id: 2504.07199
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2504.07199
content.type: new_line
content.type: new_line
content.raw: The Annif models trained for this task are available here: 
content.type: text
content.value: The Annif models trained for this task are available here: 
content.raw: https://huggingface.co/NatLibFi/Annif-LLMs4Subjects-data
content.id: NatLibFi/Annif-LLMs4Subjects-data
content.type: model
content.type: resource
content.url: https://huggingface.co/NatLibFi/Annif-LLMs4Subjects-data
rawContent: We (@osma, @MonaLehtinen & me, i.e. the Annif team at the National Library of Finland) recently took part in the LLMs4Subjects challenge at the SemEval-2025 workshop. The task was to use large language models (LLMs) to generate good quality subject indexing for bibliographic records, i.e. titles and abstracts.
  
  We are glad to report that our system performed well; it was ranked
  
  🥇 1st in the category where the full vocabulary was used
  🥈 2nd in the smaller vocabulary category
  🏅 4th in the qualitative evaluations.
  
  14 participating teams developed their own solutions for generating subject headings and the output of each system was assessed using both quantitative and qualitative evaluations. Research papers about most of the systems are going to be published around the time of the workshop in late July, and many pre-prints are already available.
  
  We applied Annif together with several LLMs that we used to preprocess the data sets: translated the GND vocabulary terms to English, translated bibliographic records into English and German as required, and generated additional synthetic training data. After the preprocessing, we used the traditional machine learning algorithms in Annif as well as the experimental XTransformer algorithm that is based on language models. We also combined the subject suggestions generated using English and German language records in a novel way.
  
  More information can be found in our system description preprint: https://huggingface.co/papers/2504.19675
  
  See also the task description preprint: https://huggingface.co/papers/2504.07199
  
  The Annif models trained for this task are available here: https://huggingface.co/NatLibFi/Annif-LLMs4Subjects-data
reactions.reaction: 🚀
reactions.users: andreaschandra John6666 xkkm
reactions.reaction: 🤝
reactions.users: eaddario
commentators._id: 5e762d233d77a72421292d02
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1657201383691-5e762d233d77a72421292d02.jpeg
commentators.fullname: Andreas Chandra
commentators.name: andreaschandra
commentators.type: user
commentators._id: 6818cdc2214df3575105f1a7
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6818cdc2214df3575105f1a7/qz1LS3XAFgnicwgjeCJYG.jpeg
commentators.fullname: CJ Hauser
commentators.name: CJHauser
commentators.type: user