159 42 61

Nathan Habib

SaylorTwift

AI & ML interests

None yet

Recent Activity

upvoted an article 27 days ago

Reachy Mini - The Open-Source Robot for Today's and Tomorrow's AI Builders

liked a model 27 days ago

HuggingFaceTB/SmolLM3-3B

reacted to nicolay-r's post with 🔥 27 days ago

🚀 For those who interested in summarization of the long textual reports in medical domain 📝🩺, @Xiaolihai and I delighted to share that we experiment with distillation tuning adaptation for Qwen-2.5 0.5B. We use reports from the MultiClinSum dataset and pass it through 72B version to retrieve report explanations in order to initiate ditillation tuning for 0.5B model. We experiment with passages written in English, French, Portuguese, and Spanish. 🔑 We find that using distil-technique results in 2-4% performance increment on fine-tuning and similar improvements for reports in English (non-official and official evaluation). For the other it results in systems that perform similar to the convential tuning (standard) (see result below). Dataset: https://zenodo.org/records/15459174 Competition: https://participants-area.bioasq.org/general_information/MultiClinSum/ Github: https://github.com/nicolay-r/distil-tuning-llm model: https://huggingface.co/nicolay-r/qwen25-05b-multiclinsum-distil

View all activity

Organizations

upvoted an article 27 days ago

Article

Reachy Mini - The Open-Source Robot for Today's and Tomorrow's AI Builders

and 1 other •

27 days ago

• 626

liked a model 27 days ago

HuggingFaceTB/SmolLM3-3B

Text Generation • 3B • Updated 7 days ago • 768k • • 631

reacted to nicolay-r's post with 🔥 27 days ago

Post

3546

🚀 For those who interested in summarization of the long textual reports in medical domain 📝🩺, @Xiaolihai and I delighted to share that we experiment with distillation tuning adaptation for Qwen-2.5 0.5B. We use reports from the MultiClinSum dataset and pass it through 72B version to retrieve report explanations in order to initiate ditillation tuning for 0.5B model. We experiment with passages written in English, French, Portuguese, and Spanish.

🔑 We find that using distil-technique results in 2-4% performance increment on fine-tuning and similar improvements for reports in English (non-official and official evaluation). For the other it results in systems that perform similar to the convential tuning (standard) (see result below).

Dataset: https://zenodo.org/records/15459174
Competition: https://participants-area.bioasq.org/general_information/MultiClinSum/
Github: https://github.com/nicolay-r/distil-tuning-llm
model: nicolay-r/qwen25-05b-multiclinsum-distil