When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training Paper β’ 2411.13476 β’ Published Nov 20, 2024 β’ 16
Stronger Models are NOT Stronger Teachers for Instruction Tuning Paper β’ 2411.07133 β’ Published Nov 11, 2024 β’ 39
Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free Paper β’ 2410.10814 β’ Published Oct 14, 2024 β’ 52
Gemma-APS Release Collection Gemma models for text-to-propositions segmentation. The models are distilled from fine-tuned Gemini Pro model applied to multi-domain synthetic data. β’ 3 items β’ Updated 26 days ago β’ 22
view article Article Fine-tuning LLMs to 1.58bit: extreme quantization made easy By medmekk and 5 others β’ Sep 18, 2024 β’ 253
view article Article dstack: Your LLM Launchpad - From Fine-Tuning to Serving, Simplified By chansung β’ Aug 22, 2024 β’ 13
view article Article Training and Finetuning Embedding Models with Sentence Transformers v3 By tomaarsen β’ May 28, 2024 β’ 228
Improving Text Embeddings with Large Language Models Paper β’ 2401.00368 β’ Published Dec 31, 2023 β’ 81
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools Paper β’ 2406.12793 β’ Published Jun 18, 2024 β’ 33
Aligning to Thousands of Preferences via System Message Generalization Paper β’ 2405.17977 β’ Published May 28, 2024 β’ 7
Korean Reward Modeling Collection Korean Datasets, Reward Models for RLHF β’ 16 items β’ Updated Nov 19, 2024 β’ 3
SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification Paper β’ 2305.09781 β’ Published May 16, 2023 β’ 4
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Paper β’ 2309.10400 β’ Published Sep 19, 2023 β’ 26
Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models Paper β’ 2402.14714 β’ Published Feb 22, 2024 β’ 4