title: LoRACode: LoRA Adapters for Code Embeddings
summary: Code embeddings are essential for semantic code search; however, current
  approaches often struggle to capture the precise syntactic and contextual
  nuances inherent in code. Open-source models such as CodeBERT and UniXcoder
  exhibit limitations in scalability and efficiency, while high-performing
  proprietary systems impose substantial computational costs. We introduce a
  parameter-efficient fine-tuning method based on Low-Rank Adaptation (LoRA) to
  construct task-specific adapters for code retrieval. Our approach reduces the
  number of trainable parameters to less than two percent of the base model,
  enabling rapid fine-tuning on extensive code corpora (2 million samples in 25
  minutes on two H100 GPUs). Experiments demonstrate an increase of up to 9.1% in
  Mean Reciprocal Rank (MRR) for Code2Code search, and up to 86.69% for Text2Code
  search tasks across multiple programming languages. Distinction in task-wise
  and language-wise adaptation helps explore the sensitivity of code retrieval
  for syntactical and linguistic variations.
paper_authors._id: 67ce6db07110b8bedb3344a7
paper_authors.name: Saumya Chaturvedi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T10:23:01.422Z
paper_authors._id: 6432b9a62bfb2b0ec756b374
paper_authors.avatarUrl: /avatars/5db525f2cd40fd077e751b250bbb30a5.svg
paper_authors.fullname: Saumya Chaturvedi
paper_authors.type: user
paper_authors.user: shollercoaster
paper_authors._id: 67ce6db07110b8bedb3344a8
paper_authors.name: Aman Chadha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T08:00:45.150Z
paper_authors._id: 63a4754927f1f64ed7238dac
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a4754927f1f64ed7238dac/aH-eJF-31g4vof9jv2gmI.jpeg
paper_authors.fullname: Aman Chadha
paper_authors.type: user
paper_authors.user: amanchadha
paper_authors._id: 67ce6db07110b8bedb3344a9
paper_authors.name: Laurent Bindschaedler
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T10:23:07.597Z
paper_authors._id: 6783105a17f0d939ebcdd86c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UKBXp_AGPVQjH9e_C-C5c.png
paper_authors.fullname: Laurent Bindschaedler
paper_authors.type: user
paper_authors.user: bindsch
paper_title: LoRACode: LoRA Adapters for Code Embeddings
paper_summary: Code embeddings are essential for semantic code search; however, current
  approaches often struggle to capture the precise syntactic and contextual
  nuances inherent in code. Open-source models such as CodeBERT and UniXcoder
  exhibit limitations in scalability and efficiency, while high-performing
  proprietary systems impose substantial computational costs. We introduce a
  parameter-efficient fine-tuning method based on Low-Rank Adaptation (LoRA) to
  construct task-specific adapters for code retrieval. Our approach reduces the
  number of trainable parameters to less than two percent of the base model,
  enabling rapid fine-tuning on extensive code corpora (2 million samples in 25
  minutes on two H100 GPUs). Experiments demonstrate an increase of up to 9.1% in
  Mean Reciprocal Rank (MRR) for Code2Code search, and up to 86.69% for Text2Code
  search tasks across multiple programming languages. Distinction in task-wise
  and language-wise adaptation helps explore the sensitivity of code retrieval
  for syntactical and linguistic variations.
paper_ai_keywords: Code embeddings semantic code search syntactic nuances contextual nuances CodeBERT UniXcoder parameter-efficient fine-tuning Low-Rank Adaptation (LoRA) task-specific adapters code retrieval Mean Reciprocal Rank (MRR) Code2Code search Text2Code search