Preserving Privacy, Increasing Accessibility, and Reducing Cost: An
  On-Device Artificial Intelligence Model for Medical Transcription and Note
  Generation
Background: Clinical documentation represents a significant burden for
healthcare providers, with physicians spending up to 2 hours daily on
administrative tasks. Recent advances in large language models (LLMs) offer
promising solutions, but privacy concerns and computational requirements limit
their adoption in healthcare settings. Objective: To develop and evaluate a
privacy-preserving, on-device medical transcription system using a fine-tuned
Llama 3.2 1B model capable of generating structured medical notes from medical
transcriptions while maintaining complete data sovereignty entirely in the
browser. Methods: We fine-tuned a Llama 3.2 1B model using Parameter-Efficient
Fine-Tuning (PEFT) with LoRA on 1,500 synthetic medical
transcription-to-structured note pairs. The model was evaluated against the
base Llama 3.2 1B on two datasets: 100 endocrinology transcripts and 140
modified ACI benchmark cases. Evaluation employed both statistical metrics
(ROUGE, BERTScore, BLEURT) and LLM-as-judge assessments across multiple
clinical quality dimensions. Results: The fine-tuned OnDevice model
demonstrated substantial improvements over the base model. On the ACI
benchmark, ROUGE-1 scores increased from 0.346 to 0.496, while BERTScore F1
improved from 0.832 to 0.866. Clinical quality assessments showed marked
reduction in major hallucinations (from 85 to 35 cases) and enhanced factual
correctness (2.81 to 3.54 on 5-point scale). Similar improvements were observed
on the internal evaluation dataset, with composite scores increasing from 3.13
to 4.43 (+41.5%). Conclusions: Fine-tuning compact LLMs for medical
transcription yields clinically meaningful improvements while enabling complete
on-device browser deployment. This approach addresses key barriers to AI
adoption in healthcare: privacy preservation, cost reduction, and accessibility
for resource-constrained environments.
686c91a7364e2ad167eb53e9
Johnson Thomas
claimed_verified
2025-07-08T08:06:24.155Z
64f953dd83807928d272c944
https://cdn-avatars.huggingface.co/v1/production/uploads/64f953dd83807928d272c944/xmEXTCQe6PhkruyMPwxEi.jpeg
Johnson Thomas
user
Johnyquest7
686c91a7364e2ad167eb53ea
Ayush Mudgal
686c91a7364e2ad167eb53eb
Wendao Liu
686c91a7364e2ad167eb53ec
Nisten Tahiraj
686c91a7364e2ad167eb53ed
Zeeshaan Mohammed
claimed_verified
2025-07-29T07:10:49.989Z
670ed2ce391f1a70213db609
/avatars/692b661a88cc3455dca2442df7961861.svg
Zeeshaan Mohammed
user
zeeshaan-ai
686c91a7364e2ad167eb53ee
Dhruv Diddi
claimed_verified
2025-07-30T09:05:29.041Z
63477bb66f8773f2a28daa15
/avatars/9a369763a73278cddcf2abcae594865d.svg
Dhruv Diddi
user
ddiddi
Preserving Privacy, Increasing Accessibility, and Reducing Cost: An
  On-Device Artificial Intelligence Model for Medical Transcription and Note
  Generation
Background: Clinical documentation represents a significant burden for
healthcare providers, with physicians spending up to 2 hours daily on
administrative tasks. Recent advances in large language models (LLMs) offer
promising solutions, but privacy concerns and computational requirements limit
their adoption in healthcare settings. Objective: To develop and evaluate a
privacy-preserving, on-device medical transcription system using a fine-tuned
Llama 3.2 1B model capable of generating structured medical notes from medical
transcriptions while maintaining complete data sovereignty entirely in the
browser. Methods: We fine-tuned a Llama 3.2 1B model using Parameter-Efficient
Fine-Tuning (PEFT) with LoRA on 1,500 synthetic medical
transcription-to-structured note pairs. The model was evaluated against the
base Llama 3.2 1B on two datasets: 100 endocrinology transcripts and 140
modified ACI benchmark cases. Evaluation employed both statistical metrics
(ROUGE, BERTScore, BLEURT) and LLM-as-judge assessments across multiple
clinical quality dimensions. Results: The fine-tuned OnDevice model
demonstrated substantial improvements over the base model. On the ACI
benchmark, ROUGE-1 scores increased from 0.346 to 0.496, while BERTScore F1
improved from 0.832 to 0.866. Clinical quality assessments showed marked
reduction in major hallucinations (from 85 to 35 cases) and enhanced factual
correctness (2.81 to 3.54 on 5-point scale). Similar improvements were observed
on the internal evaluation dataset, with composite scores increasing from 3.13
to 4.43 (+41.5%). Conclusions: Fine-tuning compact LLMs for medical
transcription yields clinically meaningful improvements while enabling complete
on-device browser deployment. This approach addresses key barriers to AI
adoption in healthcare: privacy preservation, cost reduction, and accessibility
for resource-constrained environments.
A fine-tuned Llama 3.2 1B model using PEFT with LoRA improves medical transcription quality and enables on-device, privacy-preserving deployment in healthcare.
Parameter-Efficient Fine-Tuning PEFT LoRA medical transcription structured medical notes ROUGE BERTScore BLEURT clinical quality dimensions major hallucinations factual correctness