---
library_name: transformers
tags:
- medical
license: apache-2.0
language:
- fr
- en
base_model:
- mistralai/Mistral-7B-v0.1
---

## Model Description

MedMistral-SFT-7B is a French medical language model based on Mistral-7B-v0.1, adapted for medical domain applications through direct Supervised Fine-Tuning (SFT). 

## Model Details

- **Model Type**: Causal Language Model
- **Base Model**: Mistral-7B-v0.1
- **Language**: French
- **Domain**: Medical/Healthcare
- **License**: Apache 2.0
- **Paper**: [Adaptation des connaissances médicales pour les grands modèles de langue : Stratégies et analyse comparative](https://github.com/ikram28/medllm-strategies)

## Training Details

### Supervised Fine-Tuning (SFT)
- **Dataset**: 30K French medical question-answer pairs
  - 10K native French medical questions
  - 10K translated medical questions from English resources  
  - 10K generated questions from French medical texts
- **Method**: DoRA (Weight-Decomposed Low-Rank Adaptation)
- **Training Duration**: 10 epochs
- **Rank**: 16
- **Alpha**: 16
- **LoRA Dropout**: 0.05
- **Learning Rate**: 2e-5
- **Batch Size**: 4
- **Gradient Accumulation**: 2
- **Optimizer**: AdamW
- **Scheduler**: Cosine
- **Target Modules**: QKVOGUD


## Computational Impact

- **Training Time**: 53 hours
- **Carbon Emissions**: 2.62 kgCO2e


## Ethical Considerations

- **Medical Accuracy**: This model is for research and educational purposes only. All outputs should be verified by qualified medical professionals
- **Bias**: Training data may contain biases present in medical literature and online medical resources



## Citation

If you use this model, please cite:

```bibtex

```

## Contact

For questions about this model, please contact: ikram.belmadani@lis-lab.fr