metadata
library_name: transformers
tags:
- medical
license: apache-2.0
language:
- fr
- en
base_model:
- ik-ram28/MedMistralInstruct-CPT-7B
- mistralai/Mistral-7B-Instruct-v0.1
MedMistralInstruct-CPT-SFT-7B
Model Description
MedMistralInstruct-CPT-SFT-7B is a French medical language model based on Mistral-7B-Instruct-v0.1, adapted through Continual Pre-Training followed by Supervised Fine-Tuning.
Model Details
- Model Type: Causal Language Model
- Base Model: Mistral-7B-Instruct-v0.1
- Language: French
- Domain: Medical/Healthcare
- Parameters: 7 billion
- License: Apache 2.0
Training Details
Continual Pre-Training (CPT)
- Dataset: NACHOS corpus (7.4 GB French medical texts)
- Training Duration: 2.8 epochs
- Hardware: 32 NVIDIA A100 80GB GPUs
- Training Time: ~40 hours
Supervised Fine-Tuning (SFT)
- Dataset: 30K French medical question-answer pairs
- Method: DoRA (Weight-Decomposed Low-Rank Adaptation)
- Training Duration: 10 epochs
- Hardware: 1 NVIDIA H100 80GB GPU
- Training Time: ~42 hours
Computational Requirements
- Carbon Emissions: 33.96 kgCO2e (CPT+SFT)
- Training Time: 82 hours total (CPT+SFT)
Ethical Considerations
- Medical Accuracy: For research and educational purposes only
- Professional Oversight: Requires verification by qualified medical professionals
- Bias Awareness: May contain biases from training data
- Privacy: Do not input private health information
Citation
Contact
For questions about these models, please contact: [email protected]