|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- medical |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- fr |
|
|
- en |
|
|
base_model: |
|
|
- mistralai/Mistral-7B-Instruct-v0.1 |
|
|
--- |
|
|
|
|
|
|
|
|
### Model Description |
|
|
|
|
|
MedMistralInstruct-CPT-7B is adapted from Mistral-7B-Instruct-v0.1 through Continual Pre-Training, maintaining instruction-following capabilities while gaining medical domain knowledge. |
|
|
|
|
|
### Model Details |
|
|
|
|
|
- **Model Type**: Causal Language Model |
|
|
- **Base Model**: Mistral-7B-Instruct-v0.1 |
|
|
- **Language**: French |
|
|
- **Domain**: Medical/Healthcare |
|
|
- **Parameters**: 7 billion |
|
|
- **License**: Apache 2.0 |
|
|
|
|
|
### Training Details |
|
|
|
|
|
**Continual Pre-Training (CPT)** |
|
|
- **Dataset**: NACHOS corpus (7.4 GB French medical texts) |
|
|
- **Training Duration**: 2.8 epochs |
|
|
- **Hardware**: 32 NVIDIA A100 80GB GPUs |
|
|
- **Training Time**: ~40 hours |
|
|
|
|
|
|
|
|
### Computational Requirements |
|
|
- **Carbon Emissions**: 32.89 kgCO2e |
|
|
- **Training Time**: 40 hours |
|
|
|
|
|
|
|
|
### Ethical Considerations |
|
|
- **Medical Accuracy**: For research and educational purposes only |
|
|
- **Professional Oversight**: Requires verification by qualified medical professionals |
|
|
- **Bias Awareness**: May contain biases from training data |
|
|
- **Privacy**: Do not input private health information |
|
|
|
|
|
### Citation |
|
|
```bibtex |
|
|
|
|
|
``` |
|
|
|
|
|
### Contact |
|
|
For questions about these models, please contact: [email protected] |
