metadata
library_name: transformers
tags:
- medical
license: apache-2.0
language:
- fr
- en
base_model:
- mistralai/Mistral-7B-Instruct-v0.1
Model Description
MedMistralInstruct-CPT-7B is adapted from Mistral-7B-Instruct-v0.1 through Continual Pre-Training, maintaining instruction-following capabilities while gaining medical domain knowledge.
Model Details
- Model Type: Causal Language Model
- Base Model: Mistral-7B-Instruct-v0.1
- Language: French
- Domain: Medical/Healthcare
- Parameters: 7 billion
- License: Apache 2.0
Training Details
Continual Pre-Training (CPT)
- Dataset: NACHOS corpus (7.4 GB French medical texts)
- Training Duration: 2.8 epochs
- Hardware: 32 NVIDIA A100 80GB GPUs
- Training Time: ~40 hours
Computational Requirements
- Carbon Emissions: 32.89 kgCO2e
- Training Time: 40 hours
Ethical Considerations
- Medical Accuracy: For research and educational purposes only
- Professional Oversight: Requires verification by qualified medical professionals
- Bias Awareness: May contain biases from training data
- Privacy: Do not input private health information
Citation
Contact
For questions about these models, please contact: [email protected]