---
library_name: peft
license: gemma
base_model: google/gemma-2-9b-it
tags:
- generated_from_trainer
model-index:
- name: gemma-2-9b-it-009
  results: []
language:
- es
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gemma-2-9b-it-009

Éste modelo es un afinamiento del modelo [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)  sobre un dataset 
del acuerdo 009 que reglamenta los deberes y derechos de los estdiantes en la Universidad del Valle. Fueron entrenados 1700 ejemplos
de Contexto-pregunta-respuesta. El modelo logra una precisión  de una función de pérdida sobre el conjunto de 
validación de 0.3876 y una pérdida entrenamiento
final de 0.5801.

## Model description

Gemma es el modelo release de Gemini con resultados sorprendentes pero  son modelos muy grandes y para usarlos
se necesitan submódulos offload_dir.

## Intended uses & limitations

El modelo fue entrenado en una proceador Intel corel de 9000  de 8th generación de 32 de memoria RAM y una RTX 4090 de 24 VRAM. La GPU
funciona en una tarjeta MSI, la Z390-A PRO.

## Training and evaluation data

More information needed

## Training procedure

El modelo fue entreanado durante 7 horas a razón de un batch_size=1 y  10 épocas.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 4
- optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.666         | 0.2694 | 100  | 1.6166          |
| 1.4221        | 0.5387 | 200  | 1.4583          |
| 1.3024        | 0.8081 | 300  | 1.3236          |
| 1.1221        | 1.0754 | 400  | 1.2181          |
| 1.0532        | 1.3448 | 500  | 1.0983          |
| 0.932         | 1.6141 | 600  | 0.9850          |
| 0.9859        | 1.8835 | 700  | 0.8838          |
| 0.7822        | 2.1508 | 800  | 0.7971          |
| 0.6687        | 2.4202 | 900  | 0.7172          |
| 0.6775        | 2.6896 | 1000 | 0.6710          |
| 0.6049        | 2.9589 | 1100 | 0.6171          |
| 0.5734        | 3.2263 | 1200 | 0.5802          |
| 0.4267        | 3.4956 | 1300 | 0.5537          |
| 0.5499        | 3.7650 | 1400 | 0.5336          |
| 0.4234        | 4.0323 | 1500 | 0.5121          |
| 0.4843        | 4.3017 | 1600 | 0.4949          |
| 0.4118        | 4.5710 | 1700 | 0.4809          |
| 0.3518        | 4.8404 | 1800 | 0.4662          |
| 0.2949        | 5.1077 | 1900 | 0.4510          |
| 0.3606        | 5.3771 | 2000 | 0.4443          |
| 0.366         | 5.6465 | 2100 | 0.4408          |
| 0.3728        | 5.9158 | 2200 | 0.4238          |
| 0.3225        | 6.1832 | 2300 | 0.4235          |
| 0.3028        | 6.4525 | 2400 | 0.4224          |
| 0.3983        | 6.7219 | 2500 | 0.4068          |
| 0.28          | 6.9912 | 2600 | 0.4063          |
| 0.2957        | 7.2586 | 2700 | 0.4071          |
| 0.3566        | 7.5279 | 2800 | 0.4010          |
| 0.3433        | 7.7973 | 2900 | 0.3976          |
| 0.3296        | 8.0646 | 3000 | 0.3958          |
| 0.2444        | 8.3340 | 3100 | 0.3951          |
| 0.2464        | 8.6034 | 3200 | 0.3913          |
| 0.267         | 8.8727 | 3300 | 0.3881          |
| 0.279         | 9.1401 | 3400 | 0.3899          |
| 0.2737        | 9.4094 | 3500 | 0.3898          |
| 0.234         | 9.6788 | 3600 | 0.3884          |
| 0.3132        | 9.9481 | 3700 | 0.3876          |


### Framework versions

- PEFT 0.15.2
- Transformers 4.51.3
- Pytorch 2.6.0+cu126
- Datasets 3.5.0
- Tokenizers 0.21.1