assiddeek/my-custom-distilbert-model

README.md

@@ -0,0 +1,60 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: distilbert-base-uncased
+tags:
+- generated_from_trainer
+model-index:
+- name: results
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# results
+
+This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 6.1340
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- num_epochs: 3
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| No log        | 1.0   | 1    | 6.2041          |
+| No log        | 2.0   | 2    | 6.1561          |
+| No log        | 3.0   | 3    | 6.1340          |
+
+
+### Framework versions
+
+- Transformers 4.46.2
+- Pytorch 2.5.1+cu121
+- Datasets 3.1.0
+- Tokenizers 0.20.3

config.json

@@ -0,0 +1,24 @@
+{
+  "_name_or_path": "distilbert-base-uncased",
+  "activation": "gelu",
+  "architectures": [
+    "DistilBertForQuestionAnswering"
+  ],
+  "attention_dropout": 0.1,
+  "dim": 768,
+  "dropout": 0.1,
+  "hidden_dim": 3072,
+  "initializer_range": 0.02,
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "pad_token_id": 0,
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": false,
+  "tie_weights_": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.2",
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0df7221653072e1de8a7868bf1b0806613d816213465aac0d5c186189c51cb4e
+size 265470032

runs/Dec03_04-55-19_60f451b8613b/events.out.tfevents.1733201724.60f451b8613b.299.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf941316cef1bdc302ae18723da4dcb43a68feab426e46b808384f204fa1b237
+size 6062

runs/Dec03_07-55-16_60f451b8613b/events.out.tfevents.1733212520.60f451b8613b.299.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c83207f652c8b883129d1bcb5cc3351d4f404eaab07b575c2cfc896e7c1f1c22
+size 6062

runs/Dec03_08-00-02_60f451b8613b/events.out.tfevents.1733212803.60f451b8613b.299.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4000af60087b5b877f198ea81ab0f7a43c3b04434ac23bbf493f17f9c4c7690e
+size 6062

runs/Dec03_08-17-57_60f451b8613b/events.out.tfevents.1733213878.60f451b8613b.299.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38e6cf9527cc1a26f4135cd4ea704d3c608c85a36205cc95df7c503039f1cc3f
+size 6062

runs/Dec03_08-19-31_60f451b8613b/events.out.tfevents.1733213972.60f451b8613b.299.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51edd8bde37af0667196fdc584e48336ea26c46f6b8366067af1654da1783c5d
+size 6062

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d9706e98ba521f5146070bc15e0d67ce9d42997ef9feb206b73b3f7a2154656a
+size 5240