docs: add README.md

README.md

@@ -1,13 +1,21 @@
 ---
-library_name: transformers
-tags: []
+# For reference on model card metadata, see the spec: https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1
+# Doc / guide: https://huggingface.co/docs/hub/model-cards
+base_model:
+- Qwen/Qwen2.5-0.5B
+datasets: []
+languages:
+- en
+metrics: []
+pipeline_tag: text-generation
+
 ---
 
-# Model Card for Model ID
+# Model Card for ldp72/Test-Qwen-Marcel.5-0.5B-it
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-
+This model was finetuned by performing instruct tuning on Telco domain datatsets.
 
 ## Model Details
 
@@ -15,15 +23,16 @@ tags: []
 
 <!-- Provide a longer summary of what this model is. -->
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
 
 - **Developed by:** [More Information Needed]
 - **Funded by [optional]:** [More Information Needed]
 - **Shared by [optional]:** [More Information Needed]
 - **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
+- **Language(s) (NLP):** English
 - **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Finetuned from model [optional]:** ['Qwen/Qwen2.5-0.5B']
+- **Date [optional]:** 2025-07-16 14:40:15
 
 ### Model Sources [optional]
 
@@ -41,7 +50,30 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-[More Information Needed]
+
+This model can be used with the `transformers` library using `pipeline` abstraction as follows:
+
+```python
+import torch
+from transformers import pipeline
+
+model_id = "ldp72/Test-Qwen-Marcel.5-0.5B-it"
+pipe = pipeline(
+"text-generation",
+model=model_id,
+torch_dtype=torch.bfloat16,
+device_map="auto",
+)
+messages = [
+{"role": "system", "content": "You are chatbot specialized on Telco domain."},
+{"role": "user", "content": "Can you give a sample of your specialized knowledge?"},
+]
+outputs = pipe(
+messages,
+max_new_tokens=256,
+)
+print(outputs[0]["generated_text"][-1])
+```
 
 ### Downstream Use [optional]
 
@@ -75,11 +107,182 @@ Use the code below to get started with the model.
 
 ## Training Details
 
+This model was finetuned with [Orange internal fine tuning tools](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/)  with the Docker Image tagged `0.1.1` in the [registry](https://gitlab.tech.orange/NEPAL/knowledge/orangelm/lm-adaptation/container_registry/84664) and the following configuration file:
+
+```yaml
+data:
+dataset_name:
+train:
+-   path: telco-lm/arxiv-abstract-generation-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-dsp.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-networkengineering.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-security.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-3gpp-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-5gamericas-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-huawei-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-itu-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-mef-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-ngmn-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-rfc-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/teleqna-mcqa-cot-telco-instructions
+revision: legacy
+-   path: telco-lm/tii-huawei-qa-open-qa-telco-instructions
+revision: legacy
+validation_abstract_generation:
+-   path: telco-lm/arxiv-abstract-generation-telco-instructions
+revision: legacy
+split: validation
+validation_general:
+-   path: telco-lm/slim-orca-multi-task-general-instructions
+revision: legacy
+split: validation
+validation_synthetic:
+-   path: telco-lm/synthetic-dsp.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-security.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-networkengineering.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-rfc-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-3gpp-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-5gamericas-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-itu-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-mef-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-huawei-multi-task-telco-instructions
+revision: legacy
+split: validation
+-   path: telco-lm/synthetic-technical-ngmn-multi-task-telco-instructions
+revision: legacy
+split: validation
+validation_telco_qa:
+-   path: telco-lm/tii-huawei-qa-open-qa-telco-instructions
+revision: legacy
+split: validation
+validation_telco_qcm:
+-   path: telco-lm/teleqna-mcqa-cot-telco-instructions
+revision: legacy
+split: validation
+debug: true
+implementation_name: instructions
+description:
+contributors:
+-   email: [email protected]
+first_name: Loïc
+last_name: Fosse
+-   email: [email protected]
+first_name: Lionel
+last_name: Delphin-Poulat
+-   email: [email protected]
+first_name: Ismaël
+last_name: Rousseau
+domain: Telco
+languages:
+- en
+model_name: ldp72/Test-Qwen-Marcel.5-0.5B-it
+image:
+version: 0.1.1
+model:
+attn_implementation: flash_attention_2
+chat_template_tokenizer: Qwen/Qwen2.5-0.5B-Instruct
+model_name_or_path: Qwen/Qwen2.5-0.5B
+trust_remote_code: true
+training:
+bf16: true
+dataloader_num_workers: 4
+dataloader_persistent_workers: true
+dataloader_pin_memory: true
+dataloader_prefetch_factor: 2
+disable_tqdm: true
+eval_accumulation_steps: 1
+eval_steps: 10
+eval_strategy: steps
+fp16: false
+gradient_accumulation_steps: 2
+gradient_checkpointing: true
+group_by_length: false
+learning_rate: 2.0e-05
+log_level: debug
+logging_dir: /outputs/Telco-Qwen2.5-0.5B-it-profiling-nodeepspeed-1gpu-2/logs
+logging_steps: 10
+lr_scheduler_type: cosine
+max_grad_norm: 1.0
+max_steps: -1
+num_train_epochs: 2
+optim: paged_adamw_32bit
+output_dir: /outputs/Telco-Qwen2.5-0.5B-it-profiling-nodeepspeed-1gpu-2
+per_device_eval_batch_size: 2
+per_device_train_batch_size: 2
+push_to_hub: false
+report_to: tensorboard
+save_steps: 0
+save_strategy: epoch
+save_total_limit: 1
+seed: 42
+torch_compile: false
+training_type: instruct-tuning
+use_liger_kernel: false
+warmup_ratio: 0.05
+weight_decay: 0.1
+```
+
 ### Training Data
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+This model was trained on the following datasets:
+
+```yaml
+-   path: telco-lm/arxiv-abstract-generation-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-dsp.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-networkengineering.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-security.stackexchange.com-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-3gpp-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-5gamericas-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-huawei-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-itu-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-mef-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-ngmn-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/synthetic-technical-rfc-multi-task-telco-instructions
+revision: legacy
+-   path: telco-lm/teleqna-mcqa-cot-telco-instructions
+revision: legacy
+-   path: telco-lm/tii-huawei-qa-open-qa-telco-instructions
+revision: legacy
+```
 
 ### Training Procedure
 
@@ -89,10 +292,47 @@ Use the code below to get started with the model.
 
 [More Information Needed]
 
-
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Training regime:** This model was trained with the following hyperparameters for `SFTTrainer`,other parameters were set as default:
+
+```yaml
+bf16: true
+dataloader_num_workers: 4
+dataloader_persistent_workers: true
+dataloader_pin_memory: true
+dataloader_prefetch_factor: 2
+disable_tqdm: true
+eval_accumulation_steps: 1
+eval_steps: 10
+eval_strategy: steps
+fp16: false
+gradient_accumulation_steps: 2
+gradient_checkpointing: true
+group_by_length: false
+learning_rate: 2.0e-05
+log_level: debug
+logging_dir: /outputs/Telco-Qwen2.5-0.5B-it-profiling-nodeepspeed-1gpu-2/logs
+logging_steps: 10
+lr_scheduler_type: cosine
+max_grad_norm: 1.0
+max_steps: -1
+num_train_epochs: 2
+optim: paged_adamw_32bit
+output_dir: /outputs/Telco-Qwen2.5-0.5B-it-profiling-nodeepspeed-1gpu-2
+per_device_eval_batch_size: 2
+per_device_train_batch_size: 2
+push_to_hub: false
+report_to: tensorboard
+save_steps: 0
+save_strategy: epoch
+save_total_limit: 1
+seed: 42
+torch_compile: false
+use_liger_kernel: false
+warmup_ratio: 0.05
+weight_decay: 0.1
+``` <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
 #### Speeds, Sizes, Times [optional]
 
@@ -196,4 +436,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 ## Model Card Contact
 
-[More Information Needed]
+Thanks to [Loïc Fosse](mailto:[email protected]), [Lionel Delphin-Poulat](mailto:[email protected]), [Ismaël Rousseau](mailto:[email protected]) for adding this model.