Update README.md

README.md

@@ -7,43 +7,29 @@ tags:
 - instruction
 - empirischtech
 pipeline_tag: text-generation
+base_model:
+- meta-llama/Llama-3.1-8B-Instruct
 ---
 # LLaMa-10b-instruct model card
 
 ## Model Details
 
-* **Developed by**: [EmpirischTech](https://en.upstage.ai)/[ChaperoneAI](https://en.upstage.ai)
-* **Backbone Model**: [LLaMA](https://github.com/facebookresearch/llama/tree/llama_v1)
-* **Variations**: It has different model parameter sizes and sequence lengths: [30B/1024](https://huggingface.co/upstage/llama-30b-instruct), [30B/2048](https://huggingface.co/upstage/llama-30b-instruct-2048), [65B/1024](https://huggingface.co/upstage/llama-65b-instruct)
+* **Developed by**: [EmpirischTech](https://empirischtech.at)/[ChaperoneAI](https://chaperoneai.net)
+* **Backbone Model**: [LLaMA](https://github.com/meta-llama/llama3)
 * **Language(s)**: English
 * **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
 * **License**: This model is under a **Non-commercial** Bespoke License and governed by the Meta license. You should only use this repository if you have been granted access to the model by filling out [this form](https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf7b_fBxjY_OjhJILlKGA/viewform), but have either lost your copy of the weights or encountered issues converting them to the Transformers format
 * **Where to send comments**: Instructions on how to provide feedback or comments on a model can be found by opening an issue in the [Hugging Face community's model repository](https://huggingface.co/upstage/llama-30b-instruct-2048/discussions)
-* **Contact**: For questions and comments about the model, please email [[email protected]](mailto:contact@upstage.ai)
+* **Contact**: For questions and comments about the model, please email [[email protected]](mailto:contact@empirischtech.at)
 
-## Dataset Details
+## Training
 
-### Used Datasets
 
-- Orca-style dataset
-- No other data was used except for the dataset mentioned above
-
-### Prompt Template
-```
-### System:
-{System}
-
-### User:
-{User}
-
-### Assistant:
-{Assistant}
-```
 
 ## Usage
 
 - Tested on A100 80GB
-- Our model can handle up to 10k+ input tokens, thanks to the `rope_scaling` option
+- Our model can handle up to 132k input tokens as supported by the Llama-3.1 architecture.
 
 ```python
 import torch
@@ -69,8 +55,8 @@ output_text = tokenizer.decode(output[0], skip_special_tokens=True)
 
 ## Hardware and Software
 
-* **Hardware**: We utilized an A100x8 * 4 for training our model
-* **Training Factors**: We fine-tuned this model using a combination of the [DeepSpeed library](https://github.com/microsoft/DeepSpeed) and the [HuggingFace Trainer](https://huggingface.co/docs/transformers/main_classes/trainer)
+* **Hardware**: We utilized an A100x8  for training our model
+* **Training Factors**: The model was pretrained using a combination of the [DeepSpeed library](https://github.com/microsoft/DeepSpeed) and the [HuggingFace Trainer](https://huggingface.co/docs/transformers/main_classes/trainer)
 
 ## Evaluation Results
 
@@ -93,17 +79,32 @@ We used the [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-
 | falcon-40b-instruct | 63.4 | 61.6 | 84.3 | 55.4 | 52.5 | | |
 
 
-### Scripts for H4 Score Reproduction
+### Scripts to generate evalution results
 - Prepare evaluation environments:
 ```
-# clone the repository
-git clone https://github.com/EleutherAI/lm-evaluation-harness.git
+# install from https://github.com/EleutherAI/lm-evaluation-harness
+pip install lm-eval>=0.4.7
+
+from lm_eval import evaluator
+
+tasks_list = ["arc_challenge", "gpqa", "ifeval", "mmlu_pro", "hellaswag"]  # Benchmark dataset
+
+model_path='rwmasood/llama-3.1-10b-instruct'
+
+# Run evaluation
+results = evaluator.simple_evaluate(
+    model="hf",  # Hugging Face model
+    cache_requests=False,
+    model_args=f"pretrained={model_path}",
+    tasks=tasks_list, 
+    batch_size=4,
+    device="cuda:0" 
+)
 
-# check out the specific commit
-git checkout b281b0921b636bc36ad05c0b0b0763bd6dd43463
+# Extract results
+results = results['results']
+json_string = json.dumps(results, indent=4)
 
-# change to the repository directory
-cd lm-evaluation-harness
 ```
 
 ## Ethical Issues
@@ -114,4 +115,4 @@ cd lm-evaluation-harness
 ## Contact Us
 
 ### Why Upstage LLM?
-- [Upstage](https://en.upstage.ai)'s LLM research has yielded remarkable results. As of August 1st, our 70B model has reached the top spot in openLLM rankings, marking itself as the current leading performer globally. Recognizing the immense potential in implementing private LLM to actual businesses, we invite you to easily apply private LLM and fine-tune it with your own data. For a seamless and tailored solution, please do not hesitate to reach out to us. ► [click here to contact](https://www.upstage.ai/private-llm?utm_source=huggingface&utm_medium=link&utm_campaign=privatellm)
+- [EmpirischTech](https://empirischtech.at)/[ChaperoneAI](https://chaperoneai.net) Unlock the full potential of private LLMs for your business with ease. Customize and fine-tune them using your own data for a solution that fits your unique needs. Want a seamless integration? Let’s connect! ► [Get in touch](https://chaperoneai.net/contact)