Update README.md
Browse files
README.md
CHANGED
@@ -29,7 +29,7 @@ Current Version: `14.04.2025`
|
|
29 |
|
30 |
# Llama-SEA-LION-v3-70B-IT
|
31 |
|
32 |
-
SEA-LION is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
|
33 |
|
34 |
SEA-LION stands for _Southeast Asian Languages In One Network_.
|
35 |
|
@@ -49,7 +49,7 @@ For tokenisation, the model employs the default tokenizer used in Llama 3.1 70B
|
|
49 |
We evaluated Llama-SEA-LION-v3-70B-IT on both general language capabilities and instruction-following capabilities.
|
50 |
|
51 |
#### General Language Capabilities
|
52 |
-
For the evaluation of general language capabilities, we employed the [SEA-HELM
|
53 |
These tasks include Question Answering (QA), Sentiment Analysis (Sentiment), Toxicity Detection (Toxicity), Translation in both directions (Eng>Lang & Lang>Eng), Abstractive Summarisation (Abssum), Causal Reasoning (Causal) and Natural Language Inference (NLI).
|
54 |
|
55 |
Note: SEA-HELM is implemented using prompts to elicit answers in a strict format. For all tasks, the model is expected to provide an answer tag from which the answer is automatically extracted. For tasks where options are provided, the answer should comprise one of the pre-defined options. The scores for each task is normalised to account for baseline performance due to random chance.
|
|
|
29 |
|
30 |
# Llama-SEA-LION-v3-70B-IT
|
31 |
|
32 |
+
[SEA-LION](https://arxiv.org/abs/2504.05747) is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
|
33 |
|
34 |
SEA-LION stands for _Southeast Asian Languages In One Network_.
|
35 |
|
|
|
49 |
We evaluated Llama-SEA-LION-v3-70B-IT on both general language capabilities and instruction-following capabilities.
|
50 |
|
51 |
#### General Language Capabilities
|
52 |
+
For the evaluation of general language capabilities, we employed the [SEA-HELM evaluation benchmark](https://arxiv.org/abs/2502.14301) across a variety of tasks.
|
53 |
These tasks include Question Answering (QA), Sentiment Analysis (Sentiment), Toxicity Detection (Toxicity), Translation in both directions (Eng>Lang & Lang>Eng), Abstractive Summarisation (Abssum), Causal Reasoning (Causal) and Natural Language Inference (NLI).
|
54 |
|
55 |
Note: SEA-HELM is implemented using prompts to elicit answers in a strict format. For all tasks, the model is expected to provide an answer tag from which the answer is automatically extracted. For tasks where options are provided, the answer should comprise one of the pre-defined options. The scores for each task is normalised to account for baseline performance due to random chance.
|