Yingxu He
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -73,12 +73,9 @@ as evidenced by evaluation results on Singapore's [Multitask National Speech Cor
|
|
73 |
> MNSC is a multitask speech understanding dataset derived and further annotated from [IMDA NSC Corpus](https://www.imda.gov.sg/how-we-can-help/national-speech-corpus).
|
74 |
> It focuses on the knowledge of Singapore's local accent, localised terms, and code-switching.
|
75 |
|
76 |
-
|
77 |
-
|
78 |
-
|
79 |
-
> which uses a pre-trained large language model to evaluate task performance
|
80 |
-
> by generating and scoring responses based on relevance, coherence, and accuracy criteria.
|
81 |
-
> Refer to the [AudioBench paper](https://arxiv.org/abs/2406.16020) for more details.
|
82 |
|
83 |
<div class="table*">
|
84 |
<table>
|
@@ -568,12 +565,16 @@ With a global batch size of 640, we train the current release of MERaLiON-AudioL
|
|
568 |
|
569 |
## Citation
|
570 |
|
571 |
-
|
572 |
-
|
573 |
-
**BibTeX:**
|
574 |
-
|
575 |
-
[More Information Needed]
|
576 |
|
577 |
-
|
578 |
-
|
579 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
> MNSC is a multitask speech understanding dataset derived and further annotated from [IMDA NSC Corpus](https://www.imda.gov.sg/how-we-can-help/national-speech-corpus).
|
74 |
> It focuses on the knowledge of Singapore's local accent, localised terms, and code-switching.
|
75 |
|
76 |
+
We assess ASR and ST tasks using Word Error Rate (WER) and BLEU scores, respectively. For other tasks, we employ the LLM-as-a-Judge framework,
|
77 |
+
which uses a pre-trained large language model to evaluate task performance by generating and scoring responses based on relevance, coherence, and accuracy criteria.
|
78 |
+
Refer to the [AudioBench paper](https://arxiv.org/abs/2406.16020) for more details.
|
|
|
|
|
|
|
79 |
|
80 |
<div class="table*">
|
81 |
<table>
|
|
|
565 |
|
566 |
## Citation
|
567 |
|
568 |
+
If you find our work useful, please cite our paper:
|
|
|
|
|
|
|
|
|
569 |
|
570 |
+
```
|
571 |
+
@misc{he2024meralionaudiollmtechnicalreport,
|
572 |
+
title={MERaLiON-AudioLLM: Technical Report},
|
573 |
+
author={Yingxu He and Zhuohan Liu and Shuo Sun and Bin Wang and Wenyu Zhang and Xunlong Zou and Nancy F. Chen and Ai Ti Aw},
|
574 |
+
year={2024},
|
575 |
+
eprint={2412.09818},
|
576 |
+
archivePrefix={arXiv},
|
577 |
+
primaryClass={cs.CL},
|
578 |
+
url={https://arxiv.org/abs/2412.09818},
|
579 |
+
}
|
580 |
+
```
|