seastar105
/

whisper-base-komixv2-phn

Automatic Speech Recognition

Model card Files Files and versions

Metrics Training metrics Community

whisper-base-komixv2-phn / README.md

seastar105's picture

Update README.md

333037e verified 4 months ago

|

history blame contribute delete

2.6 kB

	---
	library_name: transformers
	tags: []
	---

	### Model Description

	OpenAI의 whisper-base 모델을 아래 데이터셋으로 학습한 모델입니다. phonetic form을 사용하여 학습되었습니다.
	- 한국어 음성 (https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe=realm&dataSetSn=123)
	- 주소 음성 데이터 (https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe=data&dataSetSn=71556)
	- 주요 영역별 회의 음성인식 데이터 (https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe=data&dataSetSn=464)
	- 저음질 전화망 음성인식 데이터 (https://www.aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&dataSetSn=571)
	- 방송 콘텐츠 대화체 음성인식 데이터 (https://www.aihub.or.kr/aihubdata/data/view.do?dataSetSn=463)

	```
	train_steps: 20000
	warmup_steps: 2000
	lr scheduler: linear warmup cosine decay
	max learning rate: 1e-4
	batch size: 256
	max_grad_norm: 1.0
	adamw_beta1: 0.9
	adamw_beta2: 0.98
	```

	### Evaluation

	https://github.com/rtzr/Awesome-Korean-Speech-Recognition

	위 레포지토리에서 주요 영역별 회의 음성을 제외한 테스트셋 결과입니다. 아래 테이블에서 whisper_base_komixv2_phn가 본 모델 성능입니다.


	\| Model \| cv_15_ko \| fleurs_ko \| kcall_testset \| kconf_test \| kcounsel_test \| klec_testset \| kspon_clean \| kspon_other \|
	\|-----------------------\|----------\|-----------\|---------------\|------------\|---------------\|--------------\|-------------\|-------------\|
	\| whisper_base \| 21.16 \| 11.89 \| 42.56 \| 27.62 \| 22.24 \| 28.65 \| 30.41 \| 27.02 \|
	\| whisper_base_komix \| 15.42 \| 7.16 \| 20.86 \| 14.24 \| 12.64 \| 13.44 \| 12.26 \| 12.12 \|
	\| whisper_base_komixv2 \| 13.04 \| 7.04 \| 10.54 \| 13.1 \| 10.65 \| 12.99 \| 12.44 \| 12.56 \|
	\| whisper_base_komixv2_phn \| 12.81 \| 8.27 \| 9.5 \| 13.26 \| 11.33 \| 14.24 \| 13.11 \| 13.3 \|
	\| whisper_large_v3 \| 5.11 \| 3.72 \| 5.45 \| 9.35 \| 3.83 \| 8.46 \| 15.08 \| 12.89 \|
	\| whisper_large_v3_turbo \| 5.38 \| 3.95 \| 5.89 \| 9.77 \| 4.21 \| 9.27 \| 16.49 \| 13.54 \|

	### Acknowledgement
	- 본 모델은 구글의 TRC 프로그램의 지원으로 학습했습니다.
	- Research supported with Cloud TPUs from Google's TPU Research Cloud (TRC)