Korean GPT Bot Sentiment Classification (ko-gpt-bot-sc)
Method
- Promt-Tuning/Prefix-tuning/Soft Embedding
- Parameters
Parameters No. All 6173039616 (100.0%) Trainable 6537216 (0.1%) Freezed 6166502400 (99.9%)
Model
LAYER NAME #PARAMS RATIO MEM(MB)
--model: 6,177,233,921 100.00% 23552.28
--learned_embedding: 6,537,216 0.11% 24.94
--transformer: 5,906,391,041 95.62% 22519.09
--wte
--weight: 264,241,152 4.28% 1008.00
--h: 5,642,141,697 91.34% 21511.06
--0: 205,549,569 3.33% 772.11
--ln_1: 8,192 0.00% 0.03
--attn: 71,303,169 1.15% 260.00
--mlp: 134,238,208 2.17% 512.08
--1(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--2(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--3(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--4(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--5(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--6(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--7(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--8(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--9(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--10(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--11(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--12(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--13(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--14(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--15(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--16(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--17(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--18(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--19(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--20(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--21(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--22(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--23(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--24(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--25(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--26(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--27(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--ln_f: 8,192 0.00% 0.03
--weight: 4,096 0.00% 0.02
--bias: 4,096 0.00% 0.02
--lm_head: 264,305,664 4.28% 1008.25
--weight: 264,241,152 4.28% 1008.00
--bias: 64,512 0.00% 0.25
Metrics
| Metric | Value |
|---|---|
| step | 520 |
| loss | 3.1814 |
| precision | recall | f1-score | support | |
|---|---|---|---|---|
| 긍정 | 0.92549 | 0.944 | 0.934653 | 500 |
| 부정 | 0.942857 | 0.924 | 0.933333 | 500 |
| accuracy | 0.934 | 0.934 | 0.934 | 0.934 |
| macro avg | 0.934174 | 0.934 | 0.933993 | 1000 |
| weighted avg | 0.934174 | 0.934 | 0.933993 | 1000 |
References
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.