Hugging Face's logo

alfaneo
/
jurisbert-base-portuguese-uncased

Fill-Mask
Transformers
PyTorch
Safetensors
bert
Generated from Trainer
Model card Files Community
2
charlesviegas commited on
Commit
29c1d9c
·
1 Parent(s): a8918f6

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +149 -0
README.md ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: bertlawbr
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # bertlawbr
13
+
14
+ This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 1.0495
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 0.0001
36
+ - train_batch_size: 16
37
+ - eval_batch_size: 16
38
+ - seed: 42
39
+ - gradient_accumulation_steps: 8
40
+ - total_train_batch_size: 128
41
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-06
42
+ - lr_scheduler_type: linear
43
+ - lr_scheduler_warmup_steps: 10000
44
+ - num_epochs: 20.0
45
+ - mixed_precision_training: Native AMP
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:------:|:---------------:|
51
+ | 6.1291 | 0.22 | 2500 | 5.9888 |
52
+ | 4.8604 | 0.44 | 5000 | 4.4841 |
53
+ | 3.3321 | 0.66 | 7500 | 3.1190 |
54
+ | 2.7579 | 0.87 | 10000 | 2.6089 |
55
+ | 2.4135 | 1.09 | 12500 | 2.3029 |
56
+ | 2.2136 | 1.31 | 15000 | 2.1244 |
57
+ | 2.0735 | 1.53 | 17500 | 1.9931 |
58
+ | 1.9684 | 1.75 | 20000 | 1.8878 |
59
+ | 1.891 | 1.97 | 22500 | 1.8077 |
60
+ | 1.8215 | 2.18 | 25000 | 1.7487 |
61
+ | 1.7577 | 2.4 | 27500 | 1.6875 |
62
+ | 1.7113 | 2.62 | 30000 | 1.6444 |
63
+ | 1.6776 | 2.84 | 32500 | 1.6036 |
64
+ | 1.6203 | 3.06 | 35000 | 1.5608 |
65
+ | 1.6018 | 3.28 | 37500 | 1.5293 |
66
+ | 1.5602 | 3.5 | 40000 | 1.5044 |
67
+ | 1.5429 | 3.71 | 42500 | 1.4753 |
68
+ | 1.5148 | 3.93 | 45000 | 1.4472 |
69
+ | 1.4786 | 4.15 | 47500 | 1.4302 |
70
+ | 1.4653 | 4.37 | 50000 | 1.4128 |
71
+ | 1.4496 | 4.59 | 52500 | 1.3991 |
72
+ | 1.4445 | 4.81 | 55000 | 1.3943 |
73
+ | 1.5114 | 5.02 | 57500 | 1.4551 |
74
+ | 1.5054 | 5.24 | 60000 | 1.4525 |
75
+ | 1.4817 | 5.46 | 62500 | 1.4259 |
76
+ | 1.48 | 5.68 | 65000 | 1.4077 |
77
+ | 1.4526 | 5.9 | 67500 | 1.3912 |
78
+ | 1.4272 | 6.12 | 70000 | 1.3726 |
79
+ | 1.4078 | 6.34 | 72500 | 1.3596 |
80
+ | 1.399 | 6.55 | 75000 | 1.3450 |
81
+ | 1.386 | 6.77 | 77500 | 1.3328 |
82
+ | 1.3704 | 6.99 | 80000 | 1.3192 |
83
+ | 1.3538 | 7.21 | 82500 | 1.3131 |
84
+ | 1.3468 | 7.43 | 85000 | 1.2916 |
85
+ | 1.323 | 7.65 | 87500 | 1.2871 |
86
+ | 1.322 | 7.86 | 90000 | 1.2622 |
87
+ | 1.2956 | 8.08 | 92500 | 1.2624 |
88
+ | 1.2869 | 8.3 | 95000 | 1.2547 |
89
+ | 1.2763 | 8.52 | 97500 | 1.2404 |
90
+ | 1.275 | 8.74 | 100000 | 1.2305 |
91
+ | 1.2709 | 8.96 | 102500 | 1.2301 |
92
+ | 1.2514 | 9.18 | 105000 | 1.2179 |
93
+ | 1.2563 | 9.39 | 107500 | 1.2134 |
94
+ | 1.2487 | 9.61 | 110000 | 1.2111 |
95
+ | 1.2337 | 9.83 | 112500 | 1.2041 |
96
+ | 1.3215 | 10.05 | 115000 | 1.2879 |
97
+ | 1.3364 | 10.27 | 117500 | 1.2850 |
98
+ | 1.3286 | 10.49 | 120000 | 1.2779 |
99
+ | 1.3202 | 10.7 | 122500 | 1.2730 |
100
+ | 1.3181 | 10.92 | 125000 | 1.2651 |
101
+ | 1.2952 | 11.14 | 127500 | 1.2544 |
102
+ | 1.2889 | 11.36 | 130000 | 1.2506 |
103
+ | 1.2747 | 11.58 | 132500 | 1.2339 |
104
+ | 1.2729 | 11.8 | 135000 | 1.2277 |
105
+ | 1.2699 | 12.02 | 137500 | 1.2201 |
106
+ | 1.2508 | 12.23 | 140000 | 1.2163 |
107
+ | 1.2438 | 12.45 | 142500 | 1.2091 |
108
+ | 1.2445 | 12.67 | 145000 | 1.2003 |
109
+ | 1.2314 | 12.89 | 147500 | 1.1957 |
110
+ | 1.2188 | 13.11 | 150000 | 1.1843 |
111
+ | 1.2071 | 13.33 | 152500 | 1.1805 |
112
+ | 1.2123 | 13.54 | 155000 | 1.1766 |
113
+ | 1.2016 | 13.76 | 157500 | 1.1661 |
114
+ | 1.2079 | 13.98 | 160000 | 1.1625 |
115
+ | 1.1884 | 14.2 | 162500 | 1.1525 |
116
+ | 1.177 | 14.42 | 165000 | 1.1419 |
117
+ | 1.1793 | 14.64 | 167500 | 1.1454 |
118
+ | 1.173 | 14.85 | 170000 | 1.1379 |
119
+ | 1.1502 | 15.07 | 172500 | 1.1371 |
120
+ | 1.1504 | 15.29 | 175000 | 1.1295 |
121
+ | 1.146 | 15.51 | 177500 | 1.1203 |
122
+ | 1.1487 | 15.73 | 180000 | 1.1137 |
123
+ | 1.1329 | 15.95 | 182500 | 1.1196 |
124
+ | 1.1259 | 16.17 | 185000 | 1.1075 |
125
+ | 1.1287 | 16.38 | 187500 | 1.1037 |
126
+ | 1.126 | 16.6 | 190000 | 1.1042 |
127
+ | 1.1199 | 16.82 | 192500 | 1.0953 |
128
+ | 1.1072 | 17.04 | 195000 | 1.0885 |
129
+ | 1.1043 | 17.26 | 197500 | 1.0877 |
130
+ | 1.1007 | 17.48 | 200000 | 1.0835 |
131
+ | 1.0879 | 17.69 | 202500 | 1.0819 |
132
+ | 1.1 | 17.91 | 205000 | 1.0744 |
133
+ | 1.0863 | 18.13 | 207500 | 1.0774 |
134
+ | 1.087 | 18.35 | 210000 | 1.0759 |
135
+ | 1.0755 | 18.57 | 212500 | 1.0618 |
136
+ | 1.0832 | 18.79 | 215000 | 1.0628 |
137
+ | 1.0771 | 19.01 | 217500 | 1.0611 |
138
+ | 1.0703 | 19.22 | 220000 | 1.0555 |
139
+ | 1.069 | 19.44 | 222500 | 1.0552 |
140
+ | 1.0706 | 19.66 | 225000 | 1.0509 |
141
+ | 1.0633 | 19.88 | 227500 | 1.0465 |
142
+
143
+
144
+ ### Framework versions
145
+
146
+ - Transformers 4.12.5
147
+ - Pytorch 1.10.1+cu113
148
+ - Datasets 1.17.0
149
+ - Tokenizers 0.10.3