“a larger accuracy drop in Chinese tasks"? how much exectaly?
#1
by
chuangzhidian
- opened
is the number significantly higher?
Hello zhidian, the chinese tasks results also listed in 'Evaluate the model' table, please refer to 'cmmlu' and 'ceval' tasks. "a larger acc drop", by which we mean a accuracy loss of more than 1% relative to 16bits.