Question on meaning of parameter of this model
#2
by
JLouisBiz
- opened
The MedIT One model is an early checkpoint in the development of the One series, evaluated after 9 billion tokens of training.
Could you please explain it?
It is 140 million parameters model trained on 9 billion tokens. I would need better explanation of it.
Does it mean it is more powerful then common small models?
Hey, no, it means that this model needs significantly more training to be useful. For now, it's just a research preview.
- 140 million parameters represent the total number of weight parameters that store information about language.
- 9 billion tokens refer to the total amount of data processed during pre-training up to the release, counted in tokens (parts of words).
I hope this information was helpful. Please let me know if you have any further questions.
I was thinking it is an improvement in model design, could you make it clear?