Why is max_position_embeddings set to 40K in config.json but original_max_position_embeddings is 32K in the README? Which one should be used?

#17
by mmyin - opened

In config.json, max_position_embeddings is set to 40K, while in the README under the yarn configuration, original_max_position_embeddings is set to 32K. Which configuration should be used?
If it is 40K, does the yarn extension mean it would expand to 160K?

Sign up or log in to comment