Hieu Lam PRO

Sounds interesting but I think there will be a big breakthrough, a new "architecture/methodology/factor/rethinking" for developing large models. That's what I think, I don't know what it is yet, haha.

thanks @danielus 🤗

@Dihelson @llama-anon @AIWizard76 @danielus
🎉 Ghost 8B Beta Released: Game-Changing Language Model

Ghost 8B Beta is a groundbreaking language model developed with a clear vision: to deliver exceptional multilingual support, superior knowledge capabilities, and all while remaining cost-effective. This model comes in two context length variations, 8k and 128k, ensuring flexibility for various tasks. Moreover, it boasts built-in multilingual functionality, making it a powerful tool for global communication and understanding.

• See detailed article: https://huggingface.co/blog/lamhieu/ghost-8b-beta-released-game-changing-language-mode
• Model card: https://huggingface.co/ghost-x/ghost-8b-beta
• Official website: https://ghost-x.org/docs/models/ghost-8b-beta

🎉 Ghost 8B Beta Released: Game-Changing Language Model

Ghost 8B Beta is a groundbreaking language model developed with a clear vision: to deliver exceptional multilingual support, superior knowledge capabilities, and all while remaining cost-effective. This model comes in two context length variations, 8k and 128k, ensuring flexibility for various tasks. Moreover, it boasts built-in multilingual functionality, making it a powerful tool for global communication and understanding.

• See detailed article: https://huggingface.co/blog/lamhieu/ghost-8b-beta-released-game-changing-language-mode
• Model card: https://huggingface.co/ghost-x/ghost-8b-beta
• Official website: https://ghost-x.org/docs/models/ghost-8b-beta

Thank you for your dedication, it sounds great. Here I would like to share some additional information and perspectives so that everyone can better understand the issues we address:

• With language models, when applying in practice we only need it to be understood at 80% or a good overview and combining with RAG will bring better accuracy. So, here we will need a good level of truth telling model and the ability to understand and work with RAG at a very good level to be most effective.
• In Italian, I'm very happy when it speaks well, it proves that my training method and source code for it were correct because it's actually live with the d0x5 version. This is all because Italian was only added later (at the same time as German), responding to the fact that sometimes it can only be described as a translation mays.
• With the ability to reason, I hope you don't misunderstand. It still works well, just when compared to some current superior models like GPT 4o or Claude 3, there will be some songs where it will "lose". It still outperforms a lot of other much larger models. For example, the question "Andrew is free from 11 am to 3 pm, Joanne is free from noon to 2 pm and then 3:30 pm to 5 pm. Hannah is available at noon for half an hour, and then 4 pm to 6 pm. What are some options for start times for a 30 minute meeting for Andrew, Hannah, and Joanne?" taken from OpenAI GPT4 home page.

One note: in reasoning tests, models often set the temperature to 0, with Ghost 8B Beta we always set it to 0.1 as the lowest. The reason is simple because if at this level the model still reasons well, then at level 0.4 (the default level of the current chat) it will still often achieve the same results, and we want to aim for practical efficiency. rather than scores. Let's try to lower the temperature with some reasoning questions to experiment.

After all, you guys are great, thank you so much everyone.

An example of reasoning about time:

An example of a long context with extensive summary capabilities: Paper: Point out the highlights and identify the ideal people to apply it..

@Dihelson It's probably because you told the model to do it again. Try telling the model to change each word. Of course, it could still be because the model misunderstood.

Try the following conversation: (1) ask to write an article -> (2) ask to translate the article into the languages ​​you want.

@AIWizard76 It hasn't gone through any real eval tests to be able to compare, but if we're just talking about ghost 8b beta, it has good translation capabilities for supported languages. It works well for translating long texts and also translating into multiple languages ​​simultaneously.

It's simple, currently the base version will not try to lengthen the text and be more "obedient". Maybe tomorrow or the next day I'll put it up for everyone to try.
Note, the current version is running everything from version "disl-0x5", the new version will improve a lot but it may not be ready right now.

thank you for your comments and encouragement 🤗
another question, how do you feel when conversing in Italian?

@danielus let me ask, is this what you want?

@danielus I noticed the explanation model because this is what the chat version (ft from ghost 8b beta, base) does for the chat task (base will not try to explain and will respect the system more strictly). The goal of answering with more information is to help users avoid having to learn more or get side answers from just one question. Of course, this can sometimes be a hassle, we'll try to balance it out.

I got the same answer from the model.

@Dihelson It supports Portuguese language, try it and let me know what you think. 🇵🇹

@Dihelson I believe you, don't worry. Please experience it happily~

A note here, the model is capable of working well with 9 major languages ​​along with function tools for the languages. It has a size that can be called a boy compared to other multilingual models (which may be lacking or inferior in things like function tools and performance).

@ZeroWw To be honest, our initial training focused more on math ability than on (abstract) reasoning. It still has just less training data, rest assured as this is a test of training recipes, expanding the training capability domains and languages ​​is just a matter of time and resources..

@Dihelson I understand, we value a model that has good reasoning capabilities and that is also our goal. In the early versions, it was focused on the immediate goals of good multilingualism, safety, functional tools support and good general performance. And it has achieved its goals. In the next stage, I will also conquer the things you said and a few other languages. Enjoy 🤗

@Dihelson @ZeroWw Thank you for pointing out the problems, in terms of reasoning we have noticed the problem. It will be improved in an upgraded version in the near future, the current version is disl-0x5 (d0x5).

I think it works well for this question, right? Have you adjusted the temperature higher?

Let's try a math problem, what do you think about this answer?