2
EV Assistant
🐨
EV Assistant using vectara-agentic
Vectara
company
AI & ML interests
retrieval augmented generation, grounded generation, large language models, LLMs, question answering, chatbot
Recent Activity
vectara's activity
ofermend
updated
5
Spaces
about 4 hours ago
ofermend
updated
a
Space
about 5 hours ago
ofermend
updated
a
Space
about 13 hours ago
joao-vectara
updated
a
Space
about 17 hours ago
stsui96
updated
a
dataset
3 days ago
Post
1546
Last year, I curated & generated a few multilingual SFT and DPO datasets by translating English SFT/DPO datasets into 9-10 languages using the
mistralai/Mistral-7B-Instruct-v0.2 model.
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
I hope it helps the community for pretraining/instruction tuning multilingual LLMs! I added a small diagram to briefly describe which datasets are added and their sources.
Happy to collaborate in either using these datasets for instruction FT, or wishes to extend translated versions of newer SFT/DPO english datasets!
nthakur/multilingual-sft-and-dpo-datasets-67eaf56fe3feca5a57cf7d74
Confusion about unlimited context length
1
#21 opened 14 days ago
by
zhm0
clefourrier
posted
an
update
29 days ago
Post
2128
Gemma3 family is out! Reading the tech report, and this section was really interesting to me from a methods/scientific fairness pov.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
nthakur
authored
a
paper
about 1 month ago
clefourrier
authored
a
paper
2 months ago
clefourrier
authored
a
paper
4 months ago
nthakur
authored
a
paper
5 months ago
nthakur
authored
a
paper
8 months ago
Post
1831
I'm excited to share our updated hallucination evaluation model (called HHEM-2.1-Open) as well as the updated leaderboard that ranks LLM by the propensity to hallucinate.
vectara/Hallucination-evaluation-leaderboard
vectara/Hallucination-evaluation-leaderboard
- 1 reply
clefourrier
authored
2
papers
10 months ago