vectara/hallucination_evaluation_model
Text Classification
•
0.1B
•
Updated
•
201k
•
318
AI & ML interests
retrieval augmented generation, grounded generation, large language models, LLMs, question answering, chatbot
Recent Activity
View all activity
ofermend
updated
a
model
about 3 hours ago
stsui96
published
a
dataset
3 days ago
stsui96
updated
a
dataset
3 days ago
nthakur
authored
a
paper
9 days ago
ahmed-d8k
updated
a
dataset
9 days ago
ofermend
updated
9
Spaces
about 1 month ago
HMC Demo
🐨
Ask questions about Harvard Management
6
Hacker News chat
🐨
chatbot with HN data using vectara-agentic
2
Justice Harvard
🐨
Teacher Assistant for Justice Harvard using vectara-agentic
2
SuperMicro Demo
🐨
Ask questions about Supermicro documents
UCSF Ortho Demo
🐨
Ask questions about UCSF Orthopedics
CFPB Assistant
🐨
CFPB Assistant using vectara-agentic
9
Legal Assistant
🐨
Legal Assistant using vectara-agentic
18
Finance assistant
🐨
Finance chatbot using vectara-agentic
3
EV Assistant
🐨
EV Assistant using vectara-agentic
forrest-vectara
updated
a
model
about 1 month ago
nthakur
authored
3
papers
3 months ago
FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
Paper
•
2504.13128
•
Published
•
8
Chatbot Arena Meets Nuggets: Towards Explanations and Diagnostics in the Evaluation of LLM Responses
Paper
•
2504.20006
•
Published
Fixing Data That Hurts Performance: Cascading LLMs to Relabel Hard Negatives for Robust Information Retrieval
Paper
•
2505.16967
•
Published
•
24
clefourrier
posted
an
update
3 months ago
Post
1258
Always surprised that so few people actually read the FineTasks blog, on
✨how to select training evals with the highest signal✨
If you're serious about training models without wasting compute on shitty runs, you absolutely should read it!!
An high signal eval actually tells you precisely, during training, how wel & what your model is learning, allowing you to discard the bad runs/bad samplings/...!
The blog covers in depth prompt choice, metrics, dataset, across languages/capabilities, and my fave section is "which properties should evals have"👌
(to know on your use case how to select the best evals for you)
Blog: HuggingFaceFW/blogpost-fine-tasks
✨how to select training evals with the highest signal✨
If you're serious about training models without wasting compute on shitty runs, you absolutely should read it!!
An high signal eval actually tells you precisely, during training, how wel & what your model is learning, allowing you to discard the bad runs/bad samplings/...!
The blog covers in depth prompt choice, metrics, dataset, across languages/capabilities, and my fave section is "which properties should evals have"👌
(to know on your use case how to select the best evals for you)
Blog: HuggingFaceFW/blogpost-fine-tasks
-
- 2 replies
Post
355
Excited to share open-rag-eval (https://github.com/vectara/open-rag-eval) a new open source project to help scale RAG evaluation. The key benefit: it does not require golden answers so much more scalable.
Would love any thoughts or feedback (or even better - if you want to contribute a PR that would be great).
Would love any thoughts or feedback (or even better - if you want to contribute a PR that would be great).