view article Article Introducing Evaluation Cards: A Live Interpretive Layer for Understanding the AI Evaluations Ecosystem evaleval • Jun 11 • 2
Jais 3 Collection The 3rd generation of the Jais LLM: models, datasets, and supporting resources • 10 items • Updated Oct 15 • 3
Arabic-Benchmark-Auditing Collection Commonly used Arabic LLM benchmarks with defective items removed by a three-judge LLM audit, plus every discarded sample and its verdicts. • 17 items • Updated 29 days ago • 1
NeMo Gym Collection Collection of RL verifiable data for NeMo Gym • 32 items • Updated 5 days ago • 64
Jais 2: A Family of Arabic-Centric Open Large Language Models Paper • 2608.13580 • Published Jul 7 • 1
view article Article 🪢 Langfuse and 🤗 Hugging Face: 5 Ways to use them Together MJannik • Mar 14, 2025 • 14
view article Article Welcome Gemma 4: Frontier multimodal intelligence on device +5 merve, pcuenq, sergiopaniego, burtenshaw, Steveeeeeeen, alvarobartt, SaylorTwift • Apr 2 • 931
ResearchGym: Evaluating Language Model Agents on Real-World AI Research Paper • 2602.15112 • Published Feb 16 • 22
view article Article OpenEnv in Practice: Evaluating Tool-Using Agents in Real-World Environments +3 christian-washington, ajasuja, santosh-iima, lewtun, burtenshaw • Feb 12 • 38
view article Article Custom Kernels for All from Codex and Claude +2 burtenshaw, sayakpaul, ariG23498, evalstate • Feb 13 • 81
view article Article We Got Claude to Build CUDA Kernels and teach open models! +2 burtenshaw, evalstate, merve, pcuenq • Jan 28 • 158
view article Article Community Evals: Because we're done trusting black-box leaderboards over the community +5 burtenshaw, SaylorTwift, kramp, merve, davanstrien, nielsr, julien-c • Feb 4 • 91
view article Article Alyah ⭐️: Toward Robust Evaluation of Emirati Dialect Capabilities in Arabic LLMs tiiuae • Jan 27 • 26