Learn: Evals

1. Define success and build testsAnthropic · Deciding what good looks like before you measure it.Mark done
2. EvaluationsOpenAI · Running an eval against your own prompts and data.Mark done
3. The LLM evaluation guidebookHugging Face · Benchmarks, judges and their pitfalls, explained.Mark done
4. A statistical approach to model evaluationsAnthropic · Error bars: telling a real gain from noise.Mark done
5. InspectUK AI Security Institute · An open framework for writing and running evals.Mark done

• Evals
• Benchmark
• Hallucination
• Guardrails
• Red teaming
• AI safety

1. MindEval2: Clinically Grounded Patients and Rubrics for Evaluating Language Models in Mental Health Support
2. AI Consilium: a Benchmark of LLM Clinical Multidisciplinary Second Opinions
3. Beyond Downstream QA: Rethinking How We Evaluate Agent Memory
4. PremiseSQL: Evaluating Premise Critique and Abstention in Text-to-SQL Systems
5. QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery

More in Language models →