Menu

View

JournalDigital

Theme

LightDarkSystem

Text size

AAA

The paper in your inbox every morning

Straight from the source · Glossary

The Tech Gazette

Evals

Tests that measure what a model or an AI product actually does on a set of tasks, used to compare versions and catch regressions.

Read more at platform.openai.com →

Learn it

1. LangChainOct 13, 2026Technical Guides
   How Orange Business Built Live Intelligence with LangChain and LangSmith
2. ArizeOct 9, 2026Technical Guides
   Alignment is a human-AI workflow: how Custom Views make agent review possible
3. ExpoOct 8, 2026Technical Guides
   Your skills need an evaluation mechanism
4. Stack Overflow BlogOct 7, 2026Technical Guides
   Part 2: Evals as a deployment gate — and how to know when they drift
5. ArizeOct 2, 2026Technical Guides
   Prompt caching benchmark: high cache reuse doesn’t always mean lower cost
6. ArizeOct 2, 2026Technical Guides
   Claude’s hillclimb loop for AI agents: start with production traces
7. CometSep 29, 2026Technical Guides
   Jev vs. LLM-as-a-Judge for AI Evals
8. DatadogSep 24, 2026Technical Guides
   Using TypeSafe’s Jev for evals in Datadog Agent Observability
9. PydanticSep 23, 2026Technical Guides
   High-volume AI scoring with Jev and Pydantic Evals
10. AWS Machine LearningSep 22, 2026Technical Guides
    Evaluate skill-equipped agents with Strands Evals and Amazon Bedrock AgentCore

In the news

1. LangChainOct 15, 2026News
   The Most Common Questions About Agent Evals, Answered
2. DockerOct 14, 2026News
   Harbor + Docker Sandboxes: Run Safe, Reproducible Agent Evals
3. PostmanOct 14, 2026News
   Steer Your Agents Until They Steer Themselves
4. CrewAIOct 14, 2026Releases
   CrewAI 1.15.28: Add crewai eval command to report where a stopped model broke
5. LangChainOct 13, 2026News
   Open Models have crossed a threshold
6. MuxOct 13, 2026News
   Let your agent cook by giving it the recipe
7. arXiv cs.AIOct 8, 2026Research
   Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
8. CrewAIOct 7, 2026Releases
   CrewAI 1.15.24: Add crewai eval to print a markdown brief when run by an agent
9. arXiv cs.AIOct 7, 2026Research
   LiveMACE: Process-Aware Evaluation of LLM Agent Capabilities in Evolving Markets
10. Devoxx · YouTubeOct 7, 2026Watch
    Exquisite Evals for Skills & MCPs by Christian Tzolov, James Ward
11. google-cloud-aiplatform (PyPI)Oct 7, 2026Releases
    google-cloud-aiplatform v2.4.0
12. ArizeOct 1, 2026News
    Alyx now remembers your work across sessions with long-term memory
13. Hamel HusainSep 30, 2026Columns
    Claude’s new auto eval tool
14. LangChainSep 29, 2026News
    Jev is now available in LangSmith Evals
15. ArizeSep 29, 2026News
    What everyone was talking about at WeAreDevelopers World Congress North America
16. ArizeSep 29, 2026News
    Anthropic says it fixed Claude’s writing. I ran the evals to check.
17. CrewAISep 28, 2026Releases
    1.15.23
18. METRSep 27, 2026News
    Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
19. arXiv cs.AISep 26, 2026Research
    Beyond Accuracy: Counterfactual Fragility and Demographic Bias in Clinical Evaluation of LLMs
20. arXiv cs.CLSep 24, 2026Research
    How Reproducible Are Evaluation Conclusions? A Self-Audit of LLM-Inferred Prompt Structure