Announcements from the labs, companies and platforms themselves.
Agents built with TanStack AI can now call OAuth-protected MCP servers through Vercel Connect, with no credentials for you to store or rotate. The new @vercel/connect/tanstack-ai subpath exports connectMCPTransport, which takes a TanStack transport config and attaches a Connect-b…
Safari Technology Preview Release 253 is now available for download for macOS Golden Gate and macOS Tahoe.
GitHub Copilot code review now offers additional personal configurations to an expanded set of Copilot plans and an enterprise-level default setting. These improvements are now generally available: A dedicated personal… The post More ways to request and configure Copilot co…
This is the final notification that Node 20 is no longer available on GitHub Actions runners. Runners now use Node 24 for JavaScript actions. The temporary ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION opt-out is no… The post Node 20 is no longer available in GitHub Actions appea…
Starting September 23, 2026, Microsoft is updating the author-signing certificate used for NuGet packages. Customers using trusted signer policies or certificate fingerprint verification should add the new certificate as soon as possible. The post Microsoft is updating its author…
You can now create as many Vercel Blob stores as you need. The previous limits of 100 stores on Hobby, 500 on Pro, and 1,000 on Enterprise no longer apply. Blob store creation is now billed alongside other Blob Advanced Operations, including put(), copy(), and list() calls. On Pr…
Next.js is preparing a scheduled September security release for September 30, 2026.
Learn what is new in Visual Studio Code 1.139 Read the full article
Local sandboxing helps reduce the potential impact of unintended commands by limiting access to files, network resources, and credentials on your machine. In the GitHub Copilot app, you configure it… The post Local sandboxing in the GitHub Copilot app appeared first on The…
What is Small Talk? Small Talk is our short Q&A with founders from the JetBrains Startup Program. They answer a handful of questions in their own words about what they’re building, why they started, and what they’ve figured out along the way. No pitch, no polish –…
New research from GitHub and Yale Program on Climate Change Communication finds strong demand for tools, measurement, and practical guidance that can help developers reduce wasted compute. The post Developers want more efficient software. Here’s what over 1000 GitHub users told u…
100 Exercises to Learn Rust is our adaptation of Mainmatter’s course of the same name, written by Luca Palmieri, Principal Engineering Consultant at Mainmatter, and it has just received its biggest update since we released it a year ago. Palmieri has been writing Rust since…
The Terraform provider for Google Cloud 8.0 builds on expanded infrastructure discovery workflows, modernizes provider defaults, removes support for retired Google Cloud services, and improves consistency between Terraform configurations and Google Cloud APIs.
Understand how Copilot agents perform and interact with models and tools. The GitHub Copilot app now supports OpenTelemetry (OTel) configuration through enterprise-managed settings. OTel is an open source observability framework.… The post OpenTelemetry in the GitHub Copilo…
GitHub Copilot for JetBrains 1.18.0 brings AI-assisted tool approvals, more control over agent conversations, and shared skills and instructions for your organization. You can also review plans with the Codex… The post New features and improvements in Copilot for JetBrains…
Drives for Vercel Sandbox are now available in public beta on Hobby, Pro, and Enterprise. A Drive is persistent storage that you mount as a directory in a Vercel Sandbox. It isn’t tied to a single sandbox, so you can reuse the same Drive across runs and different sandbox instance…
Two bots for the last mile of shipping code: Rollouts watches every change as it deploys, and Security Review reports exploitable bugs on every pull request.
This Heads-Up is part of the regular communication sent to the projects involved; it covers a new JavaDoc tag `@note` to highlight the presence of additional information in API documentation.
C++ code intelligence in GitHub Copilot CLI is now faster with support for whole codebase indexing. C++ repositories can contain millions of lines of code across deeply connected source files… The post Faster C++ code intelligence with whole codebase indexing appeared first…
Learn how Amazon CloudWatch Omni delivers AI-powered observability purpose-built for generative AI and agentic workloads. Trace, evaluate, and experiment with AI agents across any framework—directly from your IDE or a standalone web experience—using open standards and built-in ev…
The September 22, 2026 out-of-band security update for Next.js is now available
Posted by Fahd Imtiaz, Senior Product Manager, and Loryn Hairston, Product Marketing Manager, Android Developer Googlebook introduces a new category of laptops built on a shared Android foundation. High-performance hardware from partners such as HP, Dell, Lenovo, Acer, and Asus,…
AI coworkers and coding agents are spreading fast across organizations, and each...
We ran a survey asking users of OpenTelemetry and Prometheus how they collect, process, and store metrics. The goal was to understand, with real usage data rather than assumptions, how far the ecosystem has moved and whether the interoperability still causes friction. Key takeawa…
Following a successful private preview, we’re thrilled to open DigitalOcean Managed Agents to everyone. Teams can now deploy their preferred agent harness (like OpenCode, Codex CLI) or bring their own, connect agents to 16,000+ tools, and help agents do more work at scale without…
Next.js 16.3.6 and 15.5.26 are planned for a critical out-of-band security update on September 22, 2026.
We’re removing several SSH algorithms, adding a new algorithm, and requiring larger RSA SSH keys to improve security. The changes are as follows: We’re removing the ability to use RSA… The post Security improvements for SSH appeared first on The GitHub Blog.
NVIDIA DLSS 5 introduces DLSS 3D-Guided Neural Rendering and granular controls that help game developers add lifelike lighting and material detail while...
Worker Previews gives every branch its own URL, configuration, state, and observability, so you and your agents can test changes in parallel without affecting production.
To build and deploy sophisticated robotics applications that can perceive, reason and act in dynamic environments, developers need new physical AI models and tools. The ROS open framework is a project from Open Robotics that helps humans build robots. NVIDIA Isaac ROS 5.0 — a col…
We've launched Claude Opus 5.5 (claude-opus-5-5), a model for long-running agentic coding and knowledge work. It has a 1M token context window by default, 128k max output tokens, and always-on adaptive thinking, at $4 / $20 USD per MTok (Claude Opus 5 is $5 / $25). Claude Opus 5.…
Also reported by Vercel · Netlify · What's New with AWS · What's New with AWS · Microsoft Azure · GitHub Changelog · AWS Machine Learning
Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS generally available (GA): Released our next-generation text-to-speech (TTS) audio models and the Gemini API Voices endpoint (/v1beta/voices): Gemini 3.8 Flash TTS (gemini-3.8-flash-tts): Flagship creative TTS model engineered for…
More on Gemini 3.8 · Gemini
A file in a bucket is just bytes; when you upload it, there is often a job to do next with that file, and that job usually involves Postgres - a `files` row, a status, a thumbnail key. That is a perfect Neon Functions job; the missing piece was something to start the Function whe…
Starting with CodeQL CLI 2.27.0, the all-platform CodeQL bundle (i.e., codeql-bundle.tar.gz and codeql-bundle.tar.zst), which includes the binaries for all supported platforms up to this release, is marked as deprecated. In… The post Deprecation notice: All-platform CodeQL…
AI can produce code. Organizations still have to produce software. Agentic development is changing how software gets made, but it hasn’t changed what it costs to be wrong. Six months ago, we began publicly experimenting with agentic development environments. Around the same time,…
BigQuery Feature You can now publish a BigQuery data agent in Gemini Enterprise by registering the agent with Agent Registry and importing it using default Google-managed credentials. When BigQuery and Gemini Enterprise are in the same Google Cloud project and configured with a m…
The Next.js team has disclosed a critical severity vulnerability in an upstream dependency that can lead to remote code execution when ImageResponse renders untrusted input. It is patched in 15.5.26 and 16.3.6. Applications that do not pass untrusted input into ImageResponse are…
This is a major release, and the headline feature is our new AI Assistant. We're starting to roll out AI capabilities across our database tools, and SQL Manager for PostgreSQL is the first to get them. It's also available in SQL Management Studio for PostgreSQL, which ships with…
More on PostgreSQL 7.0 · PostgreSQL
Application code has fast testing loops: runners, fixtures, and red-green feedback in JavaScript, TypeScript, and Python. Postgres can be tested too, but database logic often sits outside those loops. Developers have to provision state, manage transactions, or fall back to a past…
PostgresCompare is pleased to announce the release of version 2.2.0, the first public release since 1.2.2 and the largest change in the product's history. The application has been rebuilt on a new foundation, and the 2.x series adds database pipelines, ad-hoc comparison, data-cha…
More on PostgresCompare 2.2.0 · PostgresCompare
pnpm 12.6 ships with automatic dependency deduplication, relocatable
Ground your coding agent in official documentation with one REST request. Includes a Vercel AI SDK example that uses Context7 Search as a tool.
dbt v2, which runs on the new Rust-based Fusion engine, is the first dbt release that ships with a built-in DuckDB adapter. This post covers setup, DuckLake and Iceberg catalogs, querying dbt's Parquet metadata with DuckDB, plus other v2 features that matter to DuckDB users, incl…
Modern applications rarely rely on one database. Data is often distributed across regions, business units, shards, and different technology stacks. Bringing that data together in real time should not require users to build and operate a separate integ...
At the end of August, we announced our first Maintainers in Residence, Rust Project contributors who are funded for their upstream contributions and maintenance work from the Rust Foundation Maintainers Fund (RFMF). Since then, the Rust Leadership Council has dedicated more funds…
The Helidon team is pleased to announce Helidon 27, the first release under the Tip-and-Tail model used by OpenJDK.
More on Helidon 27 · Helidon
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
xAI's Grok 4.6 is now available in Amazon Bedrock: a frontier model for long-running agents, coding, and knowledge work, with a 500K token context window and four reasoning effort levels. It runs on both the bedrock-mantle and bedrock-runtime endpoints, with Converse API and cros…
Use Jev as a judge for LangSmith evals to evaluate agent traces with faster, cheaper structured feedback across production runs, datasets, and regression tests.
Vercel Connect now includes a managed connector for Microsoft Teams. Creating one gives your organization a Teams bot that your apps and agents run. People can @mention it in channels or message it directly, and your code receives the message and replies as the bot. As a Vercel M…
The OpenTelemetry project is excited to announce the 2026 OpenTelemetry Governance Committee (GC) election. Nominations are due by 16 October 2026 23:59 AoE. The list of eligible candidates will be shared on 19 October 2026. Voting will take place between 26 October 2026 12:00 UT…
Build a commit once and deploy the same immutable artifact across multiple Render services and environments. Join the Build Reuse Private Beta to reduce redundant build time, cost, and environment drift.
Living in the Netherlands, I spend a fair amount of time on trains, and that is usually where I catch up on what the builder community is writing. Until now, that meant opening a laptop or squinting at a browser tab on my phone. This week I found myself scrolling through trending…
More on Corretto 27 · Corretto
Posted by Jan Kleinert, Developer Relations Engineer, Android for Cars Today, the games category for Android Auto and cars powered by Android Automotive OS with Google built-in is officially graduating from beta to general availability. Our early access partners have already been…
We’re open-sourcing Rebalancer, the assignment-problem solver that has been used to solve resource allocation problems throughout Meta for over nine years. Rebalancer separates several related concerns: how to specify an assignment problem, how to store it efficiently in me…
A resilient software supply chain is the foundation of modern delivery, and securing your continuous integration and continuous delivery (CI/CD) pipeline is what keeps innovation moving safely. Notable supply chain attacks more than doubled in the first half of 2026 compared to t…
Deployments now show their billable duration and CPU minutes in the dashboard, vc inspect, and the REST API. Use it to understand how each build contributes to your usage. Billable duration is the build and post-build time combined, rounded up to the next whole minute. CPU minute…
Learn about the newly launched version of the Dropbox API documentation and its modern design and features.
Python Workers allow developers to run Python web frameworks and AI orchestration libraries natively in the Cloudflare Workers runtime. You can seamlessly integrate with Cloudflare's ecosystem including D1, R2, and Workers AI without writing any JavaScript glue code.
During beta, each Function was reachable only at its Neon invocation URL, something like `https://br-cool-forest-a1b2c3d4-api.compute.c-2.us-east-2.aws.neon.tech`. Now, we support custom domains - you can put it behind `api.example.com` instead.
During the beta phase, the only way to run a Neon Function was to send it an HTTP request. That works well for jobs triggered by your app, but not so much for backend jobs. If you wanted to pull an external API into Postgres every 15 minutes, you needed an external scheduler. Als…
Access Context Manager Feature Access Context Manager supports extended session length for Workforce Identity Federation. This feature is in Preview for Looker (Google Cloud core) customers. For more information, see Configure extended session length for Workforce Identity Federa…
18 models, 113 coding tasks. The best single model gets 74.1% at $6.52. A perfect router gets 97.6% at $1.88. See how FireRouter closes the gap.
MiMo V2.6 Pro, MiMo V2.6 Flash, and MiMo V2.6 Pro UltraSpeed from Xiaomi are now available on AI Gateway. MiMo V2.6 combines coding, reasoning, and tool use with native text, image, audio, and video understanding. Its 1M token context supports long repositories, tool traces, and…
You can now call Jev from TypeSafe AI through AI Gateway using an existing TypeSafe client or the HTTP API, in addition to the AI SDK. TypeSafe client: Point an existing TypeSafe client at AI Gateway without changing its evaluation calls. HTTP API: Call Jev directly from any lang…
Grok 4.7 from SpaceXAI is now available on AI Gateway and 40% off through September 27. The discount applies automatically when you call spacexai/grok-4.7. Grok 4.7 has a 500K token context window and supports low, medium, high, and xhigh reasoning levels, giving you control over…
What's left for us to build?
Upstash Redis now supports the Array type from Redis 8.8. Here is why it was needed, how it differs from lists, the new use cases it unlocks, and when to use each one.
The Rust Security Response Team was notified that Miri stores all environment variables to target/, allowing secrets to persist in caches. While not necessary a vulnerability in and of itself, when paired with GitHub Actions caching behavior, it is possible for this to expose sec…
The following JEP is targeted to JDK 28: 401: Value Objects (Preview)
mcp-handler now has experimental support for WebMCP, the proposed web standard for exposing tools to in-browser agents. Add a single script tag to your site, and your existing MCP tools become available there too. Opt tools in by adding them to the experimental_webMcp object: The…
v0 now installs private packages from npm and custom registries using credentials stored as shared environment variables on Vercel. This makes it easier for teams to build with their existing design systems, component libraries, and internal packages directly in v0. To get starte…
Kimi K3 from Moonshot AI is now available on Amazon Bedrock, giving you a powerful new open-weight option for coding and knowledge work. It offers native vision, a 1-million-token context window, and explicit prompt caching to reduce latency and input costs.
Introducing composable, module system native and agent friendly command line tools for modern Java development By Danny Thomas, JVM Ecosystem Team Recent work on the Java language to pave the on-ramp has made it easier than ever to start a Java program and evolve it using the ful…
AI is accelerating software development at an unprecedented pace. But as code generation scales, so do the challenges of securing the code, especially emerging AI-based vulnerability exploitations. To meet these challenges, the Google AI and Infrastructure team is transforming ho…
Today we are announcing the new AgentCore runtime, a capability of Amazon Bedrock AgentCore built for the speed, flexibility, and cost efficiency that production agents demand. It reclaims memory as sessions release it and delivers consistent cold starts regardless of image size…
Amazon Bedrock continues to expand its open weight model portfolio with the same security and governance that customers rely on. Today, Kimi K3 from Moonshot AI is generally available on Amazon Bedrock, giving you a powerful new option for coding and knowledge work. Accordin…
We dive into these questions and other AI hot takes on the latest episode of the GitHub Podcast. The post Should you read the code, is RAG dead, and did Skills kill MCP? appeared first on The GitHub Blog.
We are living in genuinely interesting times. AI is disrupting software development at a pace where new models, tools, and practices appear almost daily. Many teams’ natural first instinct is to spend ever more time chasing updates. After almost two years of AI product and market…
We want local coding agents to be smart and fast, with the ability to understand a codebase, do useful work, and finish tasks without long waits. This Junie Local update makes it practical to use a more capable model on your own machine. In the first release, we had to choose bet…
Gemini 2.5 models access update: To ensure reliable performance for everyone, we are limiting access to the 2.5 models to users who have actively used them in the past. These models are not deprecated and will continue to be served until further notice through the API. For any ne…
More on Gemini 2.5 · Gemini
You have the change ready and the tests are green. Now someone has to launch the app, find the right screen, and check the flow. Often, that someone is still you, even when an agent helped write the code. You should be able to delegate that part too. Junie /demo is a new mode in…
Ktor 3.6.0 is here! This release is full of new experimental features, including typed authentication capabilities with specialized support for OpenID Connect and HTTP/3 support for the Netty engine. There are also a few quality-of-life improvements for routing and request handli…
More on Ktor 3.6.0 · Ktor
Notes from three days in the Netherlands, featuring a lightning talk on pg_clickhouse and pg_stat_ch at PGDay Lowlands and a session on PostgreSQL 19 monitoring at Percona Live Amsterdam.
Agent Platform Workbench Change 20260918-2230-rc0 Release Change 20260918-2230-rc0 Release Change Installed latest packages from upstream dependencies. Change Installed latest packages from upstream dependencies. Feature JupyterLab now forwards client-side logs (console errors, u…
GLM 5.3 FlashX is now available on AI Gateway. GLM 5.3 FlashX is a high-speed serving option for Z.ai's multimodal coding model, delivering inference at ~200 tokens per second for faster streamed responses. The higher serving speed is useful for coding agents, tool loops, and int…
More on GitLab 19.4 · GitLab
pnpm 12.5 expands Python support with editable project packages, shared workspace
You and your agents can now deploy static artifacts to Vercel in under one second through Vercel CLI. Run vercel deploy to share a prototype, publish an HTML report, or preview a page created by your coding agent. Vercel automatically detects eligible deployments, and valid artif…
You can now opt into Turbo build machines on any individual deployment. This is useful when you need to increase resources temporarily without changing project settings. You can do this in three ways: Include #VERCEL_BUILD_MACHINE=TURBO in your Git commit message before pushing t…
You can now run Harbor evals on Vercel Sandbox. Harbor is the open-source harness behind Terminal-Bench, whose registry includes many other benchmarks such as SWE-bench, tau3-bench and OSWorld. Pass --env vercel to harbor run and each trial executes in its own isolated Firecracke…
Posted by Maunik Shah, Staff Software Engineer, Alec Garcia, Software Engineer, and Joseph Yong, Technical Program Manager At Android, we are constantly working to provide developers and enterprise partners with the data they need to keep devices protected. Today, we're thrilled…
skills@1.7.0 adds Notion skills databases as an install source for agent skills. Notion skills are reusable agent skills written as Notion pages. Teams author, review, and update them in the workspace they already use, then install them into any agent the skills CLI supports. No…
Deep Life Sci is LangChain's open source agentic assistant for clinical and lab scientists. It pulls from 600K+ ClinicalTrials.gov studies, 29M PubMed abstracts, and 12M PubMed Central full-text articles, with sandboxed sub-agents for real data analysis.
An agent that needs the outside worldAn engineer at a software company is building...
Teams can now browse HashiCorp-managed pre-written policies, add them to a policy set, and apply common compliance guardrails directly in HCP Terraform.
HCP Vagrant will be deprecated in 2026. Users should begin planning migrations as box creation, support, and operations are phased out.
Most published quantizers are built from the same small set of primitives. VQ-bench is an open-source library of those primitives, plus a reproducible benchmark of 14 quantizers across VIBE datasets.
Posted by Matthew McCullough, VP, Product Management, Android Developer When we first launched Android Bench, we built a rigorous foundation for evaluating how large language models (LLMs) assist developers with real-world Android tasks. As AI models and agents rapidly evolve, we…
Antigravity Agent 09-2026: Released antigravity-preview-09-2026, which replaces and deprecates antigravity-preview-05-2026. If you run on a remote sandbox (environment: "remote") and read only output_text or model_output steps, update the agent string and nothing else changes. If…
Neon is now a complete suite of backend primitives built around the database and rooted on the lakebase architecture: Lakebase Postgres, Object Storage, Functions, Managed Better Auth, and AI Gateway. All tools are GA and ready for production. Tell your agent to deploy them.
Our Safari release notes have never been as long as they are for this version.
More on Safari 27.0 · Safari
Swift 6.4 is now available. Swift aims to be a great choice across the stack, from apps and servers to systems code, embedded devices, and the browser. This release deepens that support, and makes everyday code easier to write. Highlights include: Swift Build is now the default i…
When your Swift program hits a breakpoint and stops so you can inspect it, the debugger’s expression evaluator has to find the exact Swift module your code was built from. Until now, that lookup wasn’t always precise. The upcoming Swift 6.4 release will include changes, begun in…
Coding agents changed what it costs to build mobile apps twice. Here’s why Shopify is moving from React Native back to Swift and Kotlin.
We're joining Shopify to give Tailwind a stable long-term home where it will be actively maintained for the millions of people who depend on it.
The Kotlin 2.4.20 release is out! Here are the main highlights: For the complete list of changes, see What’s new in Kotlin 2.4.20 or the release notes on GitHub. How to install Kotlin 2.4.20 The latest version of Kotlin is included in the latest versions of IntelliJ IDEA an…
More on Kotlin 2.4.20 · Kotlin
Welcome to “What’s new in Swift,” a curated digest of releases, videos, and discussions in the Swift project and community. Here’s an update from guest contributor Simon Leeb on Swift’s progress as a language for web scenarios: Hi, Simon here! I am the creator of the elementary-s…
Kotlin Toolchain 0.12.0 is out. This release brings some long-awaited features: multiplatform libraries publication, a preview of Wasm application support, Compose Hot Reload from the command line, and more. Read on for the details, and check the release notes for the full…
This month, Svelte 5.57 shipped with new SvelteMap methods and a few quality-of-life additions while SvelteKit 3 got closer to the finish line with its Release Candidate. The sv CLI also got a new ai-tools add-on that replaces the old mcp one, and sv@next now ships a task-based s…
Remix 3 RC is available now. A full-stack JavaScript framework built on web primitives and shipped as a single dependency.
Compose Multiplatform 1.12.0 is out! This version brings new tooling for AI assistants, improvements to web resource management, and finer control over desktop window states. Here are the highlights of this release: For a complete overview of the changes, check out What’s new in…
Tutorials, case studies and technical deep dives.
Stable versions of the tools and libraries we track. Prereleases are left out.
| Project | Version | Date |
|---|---|---|
| Ollama | v0.34.4 | Sep 23 |
| ai | 7.0.113 | Sep 23 |
| llama.cpp | b11149 | Sep 23 |
| llama.cpp | v0.5.0 | Sep 23 |
| Kubernetes | v1.34.12 | Sep 23 |
| Kubernetes | v1.35.9 | Sep 23 |
| Claude Code | v2.1.281 | Sep 23 |
| Kubernetes | v1.37.1 | Sep 23 |
| llama.cpp | b11147 | Sep 23 |
| Kubernetes | v1.36.5 | Sep 23 |
| Node.js | 2026-09-23, Version 22.23.3 'Jod' (LTS), @aduh95 prepared by @juanarbol | Sep 23 |
| LangChain | langchain-anthropic==1.7.4 | Sep 23 |
| @openrouter/sdk | 1.3.21 | Sep 23 |
| @modelcontextprotocol/sdk | 1.30.1 | Sep 23 |
| Zed | v1.21.0 | Sep 23 |
| LangChain | langchain-openai==1.6.5 | Sep 23 |
| Zed | collab-staging: ui: Fix context menu selecting first entry on open (#64365) | Sep 23 |
| uv | 0.12.18 | Sep 23 |
| Terraform | v1.16.4 | Sep 23 |
| openai | 7.23.0 | Sep 23 |
| OpenAI Codex | 0.156.1 | Sep 22 |
| LangChain | langchain-openai==1.6.4 | Sep 22 |
| LangChain | langchain-anthropic==1.7.3 | Sep 22 |
| Ollama | v0.34.3 | Sep 22 |
| Next.js | v15.5.26 | Sep 22 |
| Next.js | v16.3.6 | Sep 22 |
| @anthropic-ai/sdk | 0.128.0 | Sep 22 |
| Claude Code | v2.1.280 | Sep 22 |
| next | 16.3.6 | Sep 22 |
| Node.js | 2026-09-22, Version 26.10.0 (Current), @aduh95 | Sep 22 |
| vLLM | v0.30.0 | Sep 22 |
| @google/genai | 2.24.0 | Sep 22 |
| LangChain | langchain-fireworks==1.6.2 | Sep 22 |
| LangChain | langchain-deepseek==1.1.1 | Sep 22 |
| LangChain | langchain-openrouter==0.2.9 | Sep 22 |
| langchain | 1.5.12 | Sep 21 |
| LangChain | langchain-openai==1.6.3 | Sep 21 |
| LangChain | langchain-core==1.6.4 | Sep 21 |
| Claude Code | v2.1.278 | Sep 19 |
| uv | 0.12.17 | Sep 18 |
| Claude Code | v2.1.277 | Sep 18 |
| uv | 0.12.15 | Sep 17 |
| uv | 0.12.14 | Sep 17 |
| Claude Code | v2.1.276 | Sep 17 |
| uv | 0.12.16 | Sep 17 |
| Ollama | v0.34.2 | Sep 17 |
| Claude Code | v2.1.275 | Sep 17 |
| Zed | html-v0.3.2: html: Bump to v0.3.2 (#64371) | Sep 17 |
| Zed | glsl-v0.2.5: glsl: Bump to v0.2.5 (#64370) | Sep 17 |
| Zed | proto-v0.3.4: proto: Bump to v0.3.4 (#64372) | Sep 17 |
| Deno | v2.9.7 | Sep 17 |
| Zed | v1.20.2 | Sep 17 |
| vLLM | proto-v0.3.0 | Sep 17 |
New repositories under the GitHub topics we track, and packages published to npm.
Open-source coding-agent harness you can actually change — own the loop (prompts, gates, routing, skills, terminal states), use any model, run long tasks while you're away.
Coherence infrastructure for self-evolving AI agents — on the Claude Code or Codex subscription you already have.
The email inbox API for AI agents. Send, receive, reply, and manage threaded email conversations programmatically.
Reusable model provider adapters and streaming runtime from OpenClaw
The **Model Context Protocol (MCP) client** for the [AI SDK](https://ai-sdk.dev/docs) lets you connect to MCP servers and use their tools with AI SDK functions like `generateText` and `streamText`.
Plannotator Pi extension - interactive plan review with annotations, annotate agent messages, and review code/PRs
Filesystem-first framework for durable backend AI agents that run anywhere.
🧹 Free, open-source macOS disk cleanup CLI. Scan & safely remove caches, logs, Xcode DerivedData, npm/Homebrew/pip junk and stale node_modules from your terminal. Trash-first, zero telemetry the terminal-native CleanMyMac alternative.
Cloud and zero-trust agentic workflow marketplace for skills, agents, rules, MCP references, and compliance-aware architecture.
把十二种 IM 渠道和公网 AI Office 接入本机 DeepSeek Harness。 Connect twelve IM channels and a public AI Office to a local DeepSeek Harness.
Self-learning vector database for Node.js — hybrid search, Graph RAG, FlashAttention-3, HNSW, 50+ attention mechanisms
Model Context Protocol implementation for TypeScript - Client package
Model Context Protocol implementation for TypeScript - Server package
Model Context Protocol implementation for TypeScript - Node.js middleware
Express adapters for the Model Context Protocol TypeScript server SDK - Express middleware
PostHog SDK for Model Context Protocol (MCP) servers — tracks tool usage, intent, and identity
GitLab MCP server for projects, merge requests, issues, pipelines, wiki, releases, and more
Lightweight high-performance code editor built with Rust
Modal code editor with LSP support written in Rust
Desktop companion for AI coding agents and Codex workflows
GitLab Duo provider for Vercel AI SDK
Mastra is a framework for building AI-powered applications and agents with a modern TypeScript stack.
cli for mastra
MCP (Model Context Protocol) adapter extension for Pi coding agent
Local Qwen-Image-2.1 studio for Apple Silicon: text-to-image, multi-image editing and prompt enhancing, with a CLI and a helmstudio web UI.
Headless browser automation server and OpenClaw plugin for AI agents - anti-detection, element refs, and session isolation
Mock infrastructure for AI application testing — point your SDK at one local port and every provider, protocol, and service answers deterministically.
Mobile app automation and verification for AI coding agents. CLI, MCP server, and typed Node.js API for iOS, Android, HarmonyOS, TV, web, macOS, and Linux.
Markdown and HTML renderer for Svelte 5 — built for rendering streaming AI agent output from Claude Code, ChatGPT, and agentic workflows. XSS-safe defaults, streaming-aware sanitization, token caching, TypeScript types, and Svelte 5 runes.
805 verified examples of Jev — TypeSafe AI's System One decision model — indexed by the decision each one makes, not the blog that mentioned it. Every cited call site is re-read by CI each week. Bilingual EN/中文, JSON schema, and a cross-platform compatibility table.
Different models, the same brief - a collection of demos built from shared prompts, with source, screenshots, and notes.
微信（Windows 4.x）旁挂的回复辅助：窗口截图 + 本地离线 OCR 读对方消息 → Jev 判断意图 → 3 条候选一键填入，发送永远手动
微信消息意图识别悬浮窗（macOS）：看屏 + 本地小模型判断意图和风险，再按话术生成回复候选。纯只读、不注入微信。
装在手机上的对话副驾：在微信 / QQ / X / 飞书里读懂对方、给出候选回复、一键填入输入框，发不发由你。非侵入，只读屏幕，不 hook 不改包。
Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating)
AI video prompt cheat sheet & Claude Skill: cinematic camera angles, camera movement, lighting, composition, color grading for Veo 3, Kling, Sora, Runway, Midjourney. 700+ terms with Vietnamese explanations.
Specialised AI models for logo design — a brand-analysis model turns a business into constraints, typography and symbol models construct the mark, and a composition engine produces real lockups and clear-space rules. Early access open.
660+ muapi-hosted generative-media models plus community-submitted third-party API tools (SEO, enrichment, social, scraping) — one YAML file per entry, browsable by capability.
Jev DSH 决策引擎｜面向 Agent Harness 的结构化决策插件。原生支持 DeepSeek Harness，通过 iPolloWork 支持 OpenCode、Codex Harness。
Jev-powered semantic code search for coding agents — find behavior across repositories via CLI or MCP, with exact source excerpts and line numbers.
Awesome list of TypeSafe AI Jev use cases: 74 demos ranked by likes, 150+ GitHub repos, limits, cost and API examples. CC0, sponsored by AY Automate.
A curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions.
Generate realistic teacher documents (ID cards, licenses, letters) for 13 countries via MCP.
A curated list of Jev use cases, projects, SDKs, and resources. Jev is TypeSafe AI's System One model for fast, typed decisions in software — Choice, Score, and Noul with calibrated probabilities.
A curated, source-backed list of projects built with Jev, TypeSafe AI's System One model for typed decisions.
An agent skill that finds where a photo was taken — OpenStreetMap geometry, elevation skylines, satellite imagery and street view — and shows its work. Works with Claude Code, Codex, Cursor, Gemini CLI, OpenCode and GitHub Copilot.
中国创作者给全球 AI 出的一场真实大考 | A searchable index of Bilibili AI Infinite Arena videos.
Awesome Jev: source-backed open-source ecosystem radar, plain-language project discovery, and automatic GitHub sync
MCP server for Divar — search, compare and analyze classified ads. Read-only, no API key.
Jev-compatible API endpoint based on open models (prefill-only)
Evidence-driven autonomous SOC team on AgentTeams: hash-bound approvals, HMAC audit chain, rollback and reproducible attack scenarios — auditable governance primitives for agent infrastructure.
A curated list of public projects, integrations, and discussions built on Jev — TypeSafe AI's System One model for typed decisions.
Local AI music studio unifying ACE-Step 1.5 and YuE2-3B in one Vue interface — text-to-music generation, stem separation, MIDI transcription, and LoRA fine-tuning, with a built-in multitrack DAW.
Awesome Jev: a source-backed field guide to TypeSafe's System One model, with SDKs, live demos, agent tools, and independent evaluations.
Astra plans and reviews; DeepSeek Flash builds. A native Codex workflow with phased tasks, verification, safe installation and reversible setup.
Local-first MCP plugin for continuous software-quality review by AI coding agents, powered by Jev.
An mcp connector to evaluate anything fast and cheap. Give your AI agent direct access to typesafe ai's jev model and open weight models like laya
An idiomatic, type-safe Elixir port of the official TypeScript AI SDK (ai / ai-sdk) providing unified LLM integrations, streaming text and structured outputs, tool calling, and agentic workflows. Jev is their current flagship model and is the first System One model.
Agents built with TanStack AI can now call OAuth-protected MCP servers through Vercel Connect, with no credentials for you to store or rotate.
The new @vercel/connect/tanstack-ai subpath exports connectMCPTransport, which takes a TanStack transport config and attaches a Connect-backed auth provider. The provider is called before every MCP request, so the token is always fresh.
If the user has not granted access, createMCPClient fails with a consent challenge before the model runs. Catch it with getConsentChallenge and redirect to Connect's consent URL. Otherwise, a consent error raised within a tool call would reach the model as an error string rather than the user as a redirect.
Read the TanStack AI integration docs to get started, or see the full list of supported frameworks and adapters for Vercel Connect.
Safari Technology Preview Release 253 is now available for download for macOS Golden Gate and macOS Tahoe. If you already have Safari Technology Preview installed, you can update it in System Settings under General → Software Update.
This release includes WebKit changes between: 320113@main…321067@main.
aria-keyshortcuts values containing Meta or Alt literally instead of using the macOS terminology Command and Option. (320424@main) (186342070)timeline-scope property. (320183@main) (186261285)timeline-scope instead of one established by an ancestor, which now takes priority. (320223@main) (186265325)timeline-scope hierarchy could remain active instead of yielding an inactive timeline. (320227@main) (186330965)timeline-scope value did not update timelines for animations outside of its hierarchy. (320231@main) (186331685)ViewTimeline constructor did not require a subject parameter. (320969@main) (187200265)CSSContainerRule.conditions. (320762@main) (182257864)safe and unsafe keywords with normal alignment, and change safe behavior for absolutely (and fixed positioned) boxes to keep the box within their original containing block (the viewport). (185952139)sibling-index() and sibling-count() within container query conditions. (320493@main) (186564084)background-size value collapsed the image tile to nothing. (320169@main) (140387662)[class~=foo] attribute selector did not perform as well as an equivalent class selector. (320862@main) (164128575)::selection, did not inherit as a StyleColor, which could prevent values like color-mix(in oklab, teal 50%, currentcolor) from resolving correctly. (320228@main) (184495338)Range with an already-registered highlight never painted. (320355@main) (185173794)box-shadow with a large spread on a wrapped inline element painted outside the element as full-width bands. (320150@main) (185651318)CSS.highlights iterated in hash order instead of registration order after its wrapper was garbage collected. (320343@main) (185754041)corner-shape rendered incorrectly when inner corners intersected. (320287@main) (185931169)aspect-ratio could keep a stale inline size contribution when the grid container shrank. (320117@main) (186101273)vertical-rl writing mode. (320167@main) (186274088)<details> element made its <summary> one pixel shorter. (320461@main) (186413478)attr() stopped invalidating on attribute changes after a view transition. (320420@main) (186467515)unicode-bidi and direction had no effect on an inside ::marker. (320504@main) (186468752)fit-tolerance did not interpolate between <length-percentage> values. (320383@main) (186509338)inline-block element that had no line-height of its own. (320514@main) (186614663)<length-percentage> from a calc() value to a pure <length> dropped the percentage component at 100% progress. (320585@main) (186628152)text-decoration-thickness and text-underline-offset did not preserve percentage values when interpolating. (320584@main) (186628810)align-content left the list marker behind. (320532@main) (186676835)text-indent moved an outside list marker along with the indented text. (320533@main) (186678552)tab-size was small in a proportional font. (320671@main) (186698040)content property painted blank. (320620@main) (186742805)corner-shape with thick borders rendered incorrectly. (320826@main) (186808871)fit-content() grid tracks could be stretched beyond their argument instead of capping the track’s growth limit. (320780@main) (186961437)<caption>‘s margins were missing from the table. (321056@main) (187137359)<caption> ignored its margin against the table edge. (321060@main) (187139068)right and bottom values of an out-of-flow positioned element were wrong inside a vertical inline containing block. (321061@main) (187295559)w component when mapping a 4-component point. (321030@main) (187297446)<canvas> element with no pushed OffscreenCanvas frame would fail instead of returning a transparent black image. (320202@main) (186229685)drawImage prevented its backing surface from being recycled. (320735@main) (186731221)<canvas> computed path bounds unnecessarily when the whole backing store was already marked dirty. (320709@main) (186946552)<select> could render outside the viewport, by applying safe alignment to keep it within its original containing block. (320564@main) (185952139)load event could fail to fire if a readystatechange handler triggered a new load during page completion. (320331@main) (186373617)BigInt values in Intl.PluralRules.prototype.select and Intl.PluralRules.prototype.selectRange. (320396@main) (186534585)BigInt values. (321013@main) (187336927)XMLHttpRequest) to keep using the original value. (320742@main) (180048596)Intl.DurationFormat in digital style included a stray separator when minutesDisplay was set to "auto" and minutes ended up hidden. (320182@main) (180722012)Array.prototype.toSpliced threw a TypeError instead of a RangeError when the array’s length was Infinity. (320554@main) (184438837)TypedArray setFromTypedArray could not use memmove when the region was intentionally overlapping and the spec algorithm needed to read back the modified result. (320130@main) (186145219)TypedArray constructor could produce incorrect results when copying Array content whose element access has side effects. (320185@main) (186227333)String.prototype.toLowerCase and String.prototype.toUpperCase did not scan strings inline in the DFG and FTL JIT tiers. (320293@main) (186301688)/v mode regular expression class incorrectly added U+0000 to the class. (320216@main) (186319779)Temporal.ZonedDateTime.prototype.round resolved the rounded wall-clock time using a minute-rounded offset instead of the correct offset. (320217@main) (186319834)Object.freeze() did not invalidate the megamorphic inline cache epoch, which could cause stale property accesses on a frozen object. (320276@main) (186393610)RegExp.prototype.test did not fast-fail when the input string was shorter than the pattern’s minimum possible match length. (320487@main) (186534079)Atomics.isLockFree() converted its argument with a 32-bit integer conversion instead of ToIntegerOrInfinity, which could cause it to incorrectly return true for large size values. (320715@main) (186610789)data: URL included the entire script in Error.stack instead of a truncated URL. (320770@main) (186614179)RegExp.prototype.test could skip the required read of lastIndex, silently dropping observable side effects. (320582@main) (186792229)eval calls that didn’t resolve to the real eval function, and incorrectly applied inside generator and async function bodies, which could produce incorrect results. (320687@main) (186840477)RegExp.escape incorrectly narrowed supplementary code points to 16 bits. (320682@main) (186962453)\- following a class set operand in a /v mode regular expression class incorrectly threw a SyntaxError. (320683@main) (186962513)RegExp::deleteCode() cleared a regular expression’s cached pattern atom, which could cause incorrect values from static RegExp properties such as leftContext and rightContext after the compiled code was reclaimed while idle. (320817@main) (187104089)String.prototype.at and String.prototype.codePointAt could return an incorrect value for an out-of-bounds or negative index because the JIT compiler could eliminate their bounds check. (321014@main) (187336661)MediaRecorder could hold back a lone video keyframe indefinitely instead of using it to start a new recording segment. (320219@main) (186229843)file: URL. (320685@main) (179999480)Blob range larger than 2GB truncated the read length due to an integer overflow. (320981@main) (187207663)IntersectionObserver observation on pages with many observed elements. (320395@main) (185839711)scroll-timeline name could block the main thread for multiple seconds. (320902@main) (186090970)ScrollingStateTree::insertNode performed redundant work reordering children on pages with many sibling scrolling nodes. (320527@main) (186143008)overflow: hidden, could trigger an unnecessary full-layer repaint. (320296@main) (186422878)Continued at the source.
Menu. Currently selected: ⚙️ Manage your personal review settings
GitHub Copilot code review now offers additional personal configurations to an expanded set of Copilot plans and an enterprise-level default setting. These improvements are now generally available:
Previously, personal Copilot code review settings were available only with Copilot Pro, Pro+, and Max on the “Copilot features” page. They covered a single automatic review setting without separate controls for draft pull requests or new pushes.
Under your profile → Copilot settings, a dedicated “code review” page under Copilot is now available on every Copilot plan, including Copilot Business and Copilot Enterprise. From this page you can:
Your default effort applies to reviews you request, including reviews configured to automatically review your pull request. When manually requesting a review from Copilot via the pull request page under “Reviewers”, you can still select a different review effort before requesting.
Authorized enterprise administrators can now set one default review effort (i.e., Lite, Balanced, or the GitHub default) for the whole enterprise. The default applies to organization-owned repositories through inheritance. Organizations and repositories can still set their own overrides.
Menu. Currently selected: ⚙️ Manage your personal review settings
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
This is the final notification that Node 20 is no longer available on GitHub Actions runners. Runners now use Node 24 for JavaScript actions. The temporary ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION opt-out is no longer available.
If you maintain a JavaScript action, update its runs.using value to node24 and publish a new release as soon as possible. For details, see the metadata syntax for JavaScript actions.
If you use JavaScript actions in your workflows, update to the latest versions of those actions that support Node 24. For details, see using versions for actions.
The newest versions of all first-party actions were updated to use Node 24 as referenced in our announcement changelog.
Node 24 is incompatible with macOS 13.4 and earlier, and it doesn’t officially support ARM32. Self-hosted runners using these operating systems or architectures are no longer supported. This change applies to github.com and GitHub with Data Residency.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Action required: If you validate that packages are author-signed by Microsoft using a NuGet client policy or the dotnet nuget verify command, follow the steps in this post as soon as possible to avoid potential disruptions during the transition. If you are unsure whether you are impacted, follow the steps below to check.
Microsoft uses an X.509 certificate to author-sign its NuGet packages. As soon as September 23, 2026, a new certificate will become the default Microsoft author-signing certificate for NuGet packages. Existing packages signed with an older certificate will retain their signatures, but the current certificate will no longer be used to sign new packages after the transition.
Current certificate SHA-256 fingerprint: 566A31882BE208BE4422F7CFD66ED09F5D4524A5994F50CCC8B05EC0528C1353
New certificate SHA-256 fingerprint: 9A1B131BEE0605433056A4EA3815478A8E177961A968C6C0027C1093D1FEB630
To determine whether you have a NuGet client policy configured, check for the following elements in your nuget.config. Keep in mind that nuget.config files can exist in multiple locations with different scopes.
<config>
<add key="signatureValidationMode" value="require" />
</config>
<trustedSigners>
<author name="Microsoft">
<certificate fingerprint="3F9001EA83C560D712C24CF213C3D312CB3BFF51EE89435D3430BD06B5D0EECE" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
<certificate fingerprint="AA12DA22A49BCE7D5C1AE64CC1F3D892F150DA76140F210ABD2CBFFCA2C18A27" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
<certificate fingerprint="566A31882BE208BE4422F7CFD66ED09F5D4524A5994F50CCC8B05EC0528C1353" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
</author>
</trustedSigners>dotnet nuget verify to verify that signed packages are author-signed by Microsoft.This may look like the following:
dotnet nuget verify <PackagePath> --certificate-fingerprint 3F9001EA83C560D712C24CF213C3D312CB3BFF51EE89435D3430BD06B5D0EECE --certificate-fingerprint AA12DA22A49BCE7D5C1AE64CC1F3D892F150DA76140F210ABD2CBFFCA2C18A27 --certificate-fingerprint 566A31882BE208BE4422F7CFD66ED09F5D4524A5994F50CCC8B05EC0528C1353If neither scenario applies to you, you should be unaffected by this certificate update. Microsoft NuGet packages signed with the new certificate should install in the same way as packages signed with older certificates.
If you use a NuGet client policy to enforce an allow list of trusted signers, add the new Microsoft certificate to the allow list as soon as possible. Keep the older Microsoft certificates in the policy so that you can continue to install packages signed with those certificates. If you try to install a package signed with the new certificate without updating your trusted signers, the package installation will fail with an NU3034 error.
You can add the new Microsoft author-signing certificate by running the following command:
dotnet nuget trust author Microsoft 9A1B131BEE0605433056A4EA3815478A8E177961A968C6C0027C1093D1FEB630 --algorithm SHA256The dotnet nuget trust command is available in the .NET 6 SDK and later. It updates the applicable nuget.config file. Use --configfile <Path> to update a specific configuration file.
Alternatively, add the new certificate to the existing Microsoft entry in nuget.config. The resulting entry should include both the older certificates and the new certificate:
<trustedSigners>
<author name="Microsoft">
<certificate fingerprint="3F9001EA83C560D712C24CF213C3D312CB3BFF51EE89435D3430BD06B5D0EECE" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
<certificate fingerprint="AA12DA22A49BCE7D5C1AE64CC1F3D892F150DA76140F210ABD2CBFFCA2C18A27" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
<certificate fingerprint="566A31882BE208BE4422F7CFD66ED09F5D4524A5994F50CCC8B05EC0528C1353" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
<certificate fingerprint="9A1B131BEE0605433056A4EA3815478A8E177961A968C6C0027C1093D1FEB630" hashAlgorithm="SHA256" allowUntrustedRoot="false" />
</author>
</trustedSigners>If you use dotnet nuget verify to confirm that a signed package is author-signed by Microsoft, add the new fingerprint while retaining the older fingerprints:
dotnet nuget verify <PackagePath> --certificate-fingerprint 3F9001EA83C560D712C24CF213C3D312CB3BFF51EE89435D3430BD06B5D0EECE --certificate-fingerprint AA12DA22A49BCE7D5C1AE64CC1F3D892F150DA76140F210ABD2CBFFCA2C18A27 --certificate-fingerprint 566A31882BE208BE4422F7CFD66ED09F5D4524A5994F50CCC8B05EC0528C1353 --certificate-fingerprint 9A1B131BEE0605433056A4EA3815478A8E177961A968C6C0027C1093D1FEB630Each --certificate-fingerprint option adds an accepted SHA-256 signer certificate fingerprint. Keeping all four values allows the command to verify newly signed packages and existing packages signed with an older Microsoft certificate.
If you have questions about how you may be impacted or run into issues while following these steps, please contact us.
For more general NuGet feedback and suggestions:
The post Microsoft is updating its author-signing certificate starting September 23, 2026 appeared first on .NET Blog.
Antoine du Hamel
fe2a6b2be8] - crypto: update root certificates to NSS 3.125 (Node.js GitHub Bot) #64746871167ddfd] - deps: update corepack to 0.36.0 (Node.js GitHub Bot) #65653b816fc8958] - deps: upgrade npm to 10.9.9 (npm team) #64884e306521444] - deps: update icu to 78.3 (Node.js GitHub Bot) #62324a9cb31129f] - deps: update OpenSSL 3.5.8 (Node.js GitHub Bot) #655422a3548e51c] - deps: update Undici to 6.28.1 (mcollina) #657903909ff2c4a] - node-api: support SharedArrayBuffer in napi_create_typedarray (Yilong Li) #6271066de6349ad] - node-api: add napi_create_external_sharedarraybuffer (Ben Noordhuis) #6262344cf27b8fa] - build: update binary-upload to use correct tarball name (Stewart X Addison) #65282fe2a6b2be8] - crypto: update root certificates to NSS 3.125 (Node.js GitHub Bot) #6474671feba6b69] - crypto: update root certificates to NSS 3.123.1 (Node.js GitHub Bot) #63527871167ddfd] - deps: update corepack to 0.36.0 (Node.js GitHub Bot) #656532a3548e51c] - deps: update undici to 6.28.1 (mcollina) #6579059d853a4df] - deps: update archs files for openssl-3.5.8 (Node.js GitHub Bot) #65542a9cb31129f] - deps: upgrade openssl sources to openssl-3.5.8 (Node.js GitHub Bot) #655423376e27de2] - deps: V8: cherry-pick a6eaf7574109 (Camillo Bruni) #65402b816fc8958] - deps: upgrade npm to 10.9.9 (npm team) #648844e4bd1b104] - deps: update timezone to 2026c (Node.js GitHub Bot) #645887d82841b4e] - deps: update c-ares to 1.34.8 (Node.js GitHub Bot) #6433001855a19d3] - deps: c-ares: cherry-pick 8ba37af8e3fb (René) #6411023fb398c3d] - deps: update corepack to 0.35.0 (Node.js GitHub Bot) #633755286330365] - deps: update corepack to 0.34.7 (Node.js GitHub Bot) #628106d6c3c98b1] - deps: update timezone to 2026b (Node.js GitHub Bot) #62962e306521444] - deps: update icu to 78.3 (Node.js GitHub Bot) #62324d9cb8468a3] - doc: clarify filter option of sqlite.database.applyChangeset (Antoine du Hamel) #63515c9c5662d91] - doc: add sxa GPG key (ed25519) (Stewart X Addison) #6419337f21068c4] - fs: restore fs patchability in ESM loader (Joyee Cheung) #628357c2df5dd96] - http2: avoid uaf while receiving and sending rst_stream (esgor) #641663909ff2c4a] - node-api: support SharedArrayBuffer in napi_create_typedarray (Yilong Li) #6271066de6349ad] - node-api: add napi_create_external_sharedarraybuffer (Ben Noordhuis) #62623ce9139107f] - src: escape Windows environment variables in task runner (Antoine du Hamel) #65217839480a471] - test: fix link-local dgram scope assertion (Filip Skokan) #6562940eac4a32f] - test: account for varied OpenSSL CCM final behaviours (Filip Skokan) #6554275098a9e8c] - tools: update gr2m/create-or-update-pull-request-action to v1.10.1 (Mike McCready) #6306508b6ac0416] - tools: revert OpenSSL update workflow to ubuntu-latest (Richard Lau) #6262702cafc479f] - tools: fix commit linter for semver-major release proposals (Antoine du Hamel) #629936f6cd3768d] - tools: sync mk-ca-bundle.pl with curl (Archkon) #64753bc5753d438] - tools: remove envinfo from our workflows (Antoine du Hamel) #64259d68ee9f8a5] - tools: validate version number in release proposal commit message lint (Antoine du Hamel) #6407038ee2e895f] - tools: avoid test/fixtures/wpt/README.md conflicts (Filip Skokan) #63938fdc65e489f] - tools: use different branch for tool updates on staging branches (Antoine du Hamel) #63110e5a6fde002] - tools: update gyp-next to 0.22.1 (Node.js GitHub Bot) #629615fbbad6e82] - url: handle unparsable serialized URLs in setters (Matteo Collina) #64651ed019e4854] - util: preserve function names without source map names (Hiroki Osame) #65108Windows 32-bit Installer: https://nodejs.org/dist/v22.23.3/node-v22.23.3-x86.msi
Windows 64-bit Installer: https://nodejs.org/dist/v22.23.3/node-v22.23.3-x64.msi
Windows ARM 64-bit Installer: https://nodejs.org/dist/v22.23.3/node-v22.23.3-arm64.msi
Windows 32-bit Binary: https://nodejs.org/dist/v22.23.3/win-x86/node.exe
Windows 64-bit Binary: https://nodejs.org/dist/v22.23.3/win-x64/node.exe
Windows ARM 64-bit Binary: https://nodejs.org/dist/v22.23.3/win-arm64/node.exe
macOS 64-bit Installer: https://nodejs.org/dist/v22.23.3/node-v22.23.3.pkg
macOS Apple Silicon 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-darwin-arm64.tar.gz
macOS Intel 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-darwin-x64.tar.gz
Linux 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-linux-x64.tar.xz
Linux PPC LE 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-linux-ppc64le.tar.xz
Linux s390x 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-linux-s390x.tar.xz
AIX 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-aix-ppc64.tar.gz
ARMv7 32-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-linux-armv7l.tar.xz
ARMv8 64-bit Binary: https://nodejs.org/dist/v22.23.3/node-v22.23.3-linux-arm64.tar.xz
Source Code: https://nodejs.org/dist/v22.23.3/node-v22.23.3.tar.gz
Other release files: https://nodejs.org/dist/v22.23.3/
Documentation: https://nodejs.org/docs/v22.23.3/api/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
b85dc5f0d7a9be8038c4e73450d9125b241c792ec7ec495229345f86d1e3f6fa node-v22.23.3-aix-ppc64.tar.gz
a6fcbecafce9c8fbf86ffcb5fb12a66af5f465c449ef5e2fcf7c4e651c1a9b14 node-v22.23.3-arm64.msi
23b25245dcfb9af7262f8ff142e9e2e0af025368117329e7a7458a51e5922f53 node-v22.23.3-darwin-arm64.tar.gz
72d5d8832b41c9d9646197af614ffd751406ea4d215060eb91b98864e1919a3e node-v22.23.3-darwin-arm64.tar.xz
8a677b0219178efd6eb0e475457c4afb452b521a92f6e67845a73bd85727f2a8 node-v22.23.3-darwin-x64.tar.gz
ac41874c3352937119cfec39e1a98c578fe58ce8a86d7e785b89a4706015f305 node-v22.23.3-darwin-x64.tar.xz
eff1a7e67736bdf25db421ddaa613ed605f751a7b9fe07fb473ea880d1348987 node-v22.23.3-headers.tar.gz
8448fe9f94d65aece4df26f266358644d6f41da4ce23aab09675e206edcff1d4 node-v22.23.3-headers.tar.xz
5ced2d48d1d7198739b7f86804de0171aefb6823b684b12341d3321afc3cb0b2 node-v22.23.3-linux-arm64.tar.gz
a44aeb94849a299b22df10b9e622ec2f605c2183501bc40590705131de7c740f node-v22.23.3-linux-arm64.tar.xz
8b99058ec6f4e372b9887f8ecb577a292adb132d59e5104356428a758fdb126c node-v22.23.3-linux-armv7l.tar.gz
590a2768199bdd0b848648a81cd7f07f88a5ceb27949f0c70d5e6652c3eaca69 node-v22.23.3-linux-armv7l.tar.xz
236712ddef73f56c0e0590a2c49f69c8393a2c2d82c3d62c7c165605b472b38d node-v22.23.3-linux-ppc64le.tar.gz
0214e542c73686134856784c0cbd2762b5414097640762b4afc2e33e6e499207 node-v22.23.3-linux-ppc64le.tar.xz
51f64794f5f61bdfc80aa186c8273021473997605cf90ed31ed148bee650a554 node-v22.23.3-linux-s390x.tar.gz
a311c8f0b352e7dedc13958b612ac190a56456fc837dd26cd5e5845713a1eb1d node-v22.23.3-linux-s390x.tar.xz
633f49a4f388c5f9564d56d9c0f5ffa394346d92ca0076fe21cf776ece1988bc node-v22.23.3-linux-x64-musl.tar.gz
0564c678817e6b6fd7860689283185cecdf42e644135b45636094757b9a7922b node-v22.23.3-linux-x64-musl.tar.xz
1084aa36196bba4c3a5e69a1ee388a6e4ff729dad09445fbcd434b28fe3c24af node-v22.23.3-linux-x64.tar.gz
df450af89261115ef9f9e3830c3eeb2cc9213b63c720b1af623cb5dcbe2e02de node-v22.23.3-linux-x64.tar.xz
55c853a53b4bf88c1d8fd70e6add0b22b49c0e76352d4044caaa90512734ab7e node-v22.23.3.pkg
9436c81b284889303d39f5b8bd629bf19e790c665ccb94b4994aa87220d9799a node-v22.23.3.tar.gz
bd97093e1a1e9243338950c174a693a64d4e0926a9c6ce259962bc58d5e96909 node-v22.23.3.tar.xz
33fbc3458ce5cddd694f319bcb47490fbce720c492e25573b594e205c189c89a node-v22.23.3-win-arm64.7z
33dad22e4cef5ee8f9fbb1b0d037fdacd0e56d12a4580f0d63f68b894deab535 node-v22.23.3-win-arm64.zip
fa7a98510f2edf181733dee15a200a5039a0459511366150e4c64da72a53ab94 node-v22.23.3-win-x64.7z
2b0ff57b049cda1bbcea2240eec20467018713c1efe1f7360c2681859b90ed71 node-v22.23.3-win-x64.zip
ff5ffeb0cc42d76b4bfe453638dbe9a69738fdaec967a14cd84f3b11225bb523 node-v22.23.3-win-x86.7z
1e07ac00e61b41b7ecf242de8e38883d237d5ad3fdcad7c16f2dd3a88670e2b6 node-v22.23.3-win-x86.zip
1c0efc8449987e7da5d184786a0a96da83ffa11d334421201e5c09b93017cb8d node-v22.23.3-x64.msi
c71bbe31029c99d6f9cffa4841056ecc49dcd1fb09ef739b7e85dc7b207242b4 node-v22.23.3-x86.msi
b5a3165ec6f24c0b1fe2a10fed58e03f6c1d7d141beb3b225ff030ceac6c5eb5 win-arm64/node.exe
5378f5bad777b4929d1cd4f58974e0ef03815ce5c159863f1875c4b434460c27 win-arm64/node.lib
3f21d9cee58b7168ee46827e04d381dcdeb2d1cc5974bc226ba3a1bcd6bf8988 win-arm64/node_pdb.7z
bb155f2ecad21c98cd909831cb26320e123dd010a654f3649e8ba54a7de9f433 win-arm64/node_pdb.zip
9c9245166b4a8e182e0b797da9c20136117ff24368eaff1fec8343a123c8db0e win-x64/node.exe
c243185f2347870cadb637c909bf123fc703dc65dbfa769b17ec45a4b661d782 win-x64/node.lib
1babb45798cf3a2d4e70574ba926e7ff26b71487ed940a98cdd4971f32af4cae win-x64/node_pdb.7z
ab9fe384f04575fab7b2915b02d109a1d415b184cd6b28239fdb67e12c260321 win-x64/node_pdb.zip
a8aa72dda43af5357d0a502548a54e402d4f7643be221662624a2cc876cd37bb win-x86/node.exe
eff47cc18fe2d3e40886cd49f04a85134681657d8b0f2afc55f363743cf6ddd7 win-x86/node.lib
18963f55b0ebda23c72ab023990e68bce5302645f9643b6f0ac52dc809e3c501 win-x86/node_pdb.7z
95d75dce4db28836ef7ff2ddada21c1df67cc07f7d9a727ac38b021ee3afdeaa win-x86/node_pdb.zip
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQRb6KP2yKXAHRBsCtggsaOQsWjTVgUCarQJ5gAKCRAgsaOQsWjT
Vi6XAQD18cb42T8Iokdj3jgAfoB41fqECt7DeN43VsgOoSNDZwEAxqn3W+nDSkFo
aQyV6A9gETWcFz27zSzMVA+vDrzbvQM=
=2yJE
-----END PGP SIGNATURE-----
You can now create as many Vercel Blob stores as you need. The previous limits of 100 stores on Hobby, 500 on Pro, and 1,000 on Enterprise no longer apply.
Blob store creation is now billed alongside other Blob Advanced Operations, including put(), copy(), and list() calls. On Pro that's $5.00 per million. On Hobby it counts toward the 2,000 free operations you get each month. Deleting a store is free.
Create a new store whenever you want a hard boundary instead of a pathname convention:
Separate production, staging, and preview data, and hand each environment its own credential.
Create a store per customer in a multi-tenant app, so you can export or delete one tenant's data in a single call.
Spin up a store for a preview branch or a migration, then delete it when you're done.
Storage, operations, and data transfer are still billed on what you use, so splitting the same data across more stores costs the same.
Store creation shows up under Blob Advanced Operations on your usage page and in the Observability dashboard.
Learn more in the Vercel Blob Pricing documentation.
Next.js is preparing a scheduled security release for September 30, 2026. This advance notice gives teams time to plan upgrades before patches are published.
The September 30 release will address nine vulnerabilities in Next.js: one critical, two high, five medium, and one low. We plan to publish 16.3.7 and 15.5.27 alongside the full advisories, including impact, affected versions, and upgrade instructions. We recommend upgrading to a patched version once the release is available.
We work with security researchers to secure Next.js and other open source frameworks through Vercel's Open Source Bug Bounty. Anyone interested in contributing to the security of eligible frameworks is encouraged to participate there.
Any questions or concerns regarding our security programs or vulnerability management can be sent to security@vercel.com.
Released September 23, 2026 Stable
macOSUniversalIntelApple silicon
Linux.deb.rpm.tar.gzArm instructionsSnap
Already installed? Use Check for Updates in VS Code. For upcoming features, use the Insiders build.
This release makes large agent session lists faster, extends Dev Container support to remote projects, and improves everyday editing.
Remote Dev Container sessions: Run agents inside your project's Dev Container on SSH, Tunnel, and WSL hosts.
Session list improvements: Load large session lists faster, fit more sessions on screen, and in-place session renaming.
Editor experience: Identify wrapped lines at a glance and avoid duplicate closing brackets as you type.
The agent host runs agent harnesses in a dedicated process based on the Agent Host Protocol (AHP), so you can connect to the same session from multiple VS Code windows. Learn more about its architecture and workflows in the agent host blog post.
Setting: chat.agentHost.devContainer.enabled (Agents Window only)
Let agents build and test your remote project with the right tools and dependencies, without duplicating toolchain setup on your laptop or the remote host. This release extends Dev Container sessions from local folders to projects on SSH, Tunnel, and WSL hosts.
To get started, enable chat.agentHost.devContainer.enabled and select Use Dev Container from the folder menu in the Agents Window. The remote folder must have a supported Dev Container configuration, and Docker must be available on the remote host.
Note: Dev Container sessions are rolling out gradually, so the setting might not be enabled by default for you yet. You can enable the setting manually to try the feature now.
VS Code loads and refreshes large agent session lists faster. The agent host keeps lightweight session and chat metadata in a central catalog instead of opening every conversation database each time the list is built. Full conversation content remains isolated in the individual session and chat databases.
The improvement grows with the number of sessions because the previous approach did work in proportion to your session count. Measured with around 645 sessions on a development machine:
| Operation | Before | After | Improvement |
|---|---|---|---|
| First session listing after launch | 1.3 seconds | 0.1 seconds | About 12x faster |
| Refresh the session list | 0.6 seconds | 0.15 seconds | About 4x faster |
If you have few sessions, expect a smaller difference. Sessions created before this release are migrated automatically in the background.
Fit more sessions in the sessions list by enabling Compact View in the sessions list view of the Agents Window.
Compact rows show the session title at rest and reveal workspace details when you hover over or focus the row. A row expands when the session needs input or approval, so these requests remain visible.
Progress also appears on the row for the chat that owns the work. When you collapse a session, the parent row summarizes progress from its hidden chats.
Disable Empty Groups from Filter Sessions to hide empty custom groups and the empty Chats section. This preference is stored in your profile and resets with the other sessions list filters.
Rename a session or nested chat directly in the sessions list. Double-click its title, use the Rename context menu action, or focus the row and press F2 for a session or F2 for a nested chat. Inline validation prevents blank titles, and canceling restores the previous title.
Setting: sessions.showChatTabs (Agents Window only)
An agent session can contain multiple chats, each representing a different conversation or context. When a session contains multiple chats, choose the presentation that best fits your workflow from the session header menu:
Switching presentations preserves your open chats, active chat, and conversation state. In Single mode, chats that you explicitly open to the side remain independent panes with their own header actions.
Thank you to everyone who submitted a name for the VS Code pet. The naming contest closed on September 17, 2026, and we're reviewing the eligible entries. We'll announce the winner and the pet's new name soon.
While you wait, enter /vscode-pet in chat to meet your companion and explore all its interactions and reactions.
Display word wrap indicators to make wrapped lines easier to identify. An arrow at the word wrap column on the right side of the editor indicates that a line wraps.
VS Code avoids inserting duplicate closing brackets when you type an opening bracket. If a matching closing bracket exists, VS Code uses it. Otherwise, VS Code inserts one.
AuthenticationSession exposes an access token but no information about how long that token stays valid. An extension that passes a credential to an SDK with its own refresh callback cannot distinguish between a token that never expires and one that is about to expire. As a result, the extension either refreshes the credential unnecessarily or lets a long-running operation fail when the token expires.
The authSessionExpiration proposal adds an optional expiresAfter property to AuthenticationSession:
export interface AuthenticationSession {
/**
* The access token's remaining lifetime, in milliseconds, when the authentication
* provider returns the session.
*/
readonly expiresAfter?: number;
}
The value is the remaining lifetime when the session is returned rather than an absolute expiration timestamp. The extension host can run on a different machine than the client, and the two clocks can disagree. Authentication providers that return a cached session recompute the value each time and leave it undefined when the token's expiration is unknown. The built-in Microsoft account provider supplies this value.
Try it out and let us know what you think in the API proposal issue. To learn how to build against a proposal, see using proposed APIs.
None
For users whose organization disables Agent mode by account policy, ensure the Welcome invitation opening is hidden and that alternative methods of launching the disabled Agents Window (for example, code --agents disallow circumvention of the control). #336968: Fix account policy enforcement in the Agents window
For users with enterprise-managed OpenTelemetry (OTel) settings, fix a race condition in the configuration of OTel in the Local (i.e. non-Agent Host Harness) to ensure OTel is not dropped. #336701: Fix Enterprise Managed OTel Race in Copilot Extension
Contributions to vscode:
Contributions to our issue tracking:
We really appreciate people trying our new features as soon as they are ready, so check back here often and learn what's new.
If you'd like to read release notes for previous VS Code versions, go to Updates on code.visualstudio.com.
Local sandboxing helps reduce the potential impact of unintended commands by limiting access to files, network resources, and credentials on your machine. In the GitHub Copilot app, you configure it per project for local repository and working tree sessions.
The project’s sandbox settings include:
These project settings describe the policy that the app requests when a sandboxed session starts. The effective policy can be more restrictive when enterprise-managed settings apply.
If your operating system cannot enforce the requested policy, the sandboxed shell fails with an error rather than running without a sandbox.
Local sandboxing is off by default. Open the app settings, select your project, and turn on Sandbox new sessions under “Sandbox”. This applies to new sessions in the project, not sessions already running. Changes to filesystem, network, and credential settings apply to new sessions or when an existing session restarts.
To enable sandboxing for an active local session, enter /sandbox on. This changes that session without changing the project default.
Local sandboxing does not apply to cloud sandbox sessions or sessions running on a remote host. GitHub Copilot app and Copilot CLI sandbox settings are configured separately.
Local sandboxing is in public preview and subject to change.
Learn more about configuring local sandboxing in the GitHub Copilot app.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Small Talk is our short Q&A with founders from the JetBrains Startup Program. They answer a handful of questions in their own words about what they’re building, why they started, and what they’ve figured out along the way. No pitch, no polish – just a two-minute read to meet the person behind the product.
This time, we sat down with Prasun Kumar, CEO and Founder of Oppex AI, the AI agents that help developers fix bugs that only appear in production. He talked us through what happens before an engineer gets paged, the “chaos monkey” that trains his agents, and why he has stuck with IntelliJ IDEA for 25 years.
Oppex AI builds AI agents that help developers resolve production incidents. When something breaks, it pulls together logs, cloud metrics, database health, affected customers, and recent code changes, checks whether the issue has come up before, and hands the on-call engineer a recommendation before they have even been called. Its goal is to bring mean time to resolve (MTTR) under 10 minutes. About a year in, the 15-person team has launched the product and is working with its first enterprise customers
I started as a software engineer in 2001 and have always worked with startups. Oppex AI is my seventh, and my second as a founder. I’ve always been on the tech and product side, heading engineering at companies that went on to exit. And I’ve used JetBrains the whole way through – I was an early adopter all the way back in 2001.
When scaling engineering at all those companies, the push and pull was always the same. How do you move fast without breaking something? With AI, you can generate a lot of code quickly, but things still get stuck in production. When something fails, it takes a long time to resolve, because the context is spread across so many systems. And each engineer now owns more code than ever, much of which they didn’t write themselves. So the question was simple: How do you help a developer with limited context resolve a production issue fast, with AI’s help instead of another human’s?
Before we even wake up the developer, our agents gather the context. They read the logs, pull metrics from the cloud, check whether the database is under load, and look at the live product to see which customers are affected. They check the change log in GitHub (because a lot of issues start with someone changing something) and whether this issue has come up before and how it was fixed. By the time a developer is called, it’s all assembled into a recommendation. If the problem is in the code itself, our plugin takes that context to the codebase on their machine and points to exactly where the code breaks.
Two things. First, developer logs aren’t really English, so a plain language model doesn’t understand them. Some of our customers run 5,000 machines and 250-plus microservices, and all we have is the logs, so we read them and build a knowledge graph of how the whole system connects. Second, hardening the agent. Think of it like a game. We have our agent, and we have a chaos monkey whose only job is to break the system without telling the agent how. Sometimes the chaos monkey wins, but the agent learns. We run that in a test environment, and that’s what makes it reliable in production.
About 90% of our work is in IntelliJ IDEA, because we’re heavy on Java. WebStorm handles the JavaScript front end, DataGrip the data layer, and PyCharm our smaller Python component, with JetBrains AI Assistant alongside. What keeps us there is depth. AI can write the code now, but the human’s job still involves reading a lot of this code, because you don’t blindly push AI code to production. So we use the IDE as our eyes, not just our hands. We can browse, search, and navigate fast, and see which classes depend on what. After 25 years, it still just does the right thing.
Right now, we’re laser-focused on getting mean time to resolve under 10 minutes. That’s still human-in-the-loop, i.e. we wake someone up and tell them exactly what to do. Our next goal will be an “AI-recommended, human-approved” process, where the recommendation is reliable enough that you can just click a button and you’re done. Eventually, humans won’t even have to get out of bed. When an issue arises, the AI will figure it out and fix it, and the system will heal itself. People are already generating code faster. Once maintaining it in production is automated too, the whole life cycle gets the benefit.
It’s an absolutely amazing time to be building. Features that took companies 10 years to build, you can now build in a year at a fraction of the cost. So a lot of existing categories are up for disruption, not just new ones, because if you’re thinking AI-first, the bigger companies will be slow to respond. If you understand AI and you can wield it, the opportunity is right there.
Read the full case study here.
Q: Do I qualify?
A: You qualify if your company is privately owned, established within the last five years, and has a website or other discoverable online presence.
Q: Where do I apply?
A: You can apply here.
Q: What is the timeline for the JetBrains Startup Program application process?
A: After you apply, our team will review your application within 48 hours. If you meet the criteria, you will receive an acceptance email, followed by a quote for the products. If you’re not accepted, our team will get in touch and share our reasoning. An application may be unsuccessful either due to missing information (e.g., a document or website) or because you do not meet our eligibility requirements (e.g., your business is more than five years old).
Q: What products are included in the terms “IDE subscription”, “AI subscription”, and “team or learning tool subscription”?
A: A variety of products are available through IDE subscriptions, including IDEs as well as .NET and Visual Studio tools. “Team tool subscription” refers to team tools, including TeamCity, YouTrack, Datalore, Qodana, and our learning tool (JetBrains Academy).
New research from GitHub and Yale Program on Climate Change Communication finds strong demand for tools, measurement, and practical guidance that can help developers reduce wasted compute.
Developers know efficient software matters, but many lack a clear way to find waste, measure an improvement, and make the case for fixing it.
That is the central finding from a new survey of 1,039 GitHub users conducted by GitHub and the Yale Program on Climate Change Communication. Eight in 10 respondents said they were interested in tools that help them write more energy-efficient code. Nearly as many wanted best practices for reducing software’s environmental footprint, and almost 75% wanted ways to measure the impact of their software or development process.
There is an opportunity to turn that interest into normal engineering work: identify unnecessary compute, propose a change, test it, and let maintainers decide what ships.
The survey, drawn from GitHub monthly active users in the United States, asked about climate change, AI, software efficiency, and the responsibilities of organizations across the technology sector.
The concern was clear:
These findings describe the views of survey respondents. They do not measure the environmental footprint of AI or any individual software system. The sample was drawn from GitHub users who had opted in to receive marketing communications, so the results should not be treated as representative of every developer or GitHub user.
They do show that many developers are thinking about the environmental effects of the systems they build and use.
When asked questions that also appeared in Yale’s nationally representative Climate Change in the American Mind survey , GitHub users expressed greater concern about climate change than U.S. adults overall.
GitHub users were more likely to say global warming is happening (86% compared with 68% of U.S. adults), that it is at least somewhat personally important (82% compared with 65%), and that it will harm them personally at least a moderate amount (68% compared with 45%). They were also more likely to expect at least moderate harm to future generations (82% compared with 68%) and to say they were worried about global warming (79% compared with 66%).
One note on interpretation: the data in this report are based on a non-probability sample of GitHub users who had opted in to marketing emails, so the findings describe survey respondents rather than developers generally, and differences from data for U.S. adults reflect both population and survey design differences.
Only 10% of respondents said the way they develop and write software has a large effect on reducing their personal environmental impact. Another 28% said it has a moderate effect, while 63% said the effect is small.
At the same time:
Developers are asking for the same things they expect in other areas of engineering: useful tools, credible measurements, and changes they can review.
Open-ended survey responses illustrated the concrete. Respondents asked for help estimating the footprint of repositories and CI/CD workflows, finding unnecessary GitHub Actions runs, improving code efficiency, and comparing AI use with other sources of compute demand. Several also warned against making environmental claims without evidence.
That last point matters. Faster code can reduce resource use, but runtime alone does not prove a reduction in energy use or emissions. Hardware, workload, location, time, and the source of electricity all affect the result. Developers need measurements that match the claim.
Software efficiency is already part of good engineering. It can lower infrastructure costs, improve performance, reduce latency, and free capacity. When an improvement reduces the compute needed to deliver the same successful result, it can also reduce energy use.
A practical starting point is to look for measurable waste in four areas:
The right metric depends on the change. Execution time, CPU use, memory allocation, and network transfer size can all act as useful proxies for computational demand. Each has limits, so state what you measured and what you did not.
For example, a pull request that replaces an O(n²) search with a hash-map lookup should include before-and-after measurements for a representative workload, the commands needed to reproduce the test, and any trade-offs in memory or maintainability. That is a stronger engineering case than calling the change “greener” without supporting data.
Finding efficiency work across a large repository can be slow. GitHub Agentic Workflows can help automate the search while keeping maintainers in control.
The open source Daily Efficiency Improver workflow reviews a repository for opportunities across code, data, network, I/O, and frontend performance. It prioritizes changes that can be measured, runs the repository’s tests, and can open draft pull requests with the evidence and trade-offs for maintainers to review. It does not merge changes itself.
You can add the workflow to a repository with the GitHub CLI:
gh extension install github/gh-aw
gh aw add-wizard githubnext/agentics/efficiency-improverBefore enabling a scheduled workflow, review its permissions, configuration, model use, expected run frequency, and likely compute cost. Start with a suitable test repository or run it manually. Treat every recommendation as a hypothesis until the benchmark and tests support it.
The strongest pull requests should answer five questions:
AI can help developers search, test, and document possible improvements. Humans still decide whether the evidence is sound and whether the change belongs in the codebase.
Efficiency work is easiest to sustain when it fits the tools and decisions developers already use. A repository-level workflow can surface an opportunity. A draft pull request can show the proposed fix. Benchmarks and tests can establish whether it works. Maintainers can then accept, revise, or reject the change.
That loop gives developers the practical support survey respondents asked for: tools, measurement, and a path from concern to code.
Read the full Software Developers on Climate Change, AI, and Sustainable Software report from GitHub and the Yale Program on Climate Change Communication. Then try the Daily Efficiency Improver on a suitable repository and review its findings.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Your email address
100 Exercises to Learn Rust is our adaptation of Mainmatter’s course of the same name, written by Luca Palmieri, Principal Engineering Consultant at Mainmatter, and it has just received its biggest update since we released it a year ago. Palmieri has been writing Rust since 2018, first at TrueLayer and then at AWS, and he wrote Zero to Production in Rust. Mainmatter also delivers this material directly, as an instructor-led workshop for teams. Our version is a JetBrains Academy course that opens inside RustRover. The course is free, and RustRover is free for non-commercial use.
In the sections below, we cover what the course is, what has changed in this update, what the “fast track” for the course looks like, and how to get the most out of AI assistance throughout your experience.
The course teaches Rust through exercises rather than explanations. Each lesson is a short piece of theory followed by a real Rust crate with a test suite attached. Most lessons start with a todo!() and ask you to fill it in.
The majority of the exercises share the same theme: a ticket management system. It starts as a struct with a title, a description, and a status, and ends up serving tickets to multiple threads over an asynchronous REST API. Concepts are introduced as the project needs them, for example Result and enum for errors when validation can fail, HashMap when tickets need to be looked up by ID, or Arc and Mutex when multiple clients access the store at once. And if an important concept doesn’t fit the ticket management system theme, it is introduced with an appropriate self-contained exercise.
The course assumes fluency in at least one other programming language, but no systems background or prior experience managing memory manually. Over eight chapters, it covers ownership and borrowing, traits, enums and pattern matching, error handling, collections and iterators, lifetimes, threads and channels, and futures. The course culminates in a final challenge, where theory is set aside in favor of a single, comprehensive exercise that tests your mastery of the preceding eight chapters.
Our adaptation removes setup. There’s nothing to clone, no branch to check out, and no workshop runner to install. You open the course in RustRover, and the first task is already there with tests attached and a button that runs them in place.
A rebuilt introduction. The course now opens with five short lessons: what you’re going to build, a quick tour of the interface, two minutes of IDE setup, an FAQ, and then your first Rust code. You know what you’re in for before you write anything.
An FAQ with ten answers. Some of the questions come straight from the learners, while others are just our advice in disguise.
The final challenge. All parts of the course contribute to one project: a ticket management system. At the end of the course, your final challenge will be to turn it into an asynchronous REST API. You’ll edit the Cargo.toml file and choose the dependencies yourself. It’s the one part of the course where you’ll be designing something with real crates, roughly the same way you would at work.
A requests.http file specifies the API. It sends the requests your server should handle and states the responses it expects, and you run it against your own server from inside the editor. It checks your API’s behavior, not your Rust code, so every implementation decision is yours, and you can still tell whether it works. The task description also contains a sequence of hints, from a small nudge to a full explanation, that you can use as much or as little of as you need.
All tests are visible. Every task shows the assertions that are used to check your answer, so you’ll know exactly what’s expected instead of guessing.
An experiment with dev and release profiles. In the arithmetic chapter, you compile the same overflowing code under both the dev and release Cargo profiles and watch the behavior change. Built with the dev profile, the program panics on overflow, while it wraps silently when built with the release profile. Most courses merely describe this in one sentence, but here you get to see it happen.
You don’t have to work front to back (and the FAQ says this explicitly). If you’ve written Rust before, or if you’d simply rather tackle one real program than be walked through a hundred small ones, you can skip straight to the final challenge.
The final challenge ties together everything the course covers: the ticket model, enums and error handling, the collection work, shared state and locking, and the whole async chapter by definition. You’ll quickly see which of those areas you need to revisit, because gaps in your understanding will surface as you build the API. You can hop back into those specific parts of the course with concrete questions – and you’ll get much more out of them as a result.
It’s also the quickest way to find out what it’s like to use RustRover for real-world projects.
You can easily access AI through the built-in chat. It’s best to use AI for explanations rather than asking it to write code for you. A couple of good habits to be in: Always add explicit instructions not to change your code. And when the compiler rejects your code, paste the error into the AI chat and ask it to walk you through what’s wrong instead of asking for the fix.
While you are working on the course, however, you should turn AI code completion off. The exercises are short and well represented in training data, so a completion tool will often finish them before you’ve even read the task. This defeats the purpose of having these tasks in the first place: It’s about getting stuck and working out a solution on your own. The Set up your IDE lesson will show you how to switch off AI code completion.
The course is free, RustRover is free for non-commercial use, and there’s nothing to clone.
The adaptation is open source, so if something looks wrong to you, say so. Issues and pull requests are welcome on GitHub.
The Terraform provider for Google Cloud connects Terraform configurations to Google Cloud, giving teams a consistent way to provision and manage Google Cloud infrastructure as code. Today, we are announcing the general availability of version 8.0 of the Terraform provider for Google Cloud.
This major release continues the evolution of the provider around how customers manage Google Cloud infrastructure today. It modernizes several provider defaults, removes resources and properties associated with retired or replaced Google Cloud services, and improves schema behavior to make Terraform plans more predictable.
Version 8.0 also builds on capabilities introduced throughout the 7.x release cycle, including expanded support for discovering existing infrastructure and bringing it under Terraform management through features such as Search and List.
The Google Cloud provider is continuously updated alongside Google Cloud services and Terraform itself. Since the release of version 7.0, several capabilities have expanded across the provider.
During the 7.x release cycle, the Google Cloud provider introduced support for Terraform list resources, starting with service accounts and expanding across a growing set of Google Cloud resources.
List resources provide a read-only mechanism for discovering existing infrastructure. Used with the terraform query workflow, they allow users to search for existing Google Cloud resources outside Terraform state and optionally generate Terraform resource and import configuration for the results.
Support has expanded across commonly used services including Compute Engine, IAM, BigQuery, Pub/Sub, Secret Manager, Migration Center, and Network Services.
The provider also expanded Resource Identity support during the 7.x cycle. Resource identities provide a provider-defined representation of the remote object and can be used for operations such as import alongside traditional provider-specific IDs.
Together, these capabilities make it easier to discover existing infrastructure and prepare it to be brought under Terraform management, particularly in environments where infrastructure already exists outside Terraform state.
The 7.x release cycle continued to expand support for Terraform write-only attributes, allowing sensitive values to be sent to APIs without storing those values in Terraform state.
Write-only support expanded to additional sensitive fields, including certificate private keys, AlloyDB passwords, and IAP credentials.
This gives teams more options for managing sensitive configuration while reducing the amount of credential material persisted in Terraform state.
The provider continued to add resources and capabilities as Google Cloud services evolved. This includes additional support across areas such as Vertex AI, Discovery Engine, GKE, networking, security, data services, and migration tooling.
As with previous releases, these updates are delivered continuously through the provider's regular release cadence rather than being held for a major version.
Version 8.0 uses the major-version boundary to introduce several behavioral and schema changes that could not be made safely in a minor release.
The default load_balancing_scheme for google_compute_backend_service and google_compute_global_forwarding_rule has changed from EXTERNAL to EXTERNAL_MANAGED.
Configurations that do not explicitly specify a load-balancing scheme will therefore use the modern external Application Load Balancer behavior. Users that need to retain Classic Application Load Balancer behavior should explicitly configure load_balancing_scheme = "EXTERNAL".
For migration details, refer to the google_compute_backend_service and google_compute_global_forwarding_rule sections of the version 8.0 upgrade guide.
Google Cloud provider 8.0 removes a number of resources and data sources associated with services or APIs that have been retired, replaced, or superseded.
Examples include:
google_iap_brand and google_iap_client, following the shutdown of the IAP OAuth Admin APIs.
google_notebooks_environment, google_notebooks_instance, and google_notebooks_runtime, following the end of life of the associated Notebooks products. Users should migrate to google_workbench_instance.
google_ml_engine_model, with machine learning deployments moving to Vertex AI.
google_beyondcorp_app_connection, google_beyondcorp_app_connector, and google_beyondcorp_app_gateway, with Security Gateway resources providing the replacement path.
google_vertex_ai_schedule, which is replaced by google_colab_schedule.
These are breaking removals, so configurations using these resources must be updated before upgrading. Refer to the version 8.0 upgrade guide for the migration path for each affected resource.
Version 8.0 includes several schema, validation, and behavioral changes designed to better reflect Google Cloud API behavior.
Several attributes where ordering is not significant have changed from lists to sets, including fields in Compute Service Attachments, GKE logging and monitoring configuration, and Cloud Security Compliance Frameworks. These changes help prevent perpetual diffs when APIs return values in an order different from the order represented in Terraform configuration or existing state.
Validation has also been tightened where Google Cloud APIs already require particular values. For example, source_contents is now required for google_workflows_workflow, and claim_mapping is required when creating Workforce Identity Pool Provider SCIM tenants.
These changes allow Terraform to catch more configuration issues during planning and reduce differences caused by how API responses are represented in state.
Google Cloud provider 8.0 is a major release, so users should review their configurations before upgrading.
The Terraform provider for Google Cloud 8.0 Upgrade Guide documents removed resources and data sources, field changes, validation updates, state migrations, and other breaking changes.
When planning an upgrade, we recommend that users:
Upgrade to the latest 7.x provider release first and resolve existing deprecation warnings.
Review configurations for resources and fields removed in version 8.0.
Explicitly configure load_balancing_scheme = "EXTERNAL" where Classic Application Load Balancer behavior is still required.
Review configurations affected by schema and validation changes, including attributes converted from lists to sets and write-only fields whose version attributes have changed type or are now required.
Test the upgrade in a non-production environment and carefully review the resulting terraform plan before rollout, paying particular attention to resources that Terraform plans to destroy or replace.
Some state changes, including certain integer-to-string conversions, are migrated automatically by the provider, while other changes require updates to Terraform configuration. Refer to the upgrade guide for the requirements of each affected resource.
Terraform provider for Google Cloud 8.0 is now available in the Terraform Registry.
For detailed migration guidance, review the Terraform provider for Google Cloud 8.0 Upgrade Guide. For the complete list of changes included in the release, refer to the provider changelog on GitHub.
The Google Cloud provider is developed through the continued collaboration of the Google Cloud engineering team, our HashiCorp team, and the Terraform community. Thank you to the maintainers, contributors, and users whose feedback and contributions continue to improve the provider.
Understand how Copilot agents perform and interact with models and tools. The GitHub Copilot app now supports OpenTelemetry (OTel) configuration through enterprise-managed settings.
OTel is an open source observability framework. Administrators can use it to send agent activity data to their organization’s compatible monitoring tools. This helps teams:
Configure the telemetry property in your enterprise’s managed-settings.json file to enable export and specify the endpoint that will receive the data. Prompt and response content is excluded by default—review your content-capture settings before enabling it.
Learn more about OpenTelemetry for agent monitoring and configuring enterprise-managed settings.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Menu. Currently selected: What's new
GitHub Copilot for JetBrains 1.18.0 brings AI-assisted tool approvals, more control over agent conversations, and shared skills and instructions for your organization. You can also review plans with the Codex agent and manage MCP tools with persistent controls.
AI-assisted tool approvals, called assisted approvals, are now in public preview for Copilot agent sessions. Low-risk tool calls receive automatic approval, while higher-risk actions continue to prompt you for a decision.
This gives you fewer approval interruptions for low-risk actions while keeping higher-risk decisions in your hands.
You can now re-edit a previous user message in a Copilot agent session. Before sending your replacement message, Copilot rewinds both the conversation and file changes.
This lets you revise an earlier request and continue from that point, rather than adding another message to correct the direction of the conversation.
Local and Copilot agent sessions now support organization and enterprise skills, along with organization-managed custom instructions. You can use shared skills and organizational guidance in both types of sessions.
The Codex agent now supports plan mode. You can review, refine, or approve a plan before implementation, giving you an opportunity to shape the approach before the agent starts making changes.
A new setting lets you turn the built-in GitHub MCP Server on or off without changing manually configured MCP servers. The built-in server remains enabled by default.
Copilot agent sessions also gain persistent per-tool controls for MCP servers. You can manage individual tools as well as control whether the built-in server is enabled.
A new side-by-side chat panel switcher in the session toolbar lets you chat in the editor while browsing sessions in the tool window. You can keep your conversation open alongside the session list.
Other updates make features and settings easier to discover:
/init tip and grouped it with customizationsThis update improves inline chat reliability, including preserving your edits when requests end and respecting selected thinking effort and context window settings. It also addresses Codex session startup issues, improves behavior across multiple project windows, and restores embedded editors and message re-editing on IntelliJ 2026.3 EAP builds.
Inline chat and its entry points are now hidden in JetBrains Gateway and remote development environments.
If you use a JetBrains IDE version 2025.1, you will see advance notice to upgrade to 2026.1 or later. Support remains unchanged in this release.
We encourage you to try out the latest version of the GitHub Copilot plugin and share your feedback. Your input is invaluable in helping us refine and improve the product.
Your feedback drives improvements. We’d love to hear about your experience in the following channels:
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Drives for Vercel Sandbox are now available in public beta on Hobby, Pro, and Enterprise.
A Drive is persistent storage that you mount as a directory in a Vercel Sandbox. It isn’t tied to a single sandbox, so you can reuse the same Drive across runs and different sandbox instances.
Use Drives to preserve an agent's workspace or on-disk memory, or to reuse datasets, models, and dependency trees.
Create or retrieve a Drive and mount it at a path when starting a sandbox. Read and write files through the sandbox filesystem at that path.
Anything stored under /data remains on the Drive after the sandbox stops.
A Drive supports one read-write mount at a time. After the Drive has been written to, multiple sandboxes can read from it concurrently by mounting point-in-time, read-only snapshots.
Each snapshot reflects the Drive at the moment it's mounted. Later writes aren’t included; mount a new snapshot to access them.
Each sandbox can mount up to four Drives at separate paths. Drives default to a maximum size of 1 TiB (1 GiB on Hobby) and can be configured up to 16 TiB, with higher limits available by request.
Drives are available in every Sandbox region. Each Drive stays in the region where it was created. Sandboxes that mount it must run in that region and can’t use failover regions.
Drive pricing is based on storage, reads, and writes, with rates varying by region. In iad1, storage costs $0.05 per GB-month, reads $0.0015 per GB, and writes $0.004 per GB. Hobby includes 15 GB of Drive storage and 30 GB each of reads and writes per month. See Sandbox pricing for regional rates and plan details.
Learn more in the Drives documentation.
Today we're launching two Cursor bots for the last mile of shipping code. Rollouts watches every change as it deploys and reports its health per environment. Security Review reports exploitable bugs on every pull request.
Both are available today on Teams and Enterprise plans.
Rollouts attaches a monitor to every pull request and watches the change as it deploys, reporting change health per environment: verified healthy, regression detected, or inconclusive. It's the Cursor version of Firetiger Change Monitors, rebuilt with the Bot Development Kit.
Enable it from the dashboard and connect source control, your deploy system, and your telemetry provider. Rollouts starts watching on the next pull request.
When a pull request opens, Rollouts reads the diff and the systems it touches, then writes a monitoring plan as a PR comment. The plan lists the risks it identified, the effect the change is meant to have, the signals it will check, and any gaps in instrumentation that would make the change hard to verify. Edit the plan in the PR and Rollouts uses your version.
Rollouts wakes on deploy events for the change's commit and runs the plan against your logs, metrics, and traces. It tracks each environment separately, so a change can be verified in staging and still flagged in production. Rollouts checks the change's intended effect alongside error and latency signals, and reports back on the PR when it reaches a verdict.
When Rollouts detects a regression, it names the change it suspects and notifies the author. Depending on configuration, it can also open a revert PR for review or hand the finding to a cloud agent for a fix. Rollouts does not merge or roll back on its own today.
Rollouts connects to Origin or GitHub for source control, to your continuous delivery system for deploy events, and to Datadog and other telemetry providers for signals. Feature flag integration is coming soon.
Security Review is available today. It reads every pull request in the context of the codebase and posts one review comment reporting exploitable bugs. Style and quality stay with Bugbot.
<figure><img src="https://ptht05hbb1ssoooe.public.blob.vercel-storage.com/assets/changelog/security-review-N8azgyLevr8FvNIRqJN6hk71os2Oxu.png" loading="lazy" alt="Security Review comment on a pull request reporting an exploitable bug with a severity and proposed fix" /><figcaption>Security Review comment on a pull request reporting an exploitable bug with a severity and proposed fix</figcaption></figure>
Enable it from the dashboard for the repositories you want reviewed. Draft PRs are skipped.
Security Review looks for injection across SQL, command, and template surfaces, along with authentication and authorization bypasses, including checks that a refactor stopped running. It also flags secrets and credentials committed to source, SSRF and unvalidated redirects, unsafe deserialization, and dependency changes that introduce known vulnerabilities. It traces where user input enters and what it passes through.
Each finding carries a severity, the attack path, and a proposed fix. Dismiss one with a reason and Security Review won't raise it again on that PR.
Add rules for your codebase, such as which client external calls must go through or which tables are never queried from a request handler, and Security Review enforces them on every PR.
Rollouts and Security Reviewer are available today on Teams and Enterprise plans. Enable either bot from the automations tab.
For the next 10 days, we're including usage credits so teams can try Rollouts on real changes. Teams and Enterprise customers receive credits for roughly 50 and 500 changes, respectively.
On September 23, 2026, we released versions 19.4.1, 19.3.3, 19.2.7 for GitLab Community Edition (CE) and Enterprise Edition (EE).
These versions contain important bug and security fixes, and we strongly recommend that all self-managed GitLab installations be upgraded to one of these versions immediately. GitLab.com is already running the patched version. GitLab Dedicated customers do not need to take action.
GitLab releases fixes for vulnerabilities in patch releases. There are two types of patch releases: scheduled releases and ad-hoc critical patches for high-severity vulnerabilities. Scheduled releases are released twice a month on the second and fourth Wednesdays. For more information, please visit our releases handbook and security FAQ. You can see all of GitLab release blog posts here.
For security fixes, the issues detailing each vulnerability are made public on our issue tracker 90 days after the release in which they were patched.
We are committed to ensuring that all aspects of GitLab that are exposed to customers or that host customer data are held to the highest security standards. To maintain good security hygiene, it is highly recommended that all customers upgrade to the latest patch release for their supported version. You can read more best practices in securing your GitLab instance in our blog post.
We strongly recommend that all installations running a version affected by the issues described below are upgraded to the latest version as soon as possible.
When no specific deployment type (omnibus, source code, helm chart, etc.) of a product is mentioned, it means all types are affected.
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to execute arbitrary code on the GitLab server due to a double free issue when parsing a specially crafted regular expression in a CI/CD configuration.
Impacted Versions: GitLab CE/EE: all versions from 19.2 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 9.9 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:L)
Thanks joaxcar for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to execute arbitrary code on the GitLab server due to an integer overflow issue when compiling a specially crafted regular expression in a CI/CD configuration.
Impacted Versions: GitLab CE/EE: all versions from 19.2 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 9.9 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H)
Thanks joaxcar for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to execute arbitrary JavaScript in the context of another user’s browser session due to improper sanitization of path components in the merge request diff viewer.
Impacted Versions: GitLab CE/EE: all versions from 13.11 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 8.7 (CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:C/C:H/I:H/A:N)
Thanks joaxcar for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to access sensitive CI/CD variable values from debug-mode job traces through the Duo AI troubleshooting feature due to missing authorization checks.
Impacted Versions: GitLab EE: all versions from 18.7 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 7.7 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:N/A:N)
This vulnerability has been discovered internally by GitLab team member Daniel Prause
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user with an MCP-scoped token to perform actions beyond the intended scope of that token due to improper authorization checks.
Impacted Versions: GitLab CE/EE: all versions from 18.3 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 5.4 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:N)
This vulnerability has been discovered internally by GitLab team member Amr Taha
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to spoof merge request authorship and attribute content to arbitrary existing users on the target instance due to improper reliance on ephemeral cache state during Direct Transfer imports.
Impacted Versions: GitLab CE/EE: all versions from 19.1 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 4.3 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:L/A:N)
Thanks ahacker1 for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user to read private child issue contents, including titles and descriptions, from projects they had no access to, due to missing authorization checks on linked work items within visible epics.
Impacted Versions: GitLab CE/EE: all versions from 19.0 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 4.3 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N)
Thanks rogerace for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user with developer-role permissions to bypass admin-configured AI tool governance controls for workflows in namespaces they do not control due to improper authorization checks.
Impacted Versions: GitLab EE: all versions from 19.1 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 4.3 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:L/A:N)
This vulnerability has been discovered internally by GitLab team member Rahul Barnwal
GitLab has remediated an issue that under certain conditions could have allowed an authenticated user with guest-level permissions to read private security policy content they were not authorized to access due to improper authorization enforcement.
Impacted Versions: GitLab EE: all versions from 17.9 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 4.3 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N)
Thanks rogerace for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under certain conditions could have allowed an unauthenticated user to read CI/CD job trace contents containing sensitive variable values due to improper authorization enforcement in the GraphQL API.
Impacted Versions: GitLab CE/EE: all versions from 15.11 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 3.7 (CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:N/A:N)
Thanks scyoon for reporting this vulnerability through our HackerOne bug bounty program
GitLab has remediated an issue that under a race condition, the MCP search tool’s shared state handling could have caused search results to be returned under an incorrect user context.
Impacted Versions: GitLab CE/EE: all versions from 18.6 before 19.2.7, 19.3 before 19.3.3, and 19.4 before 19.4.1
CVSS 3.1 (CVSS:3.1/AV:N/AC:H/PR:L/UI:N/S:U/C:L/I:N/A:N)
This vulnerability has been discovered internally by GitLab team member Chris Bonk
Add 19.4 release note: Improved table pasting in rich text editorAdd 19.4 release note: organization security dashboardsAdjust docs following patches for "require a commit SHA for merge requests API"This patch includes database migrations that may impact your upgrade process.
The following versions include post-deploy migrations that can run after the upgrade:
To learn more about the impact of upgrades on your installation, see:
To update GitLab, see the Update page. To update GitLab Runner, see the Updating the Runner page.
To receive patch blog notifications delivered to your inbox, visit our contact us page. To receive release notifications via RSS, subscribe to our patch release RSS feed or our RSS feed for all releases.
We’re excited to announce that the Python documentation is now available in Persian (فارسی)! 🎉
A huge thank you to everyone who contributed their time and expertise to the translation effort. Community contributions like these help make Python more accessible to people around the world.
Did you know? Many letters in the Persian script change shape depending on where they appear in a word. A letter may have isolated, initial, medial, and final forms while still representing the same character! See the W3C’s Arabic & Persian Layout Requirements, section 4.3.1, Joining Forms, for more information.
Python’s documentation is available in many other languages too, and these translations depend on contributors to keep them accurate and up to date. If you’d like to help translate or maintain Python documentation in your language, see the translation guide in the Python Developer’s Guide and the Translation dashboard.
The OpenJDK Quality Group is promoting the testing of FOSS projects with OpenJDK builds as a way to improve the overall quality of the release. This heads-up is part of a Quality Outreach update sent to the projects involved. To learn more about the program, and how-to join, please check here.
A new JavaDoc tag, @note, is being proposed to highlight useful tips or warnings for developers using an API. For example:
/**
* Determine the maximum foo in a list of bars.
*
* {@note There is always a maximum foo, even if the list is empty.}
*
* The arguments to this method must be non-null.
*/It would be rendered as:
Determine the maximum foo in a list of bars.
Note: There is always a maximum foo, even if the list is empty.
The arguments to this method must be non-null.
For both inline and block notes, the note body is rendered as a text block with a header that defaults to Note:. Inline notes are displayed with a vertical bar on the left side to make them stand out against the surrounding text:
Block notes with the default style are displayed with a small header and indented text, using the same layout as other block tags:
The top-level HTML element generated for a block note uses the CSS class block-note, while the top-level element for an inline note uses the CSS class inline-note. Additional CSS classes can be added using attributes or custom note tags as discussed below.
Additional details can be provided as attributes: name=value pairs enclosed in parentheses after the tag name and before the note body, as shown in this example:
{@note (name=value) ...}
Some attributes are recognized by the @note tag in the Standard Doclet. These attributes include header, for updating the heading of a note, kind, which is encoded as an additional CSS class, and id for adding an id attribute to an HTML element. Here is an example of using the header attribute to change the heading of a @note to warn a developer about a potential issue:
/**
* {@note (header='Caution:') Untrusted input must be verified!}
*/Would be rendered as:
Caution: Untrusted input must be verified!
A custom @note tag can be defined using the javadoc -tag option. javadoc -tag will be extended to allow for aliasing of the @note tag. Here is an example of creating a @warning tag that is an alias of @tag:
javadoc -tag 'warning:A:Warning:' ...
This could then be used as:
/**
* {@warning Remember to flush the cache before syncing.}
*/And would be rendered as:
Warning: Remember to flush the cache before syncing.
Additional customization options for @note are offered; check JDK-8363700 for details.
Feedback is also welcome through the javadoc-dev mailing list (registration required). For more details on this proposed change, check JDK-8363700.
~Menu. Currently selected: Configure indexing
C++ code intelligence in GitHub Copilot CLI is now faster with support for whole codebase indexing.
C++ repositories can contain millions of lines of code across deeply connected source files and headers. Without a reusable index, code-intelligence requests may need to rediscover project information as you navigate, making it slower to find a definition, locate references, or understand unfamiliar code.
Whole codebase indexing (WCI) creates a persistent index of symbols across your C++ project, including files that aren’t currently open. The Microsoft C++ Language Server uses your project’s compilation information to resolve types, symbols, includes, and relationships between files. WCI makes that symbol information available for reuse instead of rediscovering it for each request.
You spend less time waiting for definitions, references, implementations, and symbol search results, and more time reviewing, understanding, and changing code.
Whole codebase indexing is enabled by default because its persistent symbol index helps the Microsoft C++ Language Server efficiently understand relationships across your entire project. The language server loads the index when you first open a C++ project. You can check indexing progress at any time with /lsp logs.
Building the index for the first time can take additional time and temporarily increase memory usage, particularly for large or complex repositories. After the initial index is complete, it is reused and dynamically updated, so this overhead is primarily associated with initial setup.
To disable whole codebase indexing, you can temporarily disable WCI using the indexing documentation.
Restart your Copilot session after changing the setting.
Help us improve the Microsoft C++ language server for Copilot CLI by filling out our short survey. To report a problem or suggest an improvement, open an issue in the GitHub repository.
Menu. Currently selected: Configure indexing
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Today, Amazon CloudWatch introduces CloudWatch Omni, a unified observability experience for application and AI workloads that is app-centric, AI-powered, built on open standards, and delivered off-console. CloudWatch Omni is a purpose-built observability, evaluation, and experimentation solution for AI agents. It helps teams design, evaluate, and operate AI agents across any model provider, framework, or runtime, with an eval-driven workflow, support for the tools you already use, and observability delivered where you work: directly in your IDE and through a standalone web experience, separate from the AWS Management Console.
Organizations deploying agentic AI systems face observability challenges that traditional monitoring can’t address. Agent behavior is non-deterministic: a prompt change can degrade response quality even when standard metrics show no errors. Teams spend hours manually reviewing logs across multiple systems, unable to pinpoint what changed or why. Existing tools force teams to choose between siloed generative AI monitoring or fragmented solutions requiring constant context-switching between their coding environment and browser-based dashboards.
CloudWatch Omni captures every trace and includes built-in evaluators for correctness, coherence, retrieval quality, and tool selection, among others. You can compare prompt versions side by side in the playground, build test datasets from production traffic, run experiments across different configurations, and detect regressions automatically.
Two surfaces for development and operations
CloudWatch Omni delivers observability through two complementary surfaces. Developers get a native extension inside VS Code and Kiro (the currently supported IDEs), where traces appear as you run your agent with a playground and evaluators a click away. Operators get a standalone web experience, separate from the AWS Management Console to monitor the fleet, accessible through SSO with no AWS console needed. Both share the same data: the trace a developer debugs is the trace an operator investigates.
The Cloud Login feature connects your local IDE environment to your AWS account, enabling you to send telemetry data to Amazon CloudWatch for persistent storage, share traces with your team, and access production dashboards. This connection is optional. You can use CloudWatch Omni entirely locally during development, then connect to the cloud when you are ready to monitor agents in production.
Getting started
CloudWatch Omni offers two ways to get started: through the IDE extension (for VS Code and Kiro) or directly through the cloud experience, where you can start sending telemetry data to CloudWatch without installing any IDE extension. In this walkthrough, I install the extension, create an agent, run it, and explore the traces and evaluation tools from my IDE.
After installing the CloudWatch Omni extension from the VS Code Marketplace, the CloudWatch Omni icon appears in the Activity Bar. From the welcome screen, I selected Get started with Sample Project to load a pre-configured agent with sample trace data or use shortcut to Command Palette using Command + Shift + P (on macOS) or Ctrl + Shift + P (on Windows/Linux) and select Omni: Create a new Project
The sample project comes with an agent implementation and example datasets. Part of the getting-started experience is adding OpenTelemetry instrumentation, and CloudWatch Omni guides you through each step. You can also create a new agent from scratch. CloudWatch Omni walks you through the process using an interactive chat where you define the agent’s purpose, select a model provider, and configure tools. All data is stored locally by default. You can optionally connect to AWS to send data to Amazon CloudWatch.
After verifying the configuration, I started the local dev server and sent a question to the agent. What makes this different from a typical chatbot interface is what happens next: selecting View Trace shows exactly how the agent processed the request.
CloudWatch Omni integrates with AI code assistants such as Kiro, Claude Code, and Codex to streamline the setup process. These assistants can configure the Dev Server, install dependencies, and set up instrumentation on your behalf, so you can go from installation to running your first traced agent session in minutes without manual configuration.
Traces are essential for understanding AI agent behavior. Unlike traditional request-response systems, agents make multiple decisions per invocation: choosing tools, composing prompts, and chaining sub-calls. Without full trace visibility, diagnosing why an agent produced an incorrect answer or took an unexpected path becomes guesswork. CloudWatch Omni records every step in a structured timeline so you can pinpoint exactly where behavior diverged.
The Trace Explorer shows a detailed breakdown of every step the agent took (LLM calls, tool invocations, and reasoning steps) in a structured, hierarchical timeline. I could drill into any span to inspect inputs, outputs, token usage, and latency.
The Trace Explorer also supports Compare mode, which places two traces side by side to see how different prompts or configurations affect behavior. Compare mode is especially helpful when debugging regressions. And with Ask Assistant, an AI agent analyzes your traces to surface patterns and anomalies, answering questions like “Why did the agent call this tool twice?”
Evaluation is what turns observability into actionable quality improvement for generative AI. Traditional metrics like latency and error rate cannot tell you whether an agent’s response was helpful, coherent, or factually correct. Evaluators score each response against quality dimensions, letting you measure what users actually experience and catch regressions that standard monitoring misses entirely.
CloudWatch Omni includes 17 built-in evaluators for metrics like coherence, helpfulness, faithfulness, and routing correctness. I selected traces from the Trace Explorer, chose evaluators, and ran an evaluation, getting per-example scores and aggregate metrics without building any custom evaluation framework.
From there, I used the Playground to test different system prompts side by side, comparing multiple model and prompt configurations in real time to see how each variation affects output quality before committing changes. With the Experiments view, I could run the same dataset against two agent variants and compare their evaluation scores, latency, and token usage side by side to pick the best-performing configuration.
Figure 7. Comparing evaluations across agent variants in the Omni Experiments console
With Prompt Management, you can version and track prompt configurations over time, making it easy to roll back when a new version underperforms.
CloudWatch Omni also provides a Session Explorer to review full conversation histories and understand how agents handle multi-turn interactions, along with an Agent Topology view that visualizes the architecture of your agent system, including sub-agents, tools, and their interconnections. You can drill into any node to inspect performance and identify bottlenecks.
CloudWatch Omni also offers a dedicated web experience accessible from any browser without an IDE. Teams can access all capabilities collaboratively, including application monitoring, analytics, agent observability, and AI-powered investigations.
I curated traces into golden datasets for structured experimentation. The Experiment function runs the agent against a dataset and automatically scores results, creating benchmarks for regression testing whenever prompts or agent logic change.
If you already have an agent built with a supported framework, CloudWatch Omni provides two paths to add instrumentation: Auto-instrument with Kiro, which detects your framework and configures tracing automatically, or manual instrumentation with ready-to-use code snippets for Python and TypeScript. For detailed instrumentation guides, see the CloudWatch Omni documentation.
Supported frameworks and open standards
The walkthrough above uses the sample project, but CloudWatch Omni works with the agent frameworks teams are already using: LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, Vercel AI SDK, and more, in both Python and TypeScript. It also provides native observability for agents built with Amazon Bedrock AgentCore, and uses AgentCore’s evaluation capabilities to assess agent quality directly within the Omni workflow.
Instrumentation uses open standards (OpenInference and ADOT), whether your agents run on Lambda, ECS, EKS, or other clouds. For evaluation, Omni integrates with third-party evaluators including AutoEval and DeepEval alongside built-in datasets, a playground, and batch experiments. No re-platforming required.
CloudWatch Omni brings agent observability and application observability together in a single experience. For the application observability experience, read the companion post Introducing Amazon CloudWatch Omni: collaborative AI-powered observability for your applications.
Pricing and availability
Amazon CloudWatch Omni is now generally available. The IDE extension is free to use. You don’t need an AWS account to get started. You only need AWS credentials for Amazon Bedrock models, or API keys for other providers like OpenAI or Anthropic. Get started today by installing the extension from the VS Code Marketplace.
To explore all capabilities and get started quickly, visit CloudWatch on AWS Builder Center.
If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Share your feedback on AWS re:Post or reach out through your usual AWS Support contacts.
Happy building!
— Daniel Abib
An out-of-band security update is now available in v16.3.6 (Active LTS) and v15.5.26 (Maintenance LTS). These releases upgrade upstream dependencies, including Satori, to address an issue that could lead to remote code execution in affected Next.js versions. Version 15.5.26 includes related hardening, but Next.js 15.x is not affected by the remote code execution issue.
Please patch your Next.js dependencies to maintain the security of your applications.
ImageResponse (Critical Severity)GHSA-vcvr-r3jv-pc5j (Next.js)
Related upstream advisory: GHSA-wx4j-mvgx-mqwp (Satori)
Next.js versions >=16.2.0 <16.3.6 are affected.
The issue affects the Node.js ImageResponse implementation in next/og. Under specific conditions, improper escaping in SVG output generated by Satori could lead to remote code execution due to vulnerabilities in other upstream dependencies. The fix upgrades those dependencies.
Applications using the Edge ImageResponse implementation are not affected.
We work with a talented set of researchers to secure Next.js and other open source frameworks through Vercel's Open Source Bug Bounty. Anyone interested in contributing to the security of eligible frameworks is encouraged to participate there.
Any questions or concerns regarding our security programs or vulnerability management can be sent to security@vercel.com.
Googlebook introduces a new category of laptops built on a shared Android foundation. High-performance hardware from partners such as HP, Dell, Lenovo, Acer, and Asus, combines mobile convenience with desktop power. Googlebook offers high-resolution OLED touchscreens, dedicated keyboards, and precision trackpads with all-day battery life and OS-level Gemini Intelligence. With Googlebook, users can transition fluidly from quick interactions on their phones to rich, immersive sessions on their laptop.
Bringing your app to Googlebook opens up valuable opportunities for you across the Android ecosystem. Google Play highlights optimized titles with dedicated badging, enhanced search, and featured spots across curated store homepages. Delivering this level of quality also prepares your app for the Apps Experience Program, where you can enroll to unlock a new program rate card designed to drive business growth. Even better, when users set up their new Googlebook using their Android phone, optimized apps are prominently highlighted for easy transfer, giving your app day-one presence on their new device.
The best part? You don't need to build a separate app from the ground up to take advantage of this reach. Adaptive development is how modern Android apps naturally scale across large displays, new device postures, and emerging form factors. If your app already embraces adaptive layouts, it is primed for Googlebooks. By building on your existing foundation of adaptive UI, window size classes, and multi-input support, you can deliver an optimized experience.
On a laptop, your app operates within a desktop environment where user expectations shift toward higher information density, precision input, and active multitasking. Following desktop development and design guidance provides the principles needed to make the most of this experience. Instead of simply stretching mobile interfaces across a wide screen, an adaptive layout reorganizes content into functional groupings.
Adopt a multi-pane architecture to allow your UI to expand, reflow, or reveal richer detail as window boundaries change. With Navigation 3, you can implement adaptive scene strategies to coordinate multi-pane layouts directly from your back stack. Use ListDetailSceneStrategy and SupportingPaneSceneStrategy to enable side-by-side layouts when expanded window space is available. Scene decorators let you wrap screens with persistent desktop navigation rails. Pair these patterns with layout primitives like Grid and FlexBox, and soon alongside experimental MediaQuery and Styles APIs, to organize complex content and adjust visual styles dynamically for desktop displays.
In free-form desktop windowing, app windows can be resized dynamically at any time. Your layout decisions should respond directly to the available window space using window size classes rather than the physical display dimensions.
Desktop design also accounts for ergonomic viewing distances and precise pointer targets. Adjust your type scale for comfortable viewing across larger displays, set layout max widths to keep line lengths readable, and define explicit click targets to prevent misclicks. Explore complete design patterns in our design principles guide and discover real world inspiration in the desktop design gallery.
Once your core layout is adaptive, you can enrich your app with differentiated features that take full advantage of a desktop environment. Everyday productivity in these setups relies on versatile input methods. Jetpack Compose natively supports physical keyboard navigation and pointer selection. Elevate your app’s usability by integrating contextual cursors that provide visual feedback for text entry, pane resizing, and tool selection. Implement right click context menus and hover states; make your shortcuts discoverable through the Keyboard Shortcuts Helper.
On Googlebook, apps run in free-form windows where users can tackle multiple tasks simultaneously. Unlock side-by-side workflows by enabling multi-instance support, giving users the ability to launch independent windows for comparing content or managing multiple documents. Pair this with drag and drop to let users move text, images, and files fluidly between windows or even drop items onto an empty workspace to spin up a new task.
Go all in and customize your window frame. In desktop windowing, apps include a caption header bar that you can style with custom backgrounds, search bars, or tabs while respecting system window controls.
Beyond individual app windows, Continue On keeps experiences connected across phones, tablets, and Googlebooks with bidirectional handoff that lets users start a task on one screen and pick up seamlessly on another. Passing state through HandoffActivityData preserves context such as document position or active tabs, with optional web fallbacks to ensure smooth transitions.
Complement this by surfacing actionable information at a glance with customizable widgets. And, as you refine your app experience, benchmark against our comprehensive desktop app quality guidelines.
Developers are already bringing these patterns to life across the ecosystem. When bringing Notability to Googlebook, prior investments in tablets and foldables gave the team an immediate head start. Because their layout already relied on window size classes and adaptive scene strategies, their canvas and toolbars reflowed naturally during window resizing, while existing keyboard and trackpad support carried straight over.Testing and optimizing your app for Googlebook fits naturally into your existing development workflow.
With the desktop emulator in Android Studio, you can run a virtual desktop environment directly on your workstation to test free-form window resizing, verify multi-instance interactions, and debug mouse, trackpad, and keyboard interactions. Download Android Studio Canary to set up your virtual device today.
Help speed up your layout modernization with AI-assisted development. The adaptive skill gives your AI agents the necessary context to help refactor mobile layouts into responsive Compose containers automatically. Install the skill directly through the Android CLI to streamline your implementation.
The Googlebook lineup marks an exciting new chapter for the Android ecosystem, giving your apps a premium platform to deliver richer, more capable experiences. By building adaptively, a single codebase ensures your app looks and performs optimally across phones, foldables, tablets, and Googlebooks while unlocking elevated visibility and badging across Google Play. Explore documentation at our Googlebook developer hub, review the desktop design guide, and start building for Googlebook today!
AI coworkers and coding agents are spreading fast across organizations, and each one arrives with its own view of the business. Agents deployed in isolation lack the semantics and business definitions they need to answer accurately, rely on context that was modeled by hand at setup and has since gone stale, and return answers that contradict other agents pointed at the same data. Without a shared data foundation and business context, you cannot scale agents across an organization with confidence.
The Genie One Model Context Protocol (MCP) server is now generally available to all Databricks users. It gives any agent a single interface to retrieve structured and unstructured data, insights, and answers from Genie One, grounded in governed business context from Genie Ontology. The Genie One MCP now lives within Unity Gateway as a managed MCP Service, providing centralized governance, fine-grained policies, and audit logging across every invocation
What makes the Genie One MCP click for us is that it keeps analysis quality high regardless of which AI tool our teams choose. Some work directly in the Genie One UI; others live in Claude Cowork or their IDE all day. The MCP gives us one integration point that meets them where they already work, so the same trusted, governed answers show up consistently, no matter what tool they're using.—Fenny Sanyoto, Engineering Manager - Growth & Traveler Data Engineering, GetYourGuide
The Genie One MCP exposes Genie One over MCP, allowing any agent to communicate with Genie One as a peer agent.
The MCP exposes tools for asking questions to Genie One, getting query results, checking on incremental progress, and steering responses. The Genie One MCP App allows supported agent clients to embed Genie One’s whole process in real time with interactive visualizations and Genie Ontology citations. These capabilities allow you to integrate Genie One as your data-smart AI coworker into any agent without changing your workflow.
The MCP App provides interactive visualizations and Genie Ontology citations
By serving as a single governed entry point for agentic interactions, the Genie One MCP directly eliminates the friction of agent sprawl. Connected agent clients automatically leverage Genie Ontology via Genie One to interpret domain semantics, bridging structured relational data and unstructured document repositories without requiring custom, per-format connectors. This unified interface ensures that whether users operate within Claude, ChatGPT, Cursor, or custom internal interfaces, every user question yields a consistent answer governed by a single enterprise context layer, while intelligent routing dynamically delegates complex sub-tasks to tailored, domain-specific Genie Agents.
With the Genie One MCP, you can access trusted context from across your data estate and integrate it into any agent workflow. First, you’ll add the Genie One MCP to your agent from Unity Gateway. Once added, you can easily integrate the MCP into your workflows. Here are some popular use cases we’ve seen from our customers so far:
Consider an agent you’ve configured to create presentations: it aligns to your organization’s style guide, knows the expected format your executives prefer, and is popular with teams across your business. But when it’s time to fill those slides with business results, your teams still have to track down the right numbers, reconcile conflicting definitions, and explain what the data means. Now, you can add the Genie One MCP to this agent to bring trusted data and context into your slides, not just create the skeleton deck. While your agent works on the presentation, it kicks off requests to the Genie One MCP to retrieve the right data, which your agent integrates into its presentation.
Customer success teams may create an agent that automatically reaches out to customers based on interesting findings in their product usage patterns. But a drop in usage doesn’t mean the same thing for every customer. Teams still have to investigate what changed and what it means for that account before the agent can send a relevant message.
With the Genie One MCP added in, the agent can query Genie One to investigate usage and fetch trusted telemetry signals based on Genie Ontology. It then passes this data, along with any related context on what the usage might indicate, back to the outreach agent, which goes on to send targeted emails via your CRM.
Engineering teams using coding agents can integrate the Genie One MCP to ground their development in Genie Ontology. For example, if a developer is working on a PR to add logging to a product, their coding agent can make a request to the Genie One MCP to fetch the current definitions and queries associated with that product. This ensures the changes they make align with agreed upon business definitions.
Any time your preferred agent needs access to your governed business data, you can invoke the Genie One MCP to give it the context it needs to take confident action.
Genie One MCP allows you to leverage Genie One as your data-smart coworker from any agent your users prefer. With Genie One MCP, answers across agents stay consistent and grounded in Genie Ontology.
To see the Genie One MCP in action and set it up for your own use cases, review our new blog Genie One MCP: How to give any AI Agent the Right Business Context. To learn more, visit our documentation or contact your Databricks account team for support.
We ran a survey asking users of OpenTelemetry and Prometheus how they collect, process, and store metrics. The goal was to understand, with real usage data rather than assumptions, how far the ecosystem has moved and whether the interoperability still causes friction.
From 186 people who responded, 81 passed our screening for active OpenTelemetry-for-metrics users on a Prometheus-adjacent backend. We also filtered out observability vendor employees to focus on end users. In the analyzed sample:
How easy or difficult is it to use OpenTelemetry and Prometheus together?
This year, we asked the same question as in the similar 2024 survey to see whether end users saw progress in interoperability.
The average rating rose by 0.5 point, from 3.1 to 3.6 — as if every second respondent had moved up a full category. The clearest movement is at the difficult end of the scale: the share of respondents who found the two hard to use together dropped to roughly a third of its 2024 level. Also, nobody this year picked “Very difficult”.
Two years of work on interoperability is paying off. At the same time, since the single largest group of responses sits at “Neither easy nor difficult”, there is still a lot of work to be done in this area.
Note: The 2024 survey didn’t ask respondents whether they worked for an observability vendor, so this is not an exact apples-to-apples population match. However, putting vendor employees back into the 2026 sample (n = 108) would barely change the result for the ease of use rating (0%, 10%, 40%, 33%, 17% → 0%, 10%, 41%, 33%, 16%). To keep this year’s results consistent, we decided to stick with filtering vendor employees out.
How do you instrument infrastructure metrics collection?
Prometheus exporters are the most common single instrumentation method for
infrastructure metrics but OTel receivers are close behind. Built-in /metrics
endpoint, built-in OTLP push, and OpenTelemetry eBPF instrumentation (OBI)
follow.
When looking at how these methods combine, the picture is clearly hybrid, not either/or. Nearly half of respondents are mixing Prometheus and OTel instrumentation styles at once for infrastructure metrics, rather than doing a full migration. Among respondents using a single instrumentation style, Prometheus-only style is twice as popular as OTel-only style.
Note: Instrumentation style describes whether a respondent uses methods
native to one project only, or a mix of both. OTel-style includes using OTel
receivers, Built-in OTLP push, or OpenTelemetry eBPF Instrumentation (OBI).
Prometheus-style includes Prometheus exporters or Built-in /metrics endpoint
(no exporter). The 4 “Other” responses are write-ins: Zabbix, Heorku Telemetry
(likely “Heroku Telemetry”), textfile collector, Telegraf. All 4 respondents
also selected a real Prometheus/OTel method alongside their write-in — but in
the style chart above, a write-in places a respondent in “Other” regardless of
what else they selected.
Work in progress: The Prometheus and OTel communities are working on making Prometheus exporters run as an OTel Collector distribution. The conversations are still ongoing. The discussion is open in this issue.
How do you instrument application metrics collection?
Preferences swap for application instrumentation. OTel SDKs come out on top with Prometheus SDKs following behind them. OBI holds roughly the same share as in infrastructure instrumentation.
Instrumentation styles shift as well. The largest share of participants (41%) use only OTel style instrumentation, nearly twice as common as only Prometheus style. Fewer than a third mix styles.
Note: In application instrumentation, OTel-style includes using OTel SDKs or OpenTelemetry eBPF Instrumentation (OBI). Prometheus-style includes Prometheus SDKs. Again, there are 4 write-ins that we categorized as “Other”: already built exporters, Micrometer, textfile collector, jvm-exporter. 3 of the 4 also selected a real Prometheus/OTel method. One respondent’s original write-ins, “Self instrumentation” and “manual instrumentation for OTEl,” were recoded to plain OTel SDKs.
What do you use to process or transform metrics before sending them to storage?
Prometheus relabeling rules and the open source OTel Collector are the two most common processing steps with neither of them leading clearly.
Most respondents run a vanilla stack: only Prometheus relabeling rules and/or the plain OTel Collector, with no vendor distribution and no custom-built Collector in the pipeline. The three vanilla patterns come out close to even.
Note: “Other” combines respondents who do no transformation at all (15%, n=12) with those using a vendor distribution or custom-built Collector (20%, n=16).
What would you like us to improve to make OpenTelemetry and Prometheus work better together?
We received 19 open-ended responses with suggestions on what to improve. Three themes emerged from this data: unification of Prometheus and OTel’s data models (attributes/labels), better handling of resource attributes and metadata, and naming and formatting friction. There were also a few individual asks. Prometheus maintainers György “Krajo” Krajcsovits and Arthur Sens went through the responses and addressed each point below:
In our data, organizations with 100–999 employees have the highest OTel SDK adoption for application metrics and OTel receiver adoption for infrastructure metrics. eBPF-based instrumentation (OBI) doesn’t follow the same pattern — there, it’s the 1,000+ organizations that stand apart from every smaller band.
Adoption by organization size:
| Organization size | OTel SDKs (application) | OTel receivers (infrastructure) | eBPF / OBI (infrastructure) |
|---|---|---|---|
| 1–49 (n = 10) | 40% | 20% | 20% |
| 50–99 (n = 12) | 58% | 58% | 17% |
| 100–999 (n = 25) | 84% | 76% | 20% |
| 1,000+ (n = 34) | 62% | 53% | 3% |
Our hypothesis is that mid-size organizations — big enough to have a dedicated platform effort, small enough to move without a multi-year migration plan — might be pushing furthest into newer OTel-native tooling.
Note: This is an interesting observation and a hypothesis, not a confirmed finding: with 10–34 respondents per band, none of these gaps is big enough for a survey this size to confirm.
Platform Engineering and SRE teams lean heavily toward OSS Prometheus-compatible backends (Thanos, Cortex, Mimir), while Dev teams lean the other way, toward plain Prometheus.
Here, the dividing line looks like operational ownership rather than preference. Teams running metrics for a whole organization eventually outgrow a single Prometheus deployment, whereas teams instrumenting their own service generally don’t.
Backend choice by team type — OSS Prometheus-compatible (n = 30), Prometheus (n = 35), PromQL-compatible vendor (n = 8):
| Team type | OSS Prometheus-compatible | Prometheus | PromQL-compatible vendor |
|---|---|---|---|
| Dev | 24% | 71% | 6% |
| DevOps | 23% | 62% | 15% |
| Observability | 29% | 41% | 29% |
| Platform Engineering | 69% | 31% | 0% |
| SRE | 69% | 31% | 0% |
Note: Sysadmin (n = 6) and Operations (n = 2) respondents are excluded from this table — both groups are too small to interpret — leaving n = 73 of the 81 respondents. As with the previous breakdown, the per-band numbers here (8 to 35) are too small to draw firm conclusions.
Interoperability is measurably easier than it was two years ago, but the open-ended answers point to concrete gaps — data model differences, resource attributes and metadata gaps, and naming and formatting friction. There is still a lot of work to do on both the OpenTelemetry and the Prometheus side.
Everyone is welcome to contribute. The discussion happens in the #otel-prometheus channel in the CNCF Slack.
Following a successful private preview, we’re thrilled to open DigitalOcean Managed Agents to everyone. Teams can now deploy their preferred agent harness (like OpenCode, Codex CLI) or bring their own, connect agents to 16,000+ tools, and help agents do more work at scale without building or maintaining any infrastructure themselves. With native integration to DigitalOcean’s Inference Engine, Managed Agents brings inference tokens, agent execution, and tool use together, so you can scale your intelligence all in one place. Agents go from session creation to a response in less than a couple of seconds and resume paused work in ~300 milliseconds. With per-second active CPU billing, you pay only for the CPU your agents actually consume. Customers like OpenHands, Qencode, and Amplitude are building and scaling on Managed Agents, get started today.
Developers and teams are asking agents to do increasingly ambitious work: implement features, investigate production issues, build new applications, research across systems, and coordinate subagents across different tasks. Consider an agent investigating a spike in checkout errors: it queries logs across several services through an MCP server, writes and runs a script to reproduce the bug, tests a fix, and opens a pull request for a teammate to review before it ships to production. Querying the logs, running the reproduction script, and testing the fix can each briefly demand substantial CPU and memory. Between those steps, and while it waits on tokens or a human approval, the agent may consume little or no CPU at all. But its context, files, and working state need to stay available the whole time, so it can pick back up exactly where it left off.
Agents working beyond software development use cases also need to execute code and produce artifacts others can use. An agent helping a team plan inventory might read sales datasets and supplier PDFs, run Monte Carlo simulations of demand and delivery delays, and produce reports recommending stock levels. To do that work, it needs an isolated sandbox to install dependencies and execute code that inspects results and generates reports for analysis. The datasets, scripts, and reports must outlive the session that created them, so a teammate can review the recommendations or another agent can update the analysis as new data arrives.
A traditional VM provides an empty computer, and leaves developers to build the environment and APIs that agentic workflows desperately need to get work done. Developers are forced to invest in plumbing work to preserve the agent’s context, persist artifacts and keep them accessible beyond the agent that created them, coordinate parallel work, and security-hardened access to tools. Keeping spare VMs running helps agents start quickly but adds idle cost; provisioning and configuring capacity on demand can take minutes, slowing work. Billing for provisioned CPU also continues while agents wait for model responses, tool results, or human approval. Time spent making VMs work for agents is time developers could spend making those agents better at the work customers care about.
Agentic work needs infrastructure built for it: security hardened code execution, persistent sessions, fast startup, and governed tool access. Checkpointing and forking let that work branch, pause, and continue across devices and teammates. Active CPU billing keeps cost tied to actual consumption. Designed as purpose-built primitives for agents rather than adapted from general purpose virtual machines, Managed Agents lets developers focus on what matters most: making agents capable of more valuable work.
Managed Agents brings together two services vertically integrated to deliver a great agentic experience.
Together, they let developers scale the work their agents can do while DigitalOcean manages the execution, persistence, tool access, and infrastructure underneath. Let’s dive a bit deeper into each of these new services, their capabilities and how they enable you to scale agentic work in the cloud.
Harness Runtime gives agents a durable cloud workspace where they can execute code, work with artifacts, and continue across devices and teammates. It manages the compute, storage, and session lifecycle, so developers can run agents in parallel, explore different approaches, and return to ongoing work without reconstructing the environment or context. The runtime provides these critical capabilities these agents need:
Use coding harnesses such as Claude Code, Codex CLI, and OpenCode, general-purpose agents such as Hermes, or agents built with LangGraph. You can also package a custom agent as a standard OCI container image and turn it into a reusable environment template, bringing your dependencies, tools, and configuration without rebuilding around a DigitalOcean-specific harness.
An agent resolving a production issue might inspect a repository, read a ticket, query a database, and notify the team. Each step requires access to another system. Connecting those tools individually leaves developers managing authentication, permissions, retries, and monitoring across every integration. Action Gateway brings that work behind a single managed MCP endpoint, giving agents governed access to 16,000+ tools across 500+ providers. Connect your services such as GitHub, HubSpot, Stripe, Snowflake, PagerDuty, Box, Supabase, and Exa, alongside web search, browser automation, code execution, and your own MCP servers.
Action Gateway also works with MCP-compatible applications beyond Harness Runtime. Add its endpoint to your application’s MCP configuration to access the tools you’ve connected, with the same centralized permissions and controls
Agents work in bursts. They compile code and run tests, then wait for model responses or external tools. Harness Runtime’s CPU billing follows actual CPU consumption, so when an agent is waiting and consuming no CPU, its CPU charge falls to zero.
For example, a session with two vCPUs averaging 25% CPU utilization and a measured memory peak of 4 GB throughout an hour would cost $0.060 in CPU and memory charges, compared with $0.126 for a full hour of that allocated capacity. Storage, inference, and separately metered tools are additional. Pausing a session stops CPU and memory charges while preserving its stored state. Action Gateway adds first-party tools that require a sandbox using Harness Runtime’s compute and memory rates, while third-party tools follow their published per-use pricing.
Fast startup and resume reduce the tradeoff between responsive agents and idle infrastructure cost. When a coding agent needs an execution environment before it can begin, provisioning delays become part of the user’s wait. When that environment sits idle between tasks or while awaiting human input, keeping it running preserves responsiveness at a cost. Pausing preserves its working state; fast resume makes that state useful again quickly.
The importance of latency depends on where it occurs and how often it repeats. Startup can delay the first answer. Resume can delay the next interaction. Repeated environment transitions can reduce how much exploration or testing an agent completes within a fixed time budget. Our goal is to minimize the time agents spend waiting for infrastructure and make it practical to pause idle sessions.
That is why we measure both runtime readiness and the time to an actual agent response. Through each provider’s public API, we run the same coding agent against the same model through session creation, a first answer, pause, resume, and a second answer.
A fast startup time gets agents to useful work sooner. Create → agent response measures the full journey from a session creation request to a completed agent reply, including provisioning the microVM, starting the harness, and completing a model turn. Harness Runtime becomes ready in 886 milliseconds and delivers the first response in 3.3 seconds in this benchmark. Measuring both makes the infrastructure overhead visible alongside the wait a user actually experiences.
A faster resume makes pausing practical. Developers should be able to pause idle sessions without making the next interaction feel like it’s starting all over. Harness Runtime resumes to readiness in 305 milliseconds. In this benchmark, a resumed session delivers an agent response in 2.43 secs, comparable to the 2.47 seconds measured for an already-running session. These results support using auto-pause to stop compute and memory charges between periods of work while preserving responsiveness when users return. Active-CPU billing addresses a different part of the lifecycle: avoiding CPU charges during model or tool waits when the running agent consumes no CPU.
Command execution is where we still have work to do. Run a command measures a command round trip inside an already-running session: 189 milliseconds for Harness Runtime versus 79 milliseconds for Sprites. Managed Agents routes exec through the DigitalOcean edge and Harness Runtime control plane, providing authentication, authorization, and audit trail. Our measured command path is 110 milliseconds slower. Reducing this overhead while preserving those controls remains a performance priority for us.
† Fly.io Sprites has no resume API - a sprite wakes on its first incoming request so these two figures are derived by removing one steady-state command round trip from its measured resume, not read directly from a resume call.
Source: DigitalOcean internal benchmark, 21 September 2026. Codex CLI in each provider’s native agent mode against gpt-5.5, driven through each provider’s public API from DigitalOcean droplets in RIC1. p50 across an identical number of journeys on every provider, with warm-up runs discarded. Sessions were requested at 2 vCPU / 4 GB on every provider; the Fly.io Sprites guest reported 8 vCPU / 16 GB. Agent CLI versions differed by provider (Managed Agents 0.154.0, Sprites 0.151.0).
From the CLI, starting a session looks like this:
# Authenticate with your DigitalOcean account
doctl auth init
# Start a session. --harness builds the manifest for you and
# prompts for your Anthropic key if it isn't already exported
doctl harness-runtime launch --harness claude-code --name my-first-agent
# You're dropped straight into a chat with the agent.
# Detach any time with Ctrl-D, then reattach later,
# from any device, right where you left off
doctl harness-runtime launch my-first-agent
From your code assistant, use this prompt to create an agent:
Set me up on DigitalOcean Managed Agents and leave me with a working agent.
Docs: https://docs.digitalocean.com/products/managed-agents/ — add index.html.md to any page for the markdown version. I have nothing installed or configured yet, so install doctl and get me authenticated. Never ask me to paste a token or any other secret into this chat.
Use this spec as written. It needs no model key and it attaches the tool catalog:
name: my-first-agent
agent: opencode
tools:
- do.actions
permissions:
default: ask
Then give it a job big enough to take a few minutes — a sourced brief on what shipped this week in AI, written to its workspace. Approve the tool calls for this first run so it can work unattended, and tell me that you did. Don't wait for it to finish: hand me back the commands to check on it, read the file, and pause it.
Understanding an agent’s work should be as simple as starting a run. With DigitalOcean Insights (now in Private Preview), developers can follow a run across Harness Runtime, Action Gateway, and built-in tools in one place: what the agent executed, which tools it called, where it slowed down, and how it reached an outcome. There’s no need to piece together the story across tabs and vendors to understand what happened.
But improving agents requires learning from more than failures. Exceptional runs can reveal effective approaches worth reinforcing, just as unsuccessful runs expose behaviors worth correcting. And Signals (coming soon), will build on this visibility to help developers turn agent runs into feedback for evaluation and reinforcement learning. Together, Insights and Signals will help teams move from seeing what an agent did to understanding what made it effective, so every run becomes an opportunity to improve the next.
Qencode, a media processing company, built a support-triage agent on Harness Runtime. Before automating, their team spent hours every week manually triaging support requests across Slack, email and Intercom.
Today their agent reviews each incoming request, assesses urgency, sentiment and client revenue, and creates or updates the matching Jira ticket, flagging low-confidence cases for a team member to review. Early results suggest it’s saving the team an estimated 4 to 8 hours a week on triage and status reporting, while bringing response times down from several hours to nearly instant.
“It’s been a huge force-multiplier for our team. It gets the right ticket to the right person without anyone having to watch every thread themselves.” — Murad Mordukhay, CEO and co-founder, Qencode
DigitalOcean Managed Agents is now available in public preview. Bring your preferred harness, connect your tools, and give your agents the infrastructure to take on more work. Get started today.
September 22, 2026: Next.js 16.3.6 and 15.5.26 are now available. Read the security update.
A critical security issue has been identified in an upstream dependency. We plan to publish Next.js 16.3.6 and 15.5.26 in an out-of-band update on September 22, 2026.
The full advisory, GHSA-vcvr-r3jv-pc5j, will be published with the update and include impact, affected versions, and upgrade instructions. Upgrade to Next.js 16.3.6 or 15.5.26 as soon as they are available.
We work with security researchers to secure Next.js and other open source frameworks through Vercel's Open Source Bug Bounty. Anyone interested in contributing to the security of eligible frameworks is encouraged to participate there.
Any questions or concerns regarding our security programs or vulnerability management can be sent to security@vercel.com.
Menu. Currently selected: Schedule
We’re removing several SSH algorithms, adding a new algorithm, and requiring larger RSA SSH keys to improve security.
The changes are as follows:
ssh-rsa signature type, including ssh-rsa-cert-v01@openssh.com certificates using SHA-1).diffie-hellman-group-exchange-sha256.mlkem768x25519-sha256 for SSH sessions on github.com and GitHub Enterprise Cloud with Data Residency, except for the U.S. region.Adding ML-KEM lets us offer a newer, more performant key exchange method that is secure against quantum computers.
We’re also removing the older Diffie-Hellman method, a slow, little-used algorithm that could be broken with advances in quantum computing. For RSA, we’re removing the use of SHA-1 since it’s known to be weak, as well as increasing key sizes to align with 128-bit security requirements.
mlkem768x25519-sha256 will be enabled on github.com and GitHub Enterprise Cloud with Data Residency (except for the U.S. region).ssh-rsa signature type (i.e., RSA keys using SHA-1) and the diffie-hellman-group-exchange-sha256 key exchange algorithm.ssh-rsa signature type and the diffie-hellman-group-exchange-sha256 key exchange algorithm.ssh-rsa signature type and diffie-hellman-group-exchange-sha256 key exchange algorithm.These changes will all take effect in GitHub Enterprise Server in version 3.25, except for the addition of mlkem768x25519-sha256, which will take effect in version 3.24.
The only affected users are those connecting with a Git client over SSH or those using the unauthenticated Git protocol on GitHub Enterprise Server. If your Git remotes start with https://, nothing here will affect you.
If you’re using an existing RSA key, make sure you’re using RSA with SHA-2 (i.e., the rsa-sha2-256 and rsa-sha2-512 signature types). You do not need to generate a new key, since all RSA keys are capable of signing with all hash algorithms. As long as the SSH program or library you’re using supports RSA with SHA-2, you can continue to use the same key without a problem and most SSH implementations supporting RSA with SHA-2 will choose it automatically.
Note the distinction between the key type ssh-rsa, which applies generically to all RSA keys regardless of signature algorithm, and the confusingly named signature type ssh-rsa, which indicates an RSA key using SHA-1 (as opposed to rsa-sha2-256 and rsa-sha2-512, which refer to RSA keys using SHA-256 and SHA-512, respectively).
Here’s a list of some common software that uses SSH to connect to GitHub and the version necessary to support RSA with SHA-2 robustly with the default configuration:
| Software | Minimum Version |
|---|---|
| OpenSSH | 7.2p1 |
| JSch | 0.1.66 from this fork |
| TeamCity | 2021.2.3 |
| Go SSH | 0.16.0 |
| libssh2 | 1.11.0 |
| PuTTY | 0.82 |
Alternatively, if you’re using older software and can’t upgrade, you may be able to use an Ed25519 or ECDSA key instead. All Ed25519 and ECDSA keys we support are strong, secure, and will continue to work for the indefinite future.
For generating new keys, we recommend using an Ed25519 key whenever possible. However, if you still need an RSA key for compatibility with other services, you can generate one as long as it as at least 3072 bits in size.
diffie-hellman-group-exchange-sha256If you’re using one of the SSH implementations above that supports RSA with SHA-2, it should also support a strong key exchange mechanism.
The addition of the mlkem768x25519-sha256 shouldn’t require any changes from users. SSH clients will automatically use the new algorithm by default if configured to prefer it. Users who use an older SSH client should automatically fall back to an older key exchange algorithm.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
NVIDIA DLSS 5 introduces DLSS 3D-Guided Neural Rendering and granular controls that help game developers add lifelike lighting and material detail while preserving their artistic intent. We also look at updates to NVIDIA ACE, RTX Mega Geometry 2.0, and RTX Kit across character AI, high-density geometry, and rendering workflows.
This post covers:
DLSS 5 with 3D-guided neural rendering extends the graphics pipeline as a final neural-rendering stage. It uses the game engine’s rendered frame, including its artist-authored geometry, textures, and lighting buffers as an unyielding foundation.
The engine frame defines what must remain, while developers direct what may change. Using frame color and motion vectors, the model is designed to add lifelike lighting and material detail while preserving scene structure, character identity, and artistic intent.
Built specifically for real-time 3D rendering, DLSS 5 delivers deterministic, temporally stable output. Operating on a strict one-frame-in, one-frame-out model with game-engine motion vectors keeps results consistent as players move. The compact, specialized model runs locally on a single GeForce RTX 50 Series GPU at up to 4K.
Controls and integration for developers
DLSS 5 is available now in NBA 2K27, developed by Visual Concepts and published by 2K, for all GeForce RTX 50 Series desktop and laptop GPUs. GeForce NOW Ultimate members can also experience it when streaming from NVIDIA-operated GeForce RTX 5080-powered gaming rigs in the cloud. Visual Concepts uses overall tone and style controls plus a per-pixel uplift control mask to fine-tune character detail while respecting player likenesses.
In NBA 2K27, DLSS 5 preserves scanned facial geometry while enhancing skin subsurface scattering, light transmission through hair and ears, and contact shadows.
For more details about DLSS 5, check out our DLSS 5 article. Sign up to be notified for DLSS updates for developers here.
NVIDIA ACE offers ready-to-integrate AI models and tools for building knowledgeable, interactive, and conversational in-game characters. The latest updates expand the speech pipeline and inference framework, making it easier for developers to run AI in their games.
To run these models locally alongside game graphics, the NVIDIA In-Game Inferencing (NVIGI) SDK delivers a high-performance, streamlined path for deploying local AI models through in-process C++ execution.
Key Release Highlights
NVIDIA In-Game Inference SDK Updates
llama.cpp updates to maximize inference performance.Access NVIDIA In-Game Inference SDK here.
NVIDIA RTX Kit is a suite of rendering technologies for training and deploying AI in shaders, path tracing detailed scenes at game-ready performance, and rendering lifelike digital characters. The latest SDK updates expand support for high-density geometry, neural texture workflows, lighting, and texture filtering.
RTX Kit 2026.3 updates include:
In addition to the RTX Kit updates, RTX Mega Geometry SDK has been updated to 2.0 which adds support for streaming of continuous level-of-detail clusters for high-density meshes. The scale of detail is demonstrated in a newly released textured glTF version of Zorah.
RTX Mega Geometry is coming soon to Gears of War: E-Day, offering GeForce games higher frame rates, higher levels of image quality, and with even more responsive controls. We sat down with the Coalition’s Studio Technical Director, Kate Rayner, and Rendering Lead, Mike Perzel to learn more about Gears of War: E-Day’s integrations of RTX Mega Geometry and DLSS.
Video 3. RTX: Inside the Game | Gears of War: E-Day with DLSS 4.5 and RTX Mega Geometry
Check out the full list of game developer resources and stay up to date with the latest NVIDIA game development news:
Nothing is worse than testing out a change that works in staging, only to see it behave differently in production. That’s why we wanted to give you an environment that’s as close to production as possible — so you can battle-test your changes and make sure they behave exactly as you expect them to.
Agents are helping us push more lines of code than ever before, and larger changes mean more ground needs to be tested ahead of release. Ideally, that testing is done in a way that doesn’t slow agents down, but gives them the tools to take on more of the development lifecycle.
That’s why today we’re launching Worker Previews. Each Git branch gets a production-like place to run, with its own code, configuration, URL, observability, and state.
So now, for every change in your codebase, you can:
npx wrangler preview, using its own variables, secrets, and bindings, separate from production configuration and traffic.main. We call this the base configuration.The result is a pre-production feedback loop for every branch. Push your change to a branch, test behavior, inspect performance — before you merge to production.
This enables an Agent Development Lifecycle (ADLC) where each change is atomic, independently deployable, observable, and revisable. And it gives agents and humans the evidence they need to self-improve: catch what failed, push a fix, and verify the next deployment before it hits production.
When you start work on a new feature, the first thing you do is branch off of main. You get your own copy of the code and make your changes without affecting anything in production.
Worker Previews extend that same model beyond code. Each branch gets its own isolated environment and URL. You can run hundreds of Previews at the same time — each operating independently without affecting other Previews or production.
Production and each Preview have their own configuration — served on their own URL.
When you run npx wrangler preview, the branch gets its own copy of your Previews configuration that you have defined, running on its own URL — all under the same Worker.
In the dashboard, this works like switching branches. Click the breadcrumb next to your Worker's name (it defaults to Production) to see all your Previews:
The dashboard brings every environment into one view. Production sits alongside as many Previews as you need, so contributors can work on separate changes without fighting over a shared staging site. Unlike Wrangler environments, where each environment requires deploying and managing a separate Worker, Previews keep that isolation in one dashboard view.
Each Preview runs as a real version of your Worker. Some changes can only be validated at runtime: an API endpoint has to handle a real request and return the right response. More subjective changes, like a UI update, a new onboarding step, or a different error state, need to be experienced in context before they reach production.
For isolation to extend across your application, stateful resources need special treatment. The reason for that is that Durable Objects run on a singleton model. One instance is responsible for a given object ID, and that instance owns its storage.
If a Preview shared the same DO namespace as production, you wouldn't just be reading stale data — you could modify the same instance serving live traffic in real time (scary!).
That is why every time you run npx wrangler preview, Cloudflare automatically creates a new Durable Object namespace and Container application for that Preview — so that a failed migration or a bad schema change stays contained to that branch and that branch only.
All you need to do is export the class, add its migration, and access it through ctx.exports:
In production, ctx.exports.Counter resolves to the production namespace, while in a Preview, it resolves to that Preview’s namespace.
You now have an entire playground to experiment with. Take Sandboxes, for example, where milliseconds of improvement to startup time can make or break the experience. If you have been trying to improve cold-start performance, you can run different configurations across branches at the same time, compare their cold and warm performance side by side, and find the best setup faster.
Now that each branch runs at its own URL in an isolated environment with its own state, you can enter the feedback loop and start battle-testing every change before it reaches production.
You can send traffic to the Preview URL however you normally would — from your terminal, probe from CI, an agent, or by clicking through it yourself. Once that traffic starts flowing, every Workers Observability tool you’re already used to is available, scoped to each individual Preview.
As each request hits the Preview, Workers Observability traces its full lifecycle in a waterfall, including fetch calls, binding operations, and handler invocations. So when something fails, you can follow exactly what happened without sorting through production traffic or signals from other changes.
Observability for Previews looks just like you're already used to for production Workers. Select your Preview from the breadcrumb and open the Observability tab to see its events, errors, and traces:
To give your agents even more control, you can have them open the Preview URL in a headless browser, click through a login flow step by step, and capture a screenshot or record the entire session as replayable DOM events – with Browser Run.
Below is an example where an agent opens the Preview, captures what was rendered, and connects a failed request to Workers Observability events from the same run.
A reviewer can watch the session in real time with Live View or step in with Human in the Loop when the automation needs judgment.
If something fails, you see it from both angles: what rendered and what happened at runtime.
That gives the agent enough evidence to keep the pre-production loop running autonomously: deploy, open the URL with Playwright MCP, click through, query the traces through the Workers Observability MCP server, patch, redeploy, and verify. Every iteration stays scoped to the branch.
Just like you wouldn't reconfigure your code from scratch every time you branch, you shouldn't have to reconfigure your environment either.
You set base configuration for Previews once, in a previews block in your Wrangler configuration file.
In the dashboard under Worker → Settings, you see this inlined as Production and Previews Base. Once the base is set, run npx wrangler preview from any branch to create a Preview. If your Worker is Git-connected through Workers Builds, it happens automatically on push.
You can override any setting for only one Preview — without affecting production, the base, or other Previews.
To bring the whole setup even closer to production, your preview URLs can be served from your own custom domain. If your app runs on example.com, a Preview for a login branch could run at feature-login.previews.example.com.
If you want to keep those URLs private, you can protect your Previews with Cloudflare Access and require visitors to sign in first.
We’ve already been dogfooding Worker Previews inside Cloudflare, most notably to build and test CloudflareOS, our open-source platform for safely connecting agents to company systems.
CloudflareOS lets agents work with services such as Google, GitHub, and Slack through Gatekeepers, which control what those agents can access and change. That makes Gatekeeper changes especially sensitive, because a bug could expose data or permit an action that should never have been allowed.
Some of these bugs only appear when OAuth callbacks, permissions, approval flows, and application state are running together. Because testing each component separately cannot show us how the complete system will behave, we deploy an isolated Preview of CloudflareOS and its Gatekeepers for every change under review. We then run the full workflow, fix what fails, and test it again before merging.
We’re seeing customers use Previews for the same basic reason: some problems only show themselves when the change is actually running.
"At Supermemory, we use Cloudflare heavily, and Worker Previews are exactly the kind of developer experience improvement we wanted to see. For HTTP flows, we can preview Worker changes before they reach production, including routes backed by Durable Objects, and catch issues earlier without slowing down shipping." — Dhravya Shah, Founder, Supermemory
"Previews is amazing for Inspect [Ramp’s coding agent]. I used it to review and test an Inspect PR on my phone that is making reviewing and testing PRs with Inspect on phones responsive…with Inspect." — Dylan Garcia, Senior Staff Engineer, Ramp
You might be thinking: Didn't Workers already have preview URLs? It’s true, we did. We're now calling those Version URLs because they point to specific uploaded Worker versions. Unlike Worker Previews, they don't create an isolated environment for each branch and could only point to production resources. To learn more and compare the different workflows, check out our docs.
Worker Previews is a big improvement from what we offered before, but there's still more to come. Here's what we're working on next:
Worker Previews are available now. Get started with the docs, and if you have a feature request or run into an issue, open an issue on GitHub or join the Cloudflare Developers community on Discord.
Acknowledgements: This project was made possible by the design and implementation efforts of Greg Brimble, Patrick O’Donnell, Matt Price, Korinne Alpers, Max Peterson, Cina Saffary, Josh Wheeler, Thomas Ankcorn, Matt Rothenberg, and Brandon Strittmatter, with leadership from Brendan Irvine-Broque and Dan Carter.
To build and deploy sophisticated robotics applications that can perceive, reason and act in dynamic environments, developers need new physical AI models and tools.
The ROS open framework is a project from Open Robotics that helps humans build robots. NVIDIA Isaac ROS 5.0 — a collection of GPU-accelerated packages built on ROS, released today at the ROSCon conference in Toronto, Canada — helps humans and AI agents build robots together.
The release introduces new agentic workflows and platform support to help developers build, customize and deploy robotics applications faster.
ROS provides the open source foundation for much of modern robotics development, giving developers common tools, libraries and standards for building and connecting robot applications.
NVIDIA Isaac ROS brings NVIDIA accelerated computing, physical AI models and production-ready libraries to the nearly 1.3 million ROS users, helping developers build high-performance robotics applications using free, familiar, open source tools.
AI agents are changing how software is built, helping developers automate repetitive tasks, navigate complex codebases and move from ideas to working applications faster. Isaac ROS 5.0 brings these capabilities to robotics development.
Isaac ROS 5.0 introduces support for ROS Lyrical and Ubuntu 24.04, giving developers a path to adopt the latest ROS platform while continuing to accelerate demanding robotics workloads with NVIDIA accelerated computing. NVIDIA worked with the Open Source Robotics Alliance to contribute a standard data-handling interface to ROS Lyrical that helps robotics software work efficiently across different computing hardware, including GPUs.
Available to the entire ROS community, it gives developers a consistent way to accelerate demanding robotics applications, with CUDA providing a working example for GPU acceleration.
New NVIDIA Isaac skills for setup and manipulation provide reusable workflows that developers and AI agents can use to complete robotics development tasks. Agent-ready documentation also makes it easier for AI agents to understand Isaac ROS tools and workflows, turning developer intent into working applications faster.
Some skills go beyond assisting with individual coding tasks. A new FoundationStereo fine-tuning skill enables an AI agent to help adapt a stereo perception model to a developer’s cameras, environment and robotics application, so developers can easily achieve more accurate perception for a given sensor configuration.
FoundationPose, a foundation model for object pose estimation and tracking, now provides an agent-ready inference library that enables robots to perceive and track the position and orientation of objects up to 5.5x faster.
In addition, pick and place — a common workflow that connects detection, depth estimation and pose output — is now available as a standalone, agent-ready skill, providing robot developers more flexibility beyond Isaac ROS.
The robotics ecosystem is already extending this agentic approach to development workflows.
AgenticROS, an open source project sponsored by 3D perception technology company RealSense, connects Isaac ROS with NVIDIA Nemotron open models and NVIDIA NemoClaw blueprints, enabling AI agents to interact with ROS-based robots. RealSense is also optimizing its latest AI-native 3D stereo depth cameras, including RealSense D585 Pro, and an open source software development kit for Isaac ROS and the NVIDIA Jetson Thor edge AI platform, helping developers build perception, navigation and manipulation applications.
Intrinsic’s Open Machine Tending Solution is a reference application for computer numerical control machine tending, part of the newly released Intrinsic Core, an open source suite of preconfigured runtime services and capabilities designed to accelerate industrial robotics applications. It includes built-in compatibility with NVIDIA FoundationPose for out-of-the-box object registration, tracking and pose estimation. Using the FoundationPose perception pipeline, the solution enables robots to dynamically detect and handle parts while reducing the need for rigid, costly physical fixtures and specialized systems integration.
Seeed Studio is using NVIDIA Isaac ROS with reBot Arm, combining accelerated perception, spatial understanding and motion planning on NVIDIA Jetson Thor. This integration gives developers a practical platform for building adaptable physical AI applications, from object localization to collision-aware manipulation and autonomous pick and place.
Magna is using NVIDIA Isaac ROS as a modular, GPU-accelerated foundation for robotic perception, synchronized data collection and NVIDIA Isaac GR00T model deployment, pairing it with Isaac Sim hardware-in-the-loop testing to bring intelligent automation from research to real-world manufacturing and mobility — faster and with fewer risks.
Prefix.dev’s Pixi package-management tool makes it easier to create reproducible robot development environments, bringing together ROS with the NVIDIA CUDA platform to help developers more easily set up and share accelerated robotics workflows.
As an Isaac ROS Partner, Foxglove helps developers visualize and debug live ROS applications through its web and desktop tools, which are integrated throughout Isaac ROS tutorials and support data such as 3D topics, nvblox meshes and rosbags.
Flexiv is integrating Isaac ROS with its Rizon 4 adaptive robot, giving developers access to NVIDIA-accelerated robotics capabilities and a streamlined path from testing applications in NVIDIA Isaac Sim to deploying them on a physical robot.
Ekumen, a Grid Dynamics Company, is using GPU-accelerated Isaac ROS packages within existing ROS and Nav2 stacks to improve precision docking, 3D obstacle detection, visual localization and real-time motion planning, validating each application in Isaac Sim.
Ekumen uses isaac_ros_cumotion on a GPU to map a collision-free path for a warehouse arm in roughly 2 to 5 milliseconds.
Ouster integrates its Stereolabs ZED stereo cameras with NVIDIA Isaac ROS to deliver GPU-accelerated perception for robotics applications. The integration simplifies the development of real-time object detection, mapping and navigation while maintaining interoperability with the broader ROS ecosystem.
The applications that developers and agents build ultimately need to run on the robot.
NVIDIA Jetson is a scalable computing platform for running the physical AI stack at the edge with real-time performance, bringing together ROS, accelerated perception and navigation, AI models and application logic on the robot.
Isaac ROS 5.0 supports scalable compute, from entry-level NVIDIA Jetson Orin Nano to high-performance Jetson Thor devices, giving developers a path from development to deployment as robotics workloads become increasingly sophisticated.
Robotics companies are already using this combination to bring more AI processing directly onto their machines.
Mentee Robotics uses NVIDIA Isaac ROS as the perception and AI backbone of its MenteeBot humanoid, enabling the robot to interpret visual information and execute learned behaviors in real time. A shared software foundation across NVIDIA Jetson Orin and Jetson Thor platforms helps Mentee extend its innovations from existing robots to next-generation systems.
Universal Robots has built NVIDIA Isaac ROS into its AI Accelerator software development kit to help integrators deploy advanced perception and motion capabilities faster, without developing complex robotics software from scratch. Powered by NVIDIA Jetson at the edge, the solution enables robots to adapt to parts that are not precisely positioned, reducing reliance on costly fixtures and making manufacturing cells more flexible.
ROBOTIS, which builds the developer-friendly ROS-based TurtleBot3, is integrating Isaac ROS into its AI Worker robot, using GPU-accelerated object perception to enable vision-guided manipulation tasks including picking, placing and alignment.
ROBOTIS performs object manipulation tasks using NVIDIA Isaac ROS CuMotion.
FieldAI’s robot foundation models, which can run entirely on robots without relying on cloud connectivity, are integrating Isaac ROS on Jetson devices to take greater advantage of GPU acceleration and improve the efficiency of the on-robot AI stack.
Noble Machines is using NVIDIA Isaac ROS on Jetson to accelerate the development of general-purpose robots for industrial applications, building on ready-to-use AI and perception capabilities rather than creating them from scratch.
By combining an open robotics ecosystem, accelerated computing and new agentic development workflows, Isaac ROS 5.0 helps developers address both sides of the physical AI challenge: building increasingly capable robot applications and efficiently running them in the physical world.
Available now, Isaac ROS 5.0 is free and open source. Developers can learn more and get started with NVIDIA Isaac ROS on GitHub.
claude-opus-5-5), a model for long-running agentic coding and knowledge work. It has a 1M token context window by default, 128k max output tokens, and always-on adaptive thinking, at $4 / $20 USD per MTok (Claude Opus 5 is $5 / $25). Claude Opus 5.5 is available on the Claude API, Claude in Amazon Bedrock, Claude Platform on AWS, Claude on Google Cloud, and Claude in Microsoft Foundry. See What's new in Claude Opus 5.5 for capabilities, API changes, and migration guidance.thinking: {"type": "disabled"} and thinking: {"type": "enabled", ...} return a 400 error. Omit the thinking field and control thinking depth with the effort parameter. tool_choice types any and tool also return a 400 error, as on Claude Fable 5.1; use auto with strict tool use. On the Claude API and Google Cloud, computer use on this model requires the computer_toolset_20260801 toolset and the earlier computer_20251124 tool returns a 400 error; on Amazon Bedrock, computer_20251124 keeps working. See the migration guide.inline-tools-2026-09-15 beta header. A tool_addition block can carry the tool's full definition (tool: {"type": "tool_definition", "definition": {...}}), so you can add a tool, change its schema, or move a server tool to a newer version without editing tools or invalidating the prompt cache. The same header covers adding and removing tools by reference. With the MCP connector's mcp-client-2026-09-15 beta header as well, the definition can be an MCP toolset, and a response records each server's fetched tool list in an mcp_tool_listing block, which pins that list when you send it back.Claude Opus 5.5 from Anthropic is now available on AI Gateway. It is a step-change improvement over Opus 5, with its biggest gains in agentic coding, long-running agent tasks, and knowledge work. Anthropic cites that Opus 5.5 performs at the level of Fable 5.1, but ~30% faster and ~40% cheaper than Opus 5 per task.
Opus 5.5 is also a better collaborator over long runs. It reports back in plain language on what it did, what it found, and what it needs next, making it easier to supervise work that spans many steps or takes place over a longer period.
Opus 5.5 includes two API changes that can turn previously valid requests into HTTP 400 errors:
Thinking is always adaptive. Requests that disable thinking or set a fixed thinking budget are rejected. The model decides how much to think for each request. Use effort and prompting to steer its thinking behavior.
Forced tool use is retired. Requests cannot require a tool call or force a specific tool. Prompt the model toward the tool, then catch and retry misses in your harness. If you previously forced a tool call to return JSON, use structured outputs instead.
Use anthropic/claude-opus-5.5 across the AI SDK, OpenAI-compatible Chat Completions API, Anthropic Messages API, and coding agents connected to AI Gateway. You can also enable fast mode with the gateway speed option r anthropic/claude-opus-5.5-fast. The model has a 1M-token context window, returns up to 128K tokens, and has a June 2026 knowledge cutoff.
Regional inference and Zero Data Retention are opt-in request controls. This example pins inference to the US and ZDR:
Install the latest Vercel CLI and connect your supported coding agents to AI Gateway:
Then select anthropic/claude-opus-5.5 in the agent. In Claude Code, use /fast to toggle fast mode for the session. See the coding agents guide for other agent-specific instructions.
AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, budgets for API keys, routing rules, and more.
AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests.
Try Claude Opus 5.5 in the model playground, or view all language models available on AI Gateway.
Anthropic’s Claude Opus 5.5 model is now available through Netlify’s AI Gateway and Agent Runners with zero configuration required.
Use the Anthropic SDK directly in your Netlify Functions without managing API keys or authentication. AI Gateway handles everything automatically. Here’s an example using Claude Opus 5.5:
import Anthropic from '@anthropic-ai/sdk';
export default async () => {
const anthropic = new Anthropic();
const response = await anthropic.messages.create({
model: 'claude-opus-5-5',
max_tokens: 4096,
output_config: { effort: 'medium' },
messages: [
{
role: 'user',
content: 'How can AI improve my coding?',
},
],
});
return new Response(JSON.stringify(response), {
headers: { 'Content-Type': 'application/json' },
});
};
Claude Opus 5.5 is also available across Background Functions, Scheduled Functions, and Edge Functions. You get automatic access to Netlify’s caching, rate limiting, and authentication infrastructure.
Learn more in the AI Gateway documentation and Agent Runners documentation.
AWS now offers Claude Opus 5.5, Anthropic’s most capable Opus model yet and, the first of the Claude 5.5 model family, a better collaborator that handles long-running coding and knowledge work, reporting back clearly on what it did, what it found, and what it needs next.
According to Anthropic, Claude Opus 5.5 completes tasks using fewer tokens than Claude Opus 5, at a lower price per token, with cheaper cache reads stacking on top of the efficiency gain. Claude Opus 5.5 is the enterprise workhorse, a clear step up from Opus 5 on the work teams count on Opus to do. It handles long-running coding and knowledge work, and reports back like a good teammate, surfacing what it did, what it found, and what it needs from you. It thinks adaptively on every request, deciding how much effort each task needs.
Customers have two ways to access Claude Opus 5.5: Amazon Bedrock and Claude Platform on AWS: Amazon Bedrock gives you Opus 5.5’s advanced capabilities with zero data retention (ZDR) support by default. It keeps your data within AWS infrastructure with regional data residency and provides access through a unified service with AWS-managed features like Guardrails and Knowledge Bases. To learn more, see the Amazon Bedrock documentation and regional availability.
Claude Platform on AWS gives you direct access to Anthropic's native platform experience and capabilities via the AWS Console. Build, test, and deploy with the same APIs, features, and console experience you'd get working with Anthropic directly, unified with AWS billing and authentication. To get started, see the Claude Platform on AWS documentation.
AWS GovCloud (US) now offers Claude Opus 5.5, Anthropic’s most capable Opus model yet and, the first of the Claude 5.5 model family, a better collaborator that handles long-running coding and knowledge work, reporting back clearly on what it did, what it found, and what it needs next.
According to Anthropic, Claude Opus 5.5 completes tasks using fewer tokens than Claude Opus 5, at a lower price per token, with cheaper cache reads stacking on top of the efficiency gain. Claude Opus 5.5 is the enterprise workhorse, a clear step up from Opus 5 on the work teams count on Opus to do. It handles long-running coding and knowledge work, and reports back like a good teammate, surfacing what it did, what it found, and what it needs from you. It thinks adaptively on every request, deciding how much effort each task needs.
Amazon Bedrock gives you Opus 5.5’s advanced capabilities with zero data retention (ZDR) support by default. It keeps your data within AWS infrastructure with regional data residency and provides access through a unified service with AWS-managed features like Guardrails and Knowledge Bases. To learn more, see the Amazon Bedrock documentation and regional availability.
AI models are increasingly taking on work that extends far beyond a single prompt: building a feature across a codebase, investigating a complex issue, synthesizing hundreds of pages of information, or working through a multi-step business process.
As that work gets longer, raw intelligence is only part of what matters. The model also needs to stay focused, make good decisions along the way, communicate what it is doing, and produce work that people can quickly review and use. Today, Claude Opus 5.5 is available in Microsoft Foundry, bringing Anthropic’s most capable Opus model to developers and enterprises building AI applications and agents.
Claude Opus 5.5 is designed for everyday complex work. It advances Opus 5 across agentic coding, knowledge work, and long-running tasks while making it easier for people to understand what the model did, what it found, and what it needs next. Claude Opus 5.5 also does more with fewer tokens. Lower per-token prices and much cheaper cache reads stack on top of the efficiency gains.
Built for work that takes time
Writing a function is one thing. Building a feature that touches multiple services, tracing a production issue across a large repository, or carrying a task from investigation through implementation and validation is something else entirely. Claude Opus 5.5 is designed for these longer-running workflows.
For software development, it can work through long-running coding tasks such as building features across a codebase, debugging, refactoring, and reviewing code. It finds the root cause before changing anything, checks its work as it goes, and explains its changes in plain language, so engineers can review and trust them quickly.
That combination becomes particularly valuable when developers use models through agentic coding environments, where a session may involve dozens of steps and run for an extended period of time. The same applies beyond software development.
For knowledge workers, Claude Opus 5.5 can bring together information from multiple sources, work through long documents and spreadsheets, perform analysis, and help create artifacts such as memos, reports, and presentations. Compared with Opus 5, it produces outputs that require less editing before they are ready to share.
An AI model that communicates more like a teammate
As agents take on more autonomous work, another challenge emerges: keeping the human in the loop without overwhelming them.
An agent that performs 50 steps should not require someone to inspect 50 steps to understand whether the work was successful. Claude Opus 5.5 introduces improvements to agentic communication designed to make long-running work easier to follow. As it works, the model can surface the information that matters most:
The goal is simple: spend less time decoding what the model did and more time using the result. This matters particularly for enterprise agents, where users need to understand not only the final answer but also when an agent needs clarification, encounters a constraint, or reaches a decision point that requires human judgment.
Adpative thinking
Claude Opus 5.5 uses adaptive thinking, automatically determining how much reasoning a task requires. Rather than turning thinking on or off or manually specifying a thinking-token budget, developers use effort to influence how much work the model should put into a request. This allows the model to adapt its reasoning to the task at hand—from relatively straightforward requests to complex problems that require deeper analysis.
For developers building agents, this can reduce the amount of application logic needed to decide when and how a model should reason.
Designed for long-running agent architectures
Long-running agents create challenges beyond model intelligence. Conversations can exceed context limits. Tools available to an agent can change. Applications may need to compact earlier context while preserving the model’s understanding of the work already completed.
Alongside Claude Opus 5.5, Anthropic is introducing beta API capabilities designed for these scenarios, including asynchronous compaction, keep-tail compaction, and changing tools during a conversation while preserving thinking and prompt caching. These capabilities can help agent developers maintain continuity across longer tasks without rebuilding the state of the application every time context or available tools change.
Combined with Microsoft Foundry, developers can use Claude Opus 5.5 as part of broader agent systems that connect models with enterprise data, tools, evaluation, and operational workflows.
Expanded safeguards for more capable models
As model capabilities increase, Anthropic is also expanding the safeguards applied to Claude Opus 5.5.
Claude Opus 5.5 is the first Opus model to use safety classifiers like those introduced with Claude Fable 5.1 in areas including cybersecurity, biology, AI development, and distillation.
For common developer, educational, and knowledge-work scenarios, customers can continue using the model for tasks such as identifying software vulnerabilities or learning about biological concepts. Certain requests that Anthropic identifies as higher-risk or dual-use may be handled by another Claude model with the appropriate safeguards.
This reflects an increasingly important part of deploying more capable models: advancing what models can do while applying safeguards appropriate to the capabilities they introduce.
Pricing
| Model | Deployment Offers | Input/M Tokens | Output/M Tokens | Availability |
| Claude Opus 5.5 | Global Standard, US DataZone | $4Cache Hit – $0.20Cache Write – $5Cache Write (1 hr) – $8 | $20 | GA, Hosted on Azure |
| Claude Opus 5.5 (Long Context) | Global Standard, US DataZone | $4Cache Hit – $0.20Cache Write – $5Cache Write (1 hr) – $8 | $20 | GA, Hosted on Azure |
Build with Claude Opus 5.5 in Microsoft Foundry
Choosing a model is only the beginning of putting AI into production. Microsoft Foundry gives developers a unified place to discover models, build and evaluate AI applications and agents, connect them with enterprise data and tools, and operate those systems in production. As models become capable of taking on more complete units of work, the question is shifting from Can the model answer this prompt? to Can I trust it to carry the work forward? Claude Opus 5.5 represents another step in that direction: stronger performance on complex work, more adaptive reasoning, and clearer communication between people and the AI systems working alongside them.
Claude Opus 5.5 is available today in Microsoft Foundry.
Updated Sep 22, 2026
Version 2.0
The post Claude Opus 5.5 comes to Microsoft Foundry for long-running coding and knowledge work appeared first on Microsoft Azure Blog.
Menu. Currently selected: Availability in GitHub Copilot
Claude Opus 5.5, Anthropic’s newest Opus model, is now available in GitHub Copilot. You can use it for agentic coding, long-running agentic tasks, and knowledge work. In early testing, Opus 5.5 resolved tasks comparably to Claude Opus 5 while using significantly fewer steps and tokens. It also quickly recovered from errors in multistep tasks.
Claude Opus 5.5 watermarks its text outputs. The watermark doesn’t change the meaning, quality, or readability of outputs, nor does it add any tokens or cost. To learn more visit Anthropics’s How Claude’s text watermark works.
This model is billed at provider list pricing under usage-based billing. See Models and pricing for GitHub Copilot for details.
Claude Opus 5.5 is available to Copilot Pro+, Max, Business, and Enterprise users. You can select the model in the model picker in:
Rollout will be gradual. Check back soon if you don’t see it yet.
Copilot Enterprise and Copilot Business plan administrators can manage access to Claude Opus 5.5 through the model policy in Copilot settings. Under default model enablement, new models are automatically enabled unless an administrator has turned off the global default or explicitly disables this model.
To explore all models available in GitHub Copilot, see our documentation on models and get started with Copilot.
Join the GitHub Community to share your feedback.
Menu. Currently selected: Availability in GitHub Copilot
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Today, we’re excited to announce the availability of Claude Opus 5.5 on Amazon Bedrock and Claude Platform on AWS, the first of the Claude 5.5 model family. Claude Opus 5.5 is Anthropic’s most capable Opus model suitable for agentic coding, knowledge work, and long-running tasks.
This post covers Claude Opus 5.5’s improvements, practical guidance, and how to start building with the model on Amazon Bedrock.
According to Anthropic, Claude Opus 5.5 does more with fewer tokens than Claude Opus 5, and new pricing passes those gains straight to customers. Lower per-token prices and much cheaper cache reads stack on top of the efficiency gains. The result is an average lower cost per task than Claude Opus 5, so teams can run more ambitious agentic work at scale.
Claude Opus 5.5 is trained to communicate more clearly. As it works, it surfaces what it did, what it found, and what it needs, making long-running tasks easier to follow. Adaptive thinking is always on, and Opus 5.5 decides how much reasoning each task needs. You can use effort as your control instead of manual thinking budgets.
Claude Opus 5.5 is the first Opus model that comes with safety classifiers similar to Claude Fable 5.1 in biology, cyber security, and AI development. Requests will be refused more frequently as compared to previous Opus versions.
Claude Opus 5.5 capabilities are a good fit for industries where consistency and depth matter most. In software development, Opus 5.5 is an improvement over Opus 5 for longer-running sessions with clear communication and explainability, making it easier to use, review, and trust. For knowledge work, it requires fewer corrections compared to Opus 5 while working with and creating long documents and reports.
To try Claude Opus 5.5, open the Amazon Bedrock console, choose Test, then Playground, and select Claude Opus 5.5 as the model. From there, you can run a prompt directly against it.
Figure 1: Selecting an Anthropic Claude model in the Amazon Bedrock console Playground
Figure 2: Running a prompt against a Claude model in the Amazon Bedrock console Playground
Programmatically, you can call the model with the Anthropic Messages API against bedrock-runtime and bedrock-mantle (through the Anthropic SDK). You can also stay on the Invoke and Converse APIs on bedrock-runtime through the AWS Command Line Interface (AWS CLI) and AWS SDK.
pip install boto3.pip install anthropic.pip install aws_bedrock_token_generator.bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream.Here’s a quick example using the AWS SDK for Python (Boto3):
import boto3
import json
# Create a Bedrock Runtime client
bedrock_runtime = boto3.client(
service_name="bedrock-runtime",
region_name="us-east-1"
)
# Invoke Claude Opus 5.5
response = bedrock_runtime.invoke_model(
modelId="global.anthropic.claude-opus-5-5",
contentType="application/json",
accept="application/json",
body=json.dumps({
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "An S3 bucket serves 40 TB/month egress. Estimate the monthly egress cost at $0.09/GB, and state one architecture change to cut it. Show the calculation, keep it under 120 words."
}
]
})
)
result = json.loads(response["body"].read())
# Opus 5.5 is a reasoning model: the response may include a thinking block
# before the text block, so select the text block rather than a fixed index.
print(next(b["text"] for b in result["content"] if b["type"] == "text"))You can also use the Amazon Bedrock Converse API for a unified multi-model experience:
import boto3
# Create a Bedrock Runtime client
bedrock_runtime = boto3.client(
service_name="bedrock-runtime",
region_name="us-east-1"
)
# Invoke Claude Opus 5.5
response = bedrock_runtime.converse(
modelId="global.anthropic.claude-opus-5-5",
messages=[
{
"role": "user",
"content": [
{
"text": "Can you explain the features of Amazon Bedrock?"
}
]
}
],
inferenceConfig={
"maxTokens": 4096
}
)
if 'output' in response:
blocks = response['output']['message']['content']
print('\n'.join(b.get('text', '') for b in blocks if 'text' in b))You can also use the Anthropic Messages API through the anthropic SDK package for a streamlined experience:
from anthropic import Anthropic
from aws_bedrock_token_generator import provide_token
token = provide_token(region="us-east-1")
client = Anthropic(
base_url="https://bedrock-runtime.us-east-1.amazonaws.com/anthropic",
api_key=token,
)
# Invoke Claude Opus 5.5
response = client.messages.create(
model="global.anthropic.claude-opus-5-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Can you explain the features of Amazon Bedrock?"}],
)
print(response)You can explore the Getting Started notebook for more examples. You can monitor usage, performance, and costs through Amazon CloudWatch and AWS Cost Explorer to scale your applications as demand grows.
Claude Opus 5.5 is available today on Amazon Bedrock through the US Geo CRIS (us.), EU Geo CRIS (eu.), AU Geo CRIS (au.), JP Geo CRIS (jp.) and Global CRIS (global.) inference profiles on bedrock-runtime. The model also runs in the US East (N. Virginia) Region (us-east-1), AP SouthEast (Melbourne) Region (ap-southeast-4) on bedrock-mantle.
See the Amazon Bedrock documentation for the full list of supported Regions. For pricing information, see Amazon Bedrock pricing. It is also available through Claude Platform on AWS in North America.
Give Claude Opus 5.5 a try in the Amazon Bedrock console, in Claude Platform on AWS, or explore the Getting Started notebooks on GitHub.
Aamna is a Senior Specialist Solutions Architect for Generative AI focusing on Anthropic models and operationalizing and governing generative AI systems at scale on Amazon Bedrock. She helps ISVs solve their challenges, embrace innovation, and create new business opportunities with Amazon Bedrock.
Dani is a Sr GenAI Specialist Solutions Architect at AWS and the SA lead for Amazon Bedrock Knowledge Bases. He helps enterprises across the world design and deploy generative AI solutions using Amazon Bedrock and Anthropic’s models and capabilities to build scalable, production-ready applications.
Sofian is a technology leader with over 12 years of experience building AI solutions, and leading high-performing teams to maximize customer outcomes. He is passionate about empowering diverse talents to drive global impact and achieve their career aspirations.
Eugenio is a Sr. Product Marketing Manager for Amazon Bedrock at AWS. With several years of experience in generative AI, he helps customers navigate the evolving landscape of foundation models and generative AI to adopt solutions that deliver measurable value.
Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS generally available
(GA): Released our next-generation text-to-speech (TTS) audio models and
the Gemini API Voices endpoint (/v1beta/voices):
gemini-3.8-flash-tts): Flagship creative TTS model engineered for
studio-grade voice fidelity, nuanced acting, regional dialects, and
long-form multi-turn stability.gemini-3.8-flash-lite-tts): Fast, cost-efficient TTS model built to
replace gemini-3.1-flash-tts-preview for high-throughput production
and real-time voice agent cascades.See the Text-to-speech guide to get started.
A file in a bucket is just bytes; when you upload it, there is often a job to do next with that file, and that job usually involves Postgres - a files row, a status, a thumbnail key. That is a perfect Neon Functions job; the missing piece was something to start the Function when the object appeared, without extra application code watching the bucket.
If you store the files in Neon Object Storage, you can now create a storage_object_created function trigger. It tells Neon: “when an object is created in this bucket, invoke this Function”. The Function runs next to your database and buckets, in the same region; it receives the bucket name and object key, and then does the job you wrote.
Let’s take a closer look:
The logic is simple:
images/ ignores objects under documents/.POST. You don’t keep compute running to poll the bucket.The JSON body names the trigger and the object:
{
"version": 1,
"invocation_id": "abc123FUPHOw0Pl1ZooidgpJhvHaShi1aX40cQ0b321",
"trigger": {
"type": "storage_object_created",
"id": "trigger-1a2b3c4d-5e6f-7890-abcd-ef1234567890",
"name": "on-upload"
},
"data": {
"bucket_name": "uploads",
"object_key": "images/dog.jpg"
}
}From data.bucket_name and data.object_key, the Function can fetch the object, process it, call another service, and write results to Postgres.
Two properties worth noticing:
This is a simple trigger conceptually but extremely useful in practice. These are just a few ways we’ve been using it recently, as we tested the beta:
# Prompt you agent
Create a Neon Function that records uploads in Postgres, then trigger it on new uploads.
Docs: https://neon.com/docs/compute/functions/triggers/object-storage.md
- One unauthenticated POST route. Read data.bucket_name and data.object_key; keep it idempotent.
- Upsert a row into a `files` table (bucket, object_key, status) using the injected DATABASE_URL.
- Deploy it, then create a storage_object_created trigger on the "uploads" bucket. Upload a file to confirm a row appears.The smallest useful pipeline is a catalog: the object lives in the bucket, and the rest of your app needs to know it exists. You can define a trigger so on each create, the Function inserts a row: bucket, object key, maybe a status. From then on, you query Postgres instead of listing the bucket.
# Prompt your agent
Create a Neon Function that makes web-ready image variants, then trigger it on new uploads.
Docs: https://neon.com/docs/compute/functions/triggers/object-storage.md
- One unauthenticated POST route. Read data.bucket_name and data.object_key; keep it idempotent.
- Fetch the object, produce a WebP thumbnail and a full-size variant, write them under processed/ in the same bucket, and record the derived keys in Postgres.
- Deploy it, then create a storage_object_created trigger scoped to the prefix "originals/" so it doesn't process its own output.An uploaded image rarely has the exact format and dimensions every part of an application needs. You could define a function that:
Use a key prefix to keep the pipeline bounded. A trigger that watches originals/ can write results to processed/ without invoking itself again.
#Prompt your agent
Create a Neon Function that describes and tags uploaded images, then trigger it on new uploads.
Docs: https://neon.com/docs/compute/functions/triggers/object-storage.md
- One unauthenticated POST route. Read data.bucket_name and data.object_key; keep it idempotent.
- Fetch the image, call the Neon AI Gateway to generate alt text and a few tags, and write them to the file row in Postgres.
- Deploy it, then create a storage_object_created trigger on the bucket. Upload an image and check the row for alt text and tags.The function could also send an uploaded image through Neon AI Gateway, generate alt text, and save that text next to the file's metadata in Postgres. It can also tag or categorize the upload. Once those tags are columns or rows, the app can ask for every file tagged dog without scanning the bucket.
# Prompt your agent
Create a Neon Function that makes uploaded PDFs searchable, then trigger it on new uploads.
Docs: https://neon.com/docs/compute/functions/triggers/object-storage.md
- One unauthenticated POST route. Read data.bucket_name and data.object_key; keep it idempotent.
- Fetch the PDF, extract and chunk the text, embed each chunk via the Neon AI Gateway, and write chunks + vectors to Postgres for Lakebase Search.
- Deploy it, then create a storage_object_created trigger scoped to the prefix "docs/".For a PDF, the Function can extract the text, split it into chunks, generate embeddings through the AI Gateway, and write the chunks and vectors to Postgres. Lakebase Search then queries those embeddings for semantic or hybrid search. For an audio upload, it can transcribe the recording and save the transcript, ready to index or attach to the object.
# Prompt your agent
Create a Neon Function that moderates uploads, then trigger it on new uploads.
Docs: https://neon.com/docs/compute/functions/triggers/object-storage.md
- One unauthenticated POST route. Read data.bucket_name and data.object_key; keep it idempotent.
- Fetch the object, run moderation, and if it violates policy, delete or move it to a quarantine prefix and log the decision in Postgres.
- Deploy it, then create a storage_object_created trigger on the "incoming/" prefix of a private bucket.Run content moderation as soon as a file arrives. If it violates your service policies, the function can quarantine or delete it and record the decision in Postgres.
The function could also redact names, Social Security numbers, and email addresses from uploaded documents, or blur faces in images for privacy requirements. Keep unreviewed uploads in a private bucket or prefix while processing. An object-created trigger runs after the object is created, so it should not be treated as a gate that prevents the original upload from landing.
The entire Neon backend is branch-scoped; of course this includes Object Storage, Functions, and Function Triggers.
A child branch gets its own view of the bucket and its objects, its own function URL, and an inherited copy of the trigger. But inherited triggers are disabled on the child by default; this prevents a dev branch from processing the same inherited files again as soon as it is created.
If you want to test the upload pipeline, enable the trigger in that test branch. All test uploads and the resulting Postgres writes will then stay on the child, without changing the parent.
If you’re setting this up by hand, the cleanest path is config as code - one neon.ts file can declares the bucket, the Function, and the trigger together, and neon deploy provisions all three:
import { defineConfig } from "@neon/config/v1";
export default defineConfig({
buckets: {
uploads: { access: "public_read" },
},
functions: {
processupload: {
name: "Process upload",
source: "./functions/process-upload.ts",
},
},
triggers: {
"on-upload": {
type: "storage_object_created",
function: "processupload",
bucket: "uploads",
},
},
});Everything branches together from here. Create a branch and the child gets its own bucket, its own Function, and an inherited copy of the trigger, ready to enable when you want to test.
The fastest way to start is to hand the job to your agent. Pick one of the prompts above, point it at our docs, and build your first pipeline.
Starting with CodeQL CLI 2.27.0, the all-platform CodeQL bundle (i.e., codeql-bundle.tar.gz and codeql-bundle.tar.zst), which includes the binaries for all supported platforms up to this release, is marked as deprecated.
In mid-March 2027, we will remove the all-platform CodeQL bundle. Download the platform-specific bundle for your supported operating system and architecture instead. Linux ARM64 binaries are available only through platform-specific downloads and will not be included in the all-platform bundle. To learn more, see the documentation about supported platforms.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Back to top
Antoine du Hamel
c0a42d23e5] - (SEMVER-MINOR) crypto: add crypto.parsePKCS12() (Brian Muenzenmeyer) #65627eb4fabe81e] - doc: add araujogui to collaborators (Guilherme Araújo) #6609064fb33d791] - (SEMVER-MINOR) ffi: load libraries from a mounted VFS (Matteo Collina) #659092b1701f810] - (SEMVER-MINOR) fs: add openAsBlobSync (greenhead) #65644080e76b3d7] - (SEMVER-MINOR) net: support sending net.BoundSocket to threads and child processes (Guy Bedford) #6472513e61f6ae6] - (SEMVER-MINOR) perf_hooks: implement SlidingWindowHistogram (James M Snell) #65825a326546094] - (SEMVER-MINOR) perf_hooks: implement qrde analysis support in Histogram (James M Snell) #658060306b0a71e] - (SEMVER-MINOR) sqlite: bind undefined to NULL (Trevor Burnham) #657093c999edef7] - (SEMVER-MINOR) src,lib: add util.markPromiseAsHandled (James M Snell) #65805f7d18ec360] - (SEMVER-MINOR) test: expand histogram test coverage (James M Snell) #658253ce4d23bbb] - (SEMVER-MINOR) util: implement util.throttle (James M Snell) #65899336f33ccc1] - (SEMVER-MINOR) util: implement debounce (James M Snell) #658992bf082453b] - assert: fix TypeError on deepStrictEqual with null Map key or Set member (Sergey Sannikov) #64449c518831d00] - benchmark: add --csv option to compare.js with --analyze (James M Snell) #6592289bae46e68] - buffer: fix unaligned UTF-16LE decoding (Matteo Collina) #6590501cee9dcef] - build: do not use bundled simdutf when built with --shared-simdutf (Antoine du Hamel) #6589120606a4d19] - build: suppress OpenSSL asm warnings with clang (Richard Lau) #66023a76e97b5c5] - build: derive V8_LOGGING_LEVEL from dcheck_always_on (Joyee Cheung) #65744ac7fcf0cef] - build: sync cargo/rustc version warnings (Richard Lau) #65912f24a495b36] - build: fix quiet default for make builds (Shelley Vohr) #65826c4381765fe] - child_process: clear timeout timer on spawn-time error too (kishore280) #655068955781eeb] - crypto: remove redundent std::move call (Antoine du Hamel) #661110b88aa5a15] - crypto: read RSA-PSS restrictions from provider (Filip Skokan) #66108f90d3d3cb0] - crypto: skip private RSA parameters in key details (Filip Skokan) #6610897828d91db] - crypto: use provider EC group names (Filip Skokan) #661083de05da569] - crypto: fetch ciphers for private-key encoding (Filip Skokan) #66108ca8ee04594] - crypto: decode PKCS#1 keys through providers (Filip Skokan) #6610854a625e953] - crypto: derive keys through EVP_KDF (Filip Skokan) #66108971adece6a] - crypto: use names for asymmetric key algorithms (Filip Skokan) #65966bfb544eb00] - crypto: optimize and benchmark key preparation (Filip Skokan) #65892c0a42d23e5] - (SEMVER-MINOR) crypto: add crypto.parsePKCS12() (Brian Muenzenmeyer) #65627c58a4e0ba7] - crypto: add Hybrid KEMs to Web Cryptography (Filip Skokan) #6575970edf90851] - crypto: avoid EC reconstruction for signature sizing (Filip Skokan) #65908fab5dddd77] - crypto: avoid EC raw export reconstruction (Filip Skokan) #65908220a499614] - crypto: export EC JWK coordinates directly (Filip Skokan) #65908f1295e11db] - crypto: read EC curve metadata directly (Filip Skokan) #65908775173a5db] - crypto: optimize private EC JWK import (Filip Skokan) #65908e44669bb7d] - crypto: validate the limit of PBKDF2 iterations (Filip Skokan) #65704cd5a7611e9] - crypto: use primordials in HKDF info validation (Filip Skokan) #6570494affce749] - Revert "deps: V8: override depot_tools version" (Richard Lau) #66110870a366c49] - deps: V8: cherry-pick 95efbaf92a0d (Igor Sheludko) #66020d62a452738] - deps: V8: backport a0607c5006b8 (Antoine du Hamel) #65891e8edeffe2d] - deps: update googletest to 8eff9e336692fc95961e096564f1044c600b881d (Node.js GitHub Bot) #66009eb4fabe81e] - doc: add araujogui to collaborators (Guilherme Araújo) #660902fffb3872d] - doc: fix duplicate 'the' typo in node_platform.cc (Muhammad Al-Muzahid) #66058317abb8131] - doc: clarify sub-1000ms behavior in socket.setKeepAlive (Haram Jeong) #658691c12f815c2] - doc: remove obsolete mentioning of cl.exe on windows (Chengzhong Wu) #660382cc17dd67c] - doc: note that default signal handling resets the signal mask (Shelley Vohr) #658773a69cc1bf4] - doc: clarify QUIC async write backpressure (John Finnerty) #65947549694349d] - doc: clarify permission model scope for output paths (Rafael Gonzaga) #660043325ded3a8] - doc: fix sign-off format in AGENTS.md (Filip Skokan) #6595889aa2bd10c] - doc: fill in missing zstd docs (James M Snell) #658678d1ce4d477] - doc: add inoway46 as triager (Yuya Inoue) #655659a3697c5c7] - doc: document windowsHide for child_process.fork (Christopher Buss) #6588769911bf6b5] - doc: qualify directory read ordering for native fs (Trivikram Kamat) #658682d0c3fed12] - doc: add DOMException section to errors API reference (Avocado) #65206f744023cce] - doc: expand revert commit collaborator instructions (Mike McCready) #65848412399efd3] - doc: clarify supported Python releases (Mike McCready) #6585079331f5779] - doc: note that FreeEnvironment() runs a shared event loop (Shelley Vohr) #656916ddbb9982f] - doc,test: account for OpenSSL 4.1 behaviours (Filip Skokan) #6595664fb33d791] - (SEMVER-MINOR) ffi: load libraries from a mounted VFS (Matteo Collina) #6590907fcf1ed02] - ffi: throw ERR_INVALID_ARG_TYPE for wrong-typed pointer and size (Soul Lee) #65842b7406b0a9b] - fs: coerce FileHandle.read length like fs.read (Xia Chao) #65521cb9995be4d] - fs: honor dereference for symlinks nested in cpSync trees (Christian Aurich) #657312b1701f810] - (SEMVER-MINOR) fs: add openAsBlobSync (greenhead) #656449b3f1aa03f] - fs: throw on existing dir in cpSync with errorOnExist (Daijiro Wachi) #641242b502e0798] - fs: support removing read-only files in rmSync on Windows (Sparsh :)) #6445377d8f17ab1] - http: don't destroy socket after request completes (Barath Raj) #6595246f76ed57b] - http2: settle pending write callbacks on destroy (Matteo Collina) #66016205443721d] - http2: fix onread assert when destroying session from stream handler (Sankalp Thakur) #651167e1db4724a] - inspector: report an error when DOM storage is unavailable (Avocado) #6597395110712b8] - inspector: fix abort when two Environments own the inspector (Shelley Vohr) #658779fd3e6caa4] - inspector: fix crash when the IsolateData has no platform (Shelley Vohr) #65818cce2795ad7] - lib: fix AbortSignal.any() abort propagation (Yuya Inoue) #660142c0ccc0067] - lib: fix for FileHandle.readableWebStream (Patrick Dähne) #5884228acafe6a0] - lib: avoid repeat internal receiver checks (Filip Skokan) #65910b9b0e33923] - lib: avoid unsafe array iteration in cli table (Donghoon Kang) #658387f514d3310] - lib: fix shared buffer growability validation (Filip Skokan) #658457534117e38] - lib: optimize internal Web IDL dictionaries (Filip Skokan) #65857bdacc08fbe] - lib: validate sequence iterator objects (Filip Skokan) #658445b6ac71fb2] - lib: use Web IDL interface brand checks (Filip Skokan) #65846da341e2571] - lib,src: apply multiple updates to dtls implementation (James M Snell) #6551113eb4843d5] - meta: bump step-security/harden-runner from 2.21.0 to 2.21.1 (dependabot[bot]) #660440f759174af] - meta: bump github/codeql-action/upload-sarif from 4.37.9 to 4.38.0 (dependabot[bot]) #66046c75b830082] - meta: bump cachix/cachix-action (dependabot[bot]) #660471ada7055a4] - meta: bump github/codeql-action/init from 4.37.9 to 4.38.0 (dependabot[bot]) #660481ed8b05742] - meta: bump github/codeql-action/analyze from 4.37.9 to 4.38.0 (dependabot[bot]) #660499165abe840] - meta: bump github/codeql-action/autobuild from 4.37.9 to 4.38.0 (dependabot[bot]) #66050d03e7313cf] - meta: add joyeecheung as v8 currency strategic initiative champion (Joyee Cheung) #6596553d7592378] - meta: expand on collaborator restoration process (Chengzhong Wu) #65962fa972d02b2] - meta: add web-standards as webidl owners (Filip Skokan) #65856080e76b3d7] - (SEMVER-MINOR) net: support sending net.BoundSocket to threads and child processes (Guy Bedford) #647250d14c0e507] - path: remove StringPrototypeCharCodeAt from some methods of posix (Wiyeong Seo) #5466813e61f6ae6] - (SEMVER-MINOR) perf_hooks: implement SlidingWindowHistogram (James M Snell) #65825a326546094] - (SEMVER-MINOR) perf_hooks: implement qrde analysis support in Histogram (James M Snell) #6580632401c2229] - perf_hooks: reuse buffer for uv metrics (Donghoon Kang) #659855e2fbfe0e5] - perf_hooks: validate import normalization offset (Matteo Collina) #659504743cd8d33] - quic: fix two small bugs in HTTP/3 stream internals (Tim Perry) #659700df54fb38f] - quic: improve stream cleanup & lookup (Tim Perry) #659447cf448889c] - quic: fix timeout regression from 11ed32572b3 (James M Snell) #6602861a98e6512] - quic: fix readable stream truncation on stop-sending, abort & timeout (Tim Perry) #63967975cac593c] - quic: add promise to QuicStream for pending strms (Marten Richter) #65862ae29cc92ea] - quic: split headers out from src/quic/stream.{h/cc} (James M Snell) #65863833c58d9dc] - quic: fix crash in onStreamClose (Marten Richter) #6586184ecb21343] - quic: reject zero addressLRUSize (Christian Aurich) #65827293e69d065] - sqlite: throw on invalid URL path instead of abort (Guilherme Araújo) #660266f5fb76d0e] - sqlite: track registered user-defined functions (Trivikram Kamat) #65896baf112b639] - sqlite: always copy changeset before applying (Trivikram Kamat) #658700306b0a71e] - (SEMVER-MINOR) sqlite: bind undefined to NULL (Trevor Burnham) #657091f24901c0c] - src: fix -Wextra warning in WriteFileSync (Antoine du Hamel) #66020b98787fe2d] - src: reuse crypto GetCipherInfo in DTLS session (Ilyas Shabi) #66022410093dbf7] - src: avoid union type-punning in trace values (Khaidi Chu) #659339ec030b1e7] - src: print exception thrown during primordial initialization (Joyee Cheung) #65991a0f99f1d6c] - src: support building with the V8 sandbox (Shelley Vohr) #6223727c62235d0] - src: ffi: create fast-call metadata Symbols lazily (Matteo Collina) #660155a5abd8fd0] - src: avoid copying SEA snapshot data (Colin McDonnell) #65876a05023f331] - src: fix crash on empty, foreign or truncated --snapshot-blob files (Shelley Vohr) #659554141e22606] - src: don't kill own process group on failed spawn (Lazizbek Ergashev) #6505405f4e54eda] - src: fix external reference list race between concurrent isolates (Shelley Vohr) #657791518b7d67f] - src: keep the first snapshot blob alive for later isolates (Shelley Vohr) #6577962e2bf025c] - src: detach cppgc wrappers from their Realm before it is freed (Shelley Vohr) #65778afc3e559d2] - src: fix Stop() terminating the next Environment on the isolate (Shelley Vohr) #65819f245e53b29] - src: seed V8 from the OS CSPRNG instead of OpenSSL's DRBG (Colin McDonnell) #65796830ca7df7a] - src: fix null pointer call when running without a startup snapshot (Shelley Vohr) #6582048158fba8c] - src: stop leaking a CppHeap in CommonEnvironmentSetup (Shelley Vohr) #657923c999edef7] - (SEMVER-MINOR) src,lib: add util.markPromiseAsHandled (James M Snell) #65805e5d2a336bd] - stream: destroy half-open sockets after iteration (Matteo Collina) #659864075161405] - stream: destroy Duplex.from async function on early return (Aman Chadha(IVIXMMI)) #659637d245cfd8d] - stream: fixup stream/iter to drop at most one entry per share call (James M Snell) #6602830e1f7651c] - stream: reject unbounded at SyncShare construction (James M Snell) #660280372056cd8] - stream: make share budget failures detach before throwing (James M Snell) #6602836cc238dce] - stream: make Broadcast.from abort its background pump (James M Snell) #660283134ca6f8e] - stream: update broadcast to retain buffered data with zero consumers (James M Snell) #6602804e5c282a1] - stream: fix async iteration of undefined chunks (Caleb Everett) #65969f81a1483b3] - stream: avoid promise allocation for parked transform writes (Matteo Collina) #65625aec01c5ae1] - stream: keep webstream stream states in fast-mode objects (Matteo Collina) #65625898cd55bdf] - stream: reject closed only after sink abort settles (Lazizbek Ergashev) #657275661526006] - stream: fix ERR_INVALID_STATE when cancelling Readable.toWeb() (Richard Scarrott) #62773fb8a97f46b] - stream: improve handling of falsy errors in stream/iter (James M Snell) #65864cfee7fa3d9] - stream: amortize writable buffer compaction (Gürgün Dayıoğlu) #6584715605210f8] - stream: create write request objects lazily (Matteo Collina) #64455619470a9ae] - stream: allocate stream read buffers from a slab (Matteo Collina) #644557e79c33354] - test: consolidate crypto provider cache coverage (Filip Skokan) #66108dcc9c3c02a] - test: avoid call to chmodSync in test-fs-cp-async-file-modes (Antoine du Hamel) #66104dda22c06d7] - test: deflake test-run-watch-cwd-isolation-none-* (Antoine du Hamel) #66035d63928dda0] - test: move permission FFI test to native suite (Yuya Inoue) #6605980c1bf7fb5] - test: deflake user timing WPT assertions (Filip Skokan) #6603612116d1d70] - test: unskip test-watch-create-isolation-none (Antoine du Hamel) #66041e90575cb4d] - test: deflake util.throttle tests (Filip Skokan) #66034f7d18ec360] - (SEMVER-MINOR) test: expand histogram test coverage (James M Snell) #65825bfa6d49b5c] - test: implement low-risk test optimizations (James M Snell) #65926c3307ebddf] - test: update WPT for WebCryptoAPI to 55ce71bb9d (Node.js GitHub Bot) #65813ee29c56993] - test: prevent parser reuse across close scenarios (Filip Skokan) #660178a27474044] - test: cover cpSync fast path timestamp preservation (Abhinandan Kumar) #6567889297305c0] - test: fix stderr Buffer assertion in exec encoding test (greenhead) #66008d4cb916624] - test: skip test-vfs-real-provider-watch.js on IBM i (SRAVANI GUNDEPALLI) #659876515db8ab6] - test: fix RSA/DSA wrong-passphrase flake (Filip Skokan) #65983fa19ecad9d] - test: improve sequential test performance (James M Snell) #659283666d6a238] - test: schedule WPT variants individually (Filip Skokan) #65984f2aa27d82d] - test: overlap SLH-DSA signature checks (Filip Skokan) #659802170207253] - test: unref cancelled broadcast source timer (Filip Skokan) #6598071a54aaf5f] - test: collect timeout signals explicitly (Filip Skokan) #659801a15aff897] - test: skip retries in DNS timeout coverage (Filip Skokan) #659805c7e6b4a6e] - test: synchronize ordered runner events (Filip Skokan) #6598037f24eddc9] - test: reuse fixed primes in DH tests (Filip Skokan) #65980831ec42bcc] - test: avoid idle HTTP/HTTPS connections (Filip Skokan) #65980f523342bcd] - test: close WebAssembly test HTTP servers (Filip Skokan) #65980f23847a122] - test: use named parameters in DH stress test (Filip Skokan) #659807131e3b437] - test: reduce ZIP64 stress test I/O (Filip Skokan) #65980e4ca72a984] - test: avoid allocations in external memory test (Filip Skokan) #659757fe64727a2] - test: cover experimental stream iterator builtins (Jungwon Sohn) #6596488ba17bae2] - test: deflake node-api test-free-called (Christian Aurich) #659486eee01fbae] - test: fix flaky common WPT inspector test (Yuya Inoue) #65937858702e51c] - test: skip C++ symbols in tick-processor-arguments (Philipp Dunkel) #65906472215ff9b] - test: try fixing windows build replacing WMIC (James M Snell) #659497f5168149c] - test: fix flaky test-bench-stream (Matteo Collina) #658749f2d544576] - test: move sqlite length validation out of the reentry test (Trevor Burnham) #6576975a9fa65ac] - test: fix flaky cleanup in http2 test (Tim Perry) #65701f8cdf05586] - test,benchmark: use OpenSSL feature helpers (Filip Skokan) #6576288b12c03cd] - test_runner: avoid reusing v8 serializers (Yuya Inoue) #659519b3085e660] - test_runner: fix quote escaping in JUnit (Jihwan) #6597102c2302c31] - tls: propagate singleUse to the secure context (Carlos Vinicius) #660252a8b5d9269] - tls: load all CRLs from a PEM bundle (Lazizbek Ergashev) #65577d141ddb8dc] - tls: defer re-entrant calls to SSL state machine from JS (Tim Perry) #65105b7fe779632] - tools: update tools/v8 for Python 3.13 (Richard Lau) #661096105b37a3d] - tools: bump eslint-plugin-jsdoc in /tools/eslint in the eslint group (dependabot[bot]) #66102060ed69df9] - tools: disable fortify warnings in test-shared (Antoine du Hamel) #66020f8b7a2ff41] - tools: clean up handling of shared libs in shell.nix (Antoine du Hamel) #658913c9586c1fc] - tools: group CodeQL GHA updates (Antoine du Hamel) #66057aa531a34f7] - tools: summarize auto-start-ci failures (Filip Skokan) #65979b7eef15f59] - tools: bump the eslint group in /tools/eslint with 6 updates (dependabot[bot]) #660458914947d7c] - tools: update pgo build doc for linux (Chengzhong Wu) #66024a5b1659d5b] - tools: avoid workflow shell interpolation (Filip Skokan) #66013caf0ee0d7b] - tools: use self-repository references (Filip Skokan) #66013d5880dcb05] - tools: correct Slack action version comments (Filip Skokan) #6601366f182f565] - tools: make checkout credential use explicit (Filip Skokan) #66013287ece3f22] - tools: pass author to commit message validator (Filip Skokan) #660126314ccf93b] - tools: reduce test runner timing overhead (Filip Skokan) #65980441350a867] - tools: do not download build tools when linting Nix files (Antoine du Hamel) #6596115f2549f74] - tools: bump js-yaml from 4.3.1 to 4.3.2 in /tools/eslint (dependabot[bot]) #659310e449d02a6] - tools: bump js-yaml from 4.3.1 to 4.3.2 in /tools/lint-md (dependabot[bot]) #6593281eba460b5] - tools: fix commit queue error summary matching (Filip Skokan) #65913e5bbcfdd78] - tools: lint PR commit messages without approval (Filip Skokan) #6587507ae602005] - tools: unlabel author ready on base branch conflicts (Filip Skokan) #65872e3ad68289a] - tools: improve benchmark build cache reuse (Filip Skokan) #6585946ddc0220d] - tools: apply feedback to and simplify contributor guidance workflow (Filip Skokan) #6578566fd78803d] - trace_events: fix abort when Node.js does not own the V8 platform (Shelley Vohr) #659543cb3c23a19] - typings: add task_queue internal binding types (Seongeun Lee) #65662c5c7036e7b] - typings: add timeoutInfo to TimersBinding (greenhead) #6581162fe96c166] - typings: add missing sea binding properties (이혜미) #65815e51673b4ed] - url: add Symbol.toStringTag to URLPattern (Khaidi Chu) #659253ce4d23bbb] - (SEMVER-MINOR) util: implement util.throttle (James M Snell) #65899336f33ccc1] - (SEMVER-MINOR) util: implement debounce (James M Snell) #6589916e3e7eff3] - vfs: add --vfs-mount and --vfs-load startup flags (Philipp Dunkel) #65748c50cb9a553] - vfs: write RealFSProvider files to open fd (Christian Aurich) #658851bbc5488a5] - vfs: close the fs hook gaps for mounted paths (Philipp Dunkel) #65852816790e0e5] - vfs: resolve symlinks when checking rename descendants (Trivikram Kamat) #65904a9149093df] - vfs: support renaming implicit ZIP directories (Trivikram Kamat) #65752cf5d4a8fe6] - vfs: reject statfs for missing paths (Trivikram Kamat) #65693de552bd044] - vfs: return FileHandle from fs.promises.open (Trivikram Kamat) #6573008f96f142e] - vfs: give ZipProvider option bags a null prototype (Philipp Dunkel) #658535c4d319028] - vfs: commit ZipProvider handles the way open(2) does (Philipp Dunkel) #6585329d3406b6c] - vfs: apply open(2) effects to ZipProvider handles (Philipp Dunkel) #65853e379c26a92] - vfs: align virtual file handles with open(2) (Philipp Dunkel) #658547cdf5f014a] - vfs: answer for unowned paths under reserved root (Philipp Dunkel) #658145e55085bc1] - zlib: reject invalid zstd dictionaries (James M Snell) #658674cdcf7f6a4] - zlib: fix zstd reset (James M Snell) #65867678ff3561a] - zlib: improve zstd decoding across chunk boundaries (James M Snell) #65865Windows 64-bit Installer: https://nodejs.org/dist/v26.10.0/node-v26.10.0-x64.msi
Windows ARM 64-bit Installer: https://nodejs.org/dist/v26.10.0/node-v26.10.0-arm64.msi
Windows 64-bit Binary: https://nodejs.org/dist/v26.10.0/win-x64/node.exe
Windows ARM 64-bit Binary: https://nodejs.org/dist/v26.10.0/win-arm64/node.exe
macOS 64-bit Installer: https://nodejs.org/dist/v26.10.0/node-v26.10.0.pkg
macOS Apple Silicon 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-darwin-arm64.tar.gz
macOS Intel 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-darwin-x64.tar.gz
Linux 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-linux-x64.tar.xz
Linux PPC LE 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-linux-ppc64le.tar.xz
Linux s390x 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-linux-s390x.tar.xz
AIX 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-aix-ppc64.tar.gz
ARMv8 64-bit Binary: https://nodejs.org/dist/v26.10.0/node-v26.10.0-linux-arm64.tar.xz
Source Code: https://nodejs.org/dist/v26.10.0/node-v26.10.0.tar.gz
Other release files: https://nodejs.org/dist/v26.10.0/
Documentation: https://nodejs.org/docs/v26.10.0/api/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
f5a93fe73cf5583414d4d8be598c4ec7e72be3281b5820d495c2a15ce6aa13c8 node-v26.10.0-aix-ppc64.tar.gz
bd437fcd6a39e1533baf4aa225b2caaa7eb9d21a6fc32ce5020a0ffc1e04da6e node-v26.10.0-arm64.msi
751fdf7439f115d87ee2a8f3f18c065b6151852068e3e666ac60ac2996f75ac9 node-v26.10.0-darwin-arm64.tar.gz
f222f7e85cc1d5a3d84786aa86836ec3d41c4dcdf59a40e4a9d719b6b0ae28af node-v26.10.0-darwin-arm64.tar.xz
ebbe9ab9b58ad6bb54390d6e2c862c1afa7d4475fb7e8ae8146acde211bf70df node-v26.10.0-darwin-x64.tar.gz
fdaa8c236bd054de59059743ed8d2766de154d81853ca26ad89aca899696170a node-v26.10.0-darwin-x64.tar.xz
4d49056ee9dc1b29f3eb4405edcc91784226b7550ccf18c29088f6977e12f92b node-v26.10.0-headers.tar.gz
37b703e39a76d8abd4b0fad3cb50485e543985128b3acd6678b5a9f312ca02a1 node-v26.10.0-headers.tar.xz
423a41bff8e2a2fa15e702fefe2919ef95823b2378744daccb8439302534b44f node-v26.10.0-linux-arm64.tar.gz
7a6353f63eb3d04765004b4adf172616243e4522434635cb1d26288658b04ab5 node-v26.10.0-linux-arm64.tar.xz
0cc6b9a9906216f8de7d70146038fe6bab5f7e8f3a8ad29d8dbeabdf1381b52d node-v26.10.0-linux-ppc64le.tar.gz
40dcfb22e9bfbe7c53e5a62065b7aab4261e8096086e23c03547d70d59831db9 node-v26.10.0-linux-ppc64le.tar.xz
787402b47f0f8fd462a19a44c3d707052cabc95ecb946fe8041a1b5c810e3d63 node-v26.10.0-linux-s390x.tar.gz
5b2d13a530059ccd41d41ca037fbe94a4b61fd77808fd504fd75c40b4aaa6d74 node-v26.10.0-linux-s390x.tar.xz
1bdf60c8141e5152604baf8b38f31e191b3794f60fce2b71351e78c4021f0166 node-v26.10.0-linux-x64-musl.tar.gz
c335cea18f2e2ebee431e3bf9785290832fd0bdf27759496dc8cb391c7b3d68a node-v26.10.0-linux-x64-musl.tar.xz
cb5c9ce9c80d7b8821e3a258543c71b939138cf17c74d5cc44bbe85d6dbc5ad8 node-v26.10.0-linux-x64.tar.gz
ca70e9e349de048b9522abb3adc05b3bd6f43c5ffd3ec57916c7da292f59f022 node-v26.10.0-linux-x64.tar.xz
558247f16dba690dd293858923cb423aa64b60ac3b578f7cf4c3726dec3c2275 node-v26.10.0.pkg
093087c5cc00d185637f8682c5afaf3c967b557e3eefa41c46e3dc5cf3f8e8e5 node-v26.10.0.tar.gz
7b3a546d33cb7e15a43bdd7a57e0be5d5fd5ffc553e6e4c120033e66f0ba20c5 node-v26.10.0.tar.xz
9be039805e8eb2ba0e55857974638a922af5fd1707ade2c500a2feb75cbe295b node-v26.10.0-win-arm64.7z
b778640d7271566bcaa9679912cdf0684c13e824c114e41a8b696fb14af7a7aa node-v26.10.0-win-arm64.zip
16ca2cfc9a311760bdbd9bf20fef3232d6dc3c8fcd3d55e221f9c200bf95306f node-v26.10.0-win-x64.7z
9fef7eca6743a6b910989cd8e78712376b394fcb9b6e1e9c44a0799a287f90c5 node-v26.10.0-win-x64.zip
ee07551f530db927d97c2753ec20bdfdd78177160f4add57350f0c470766622d node-v26.10.0-x64.msi
63f374ed30ab433fece49304f1a20a1a96137058dd0b10d69169c104dd15b997 win-arm64/node.exe
45084cbfc0170d7a30b29ab1eeeb2b0d770995ade0a13a4cc82cf31189f95c9f win-arm64/node.lib
03a859bab7fb5a327de5629269ca809e806a70ad39c17618aec70c8ff537afe5 win-arm64/node_pdb.7z
f606bc80cb7cc457cc9e905e72e79a4f122048a078d2d785995078317ccbbe83 win-arm64/node_pdb.zip
cea6ac365f9bb9586dafd2084d996e092e7d3e7d07ab52d1f76bf53d1fca9bc4 win-x64/node.exe
caad0090dc3add94c06abd01e395e8ec0f1586c10071bda1f99480445d695f6f win-x64/node.lib
5d4052363bc42b474d45c73e4a82981254d6c15399a61bf606bd3bc9cfea22d0 win-x64/node_pdb.7z
18e979b428bbc401cdfebf7fc2d52d567dae9888a62e2f1123441e88cb5ffa34 win-x64/node_pdb.zip
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQRb6KP2yKXAHRBsCtggsaOQsWjTVgUCarJEDQAKCRAgsaOQsWjT
VscuAP44wuxjRueeqHmQhh3YtIFxeE1onzCPY+PkX3mcmK4qaAEAjG6pEdt7rzDe
uzX+p/IjTIqOAt53UQdxsyPdvDHbmAo=
=7KKC
-----END PGP SIGNATURE-----
AI can produce code. Organizations still have to produce software. Agentic development is changing how software gets made, but it hasn’t changed what it costs to be wrong.
Six months ago, we began publicly experimenting with agentic development environments. Around the same time, we introduced JetBrains Central as an open control and execution system for agent-driven development. We subsequently began rolling out JetBrains Central CLI, shared context, cloud agents, automations, governance, and AI cost controls for teams and organizations.
Today, we are bringing this work together as JetBrains Air: an open, coherent system of products for developers, teams, and organizations, inside and beyond JetBrains IDEs. It is multi-surface and multi-service. Each product solves a distinct problem, but the products work better together.
JetBrains Air marks a significant expansion in what JetBrains is building for. For 26 years, we have focused primarily on the individual developer workbench. Now, we are building for the wider system through which agentic work is initiated, executed, coordinated, reviewed, and governed.
The IDE remains important to JetBrains’ future. The era in which the whole software development system can be contained in one window is ending. As part of our continued investment, we are now bringing the foundational agentic experience into JetBrains IDEs, giving professional developers an environment where they can work effectively with agents while understanding, changing, and verifying the resulting code. JetBrains Air connects the wider system developing around it.
That system is based on a core belief that the future of agentic development will be multi-vendor. No single model, agent, or service will be right for every developer, team, or task.
This strategic shift has a practical consequence: JetBrains Air cannot be just another agent or development environment. It must connect products for individual work, team coordination, organizational control, context, and process automation – and remain open to the tools and agents developers choose, including those JetBrains does not build.
JetBrains Air includes products that are available today alongside others that will be introduced as the system develops:
Junie is JetBrains’ coding agent for professional software development. It will be supported across all Air surfaces.
Air in JetBrains IDEs gives developers the environment to direct agents and verify their output using JetBrains’ code intelligence. Air Teams turns individual agent activity into coordinated team workflows. Air Governance makes that activity visible, governable, and accountable across the organization.
But an open system cannot stop at JetBrains’ own products. The Agent Client Protocol (ACP) standardizes the connection between an IDE and an agent’s full harness, including its planning, logic, tools, model routing, and observability. Through the ACP Registry, developers can discover and run a growing range of compatible agents while continuing to work inside JetBrains IDEs.
Air Governance is designed to extend visibility and cost governance across providers and the different tools through which agentic work takes place. This means developers can choose the agent, model, or service suited to the task without forcing the organization to give up context, visibility, or control.
Together, the Air products allow work to move between developers, agents, tools, and environments without losing the context and controls surrounding it.
Since March, our products have progressed significantly, but so has our understanding of what agentic development requires.
Developers have been adopting agents faster than organizations can build the infrastructure around them. Agent capabilities have advanced, and different models and agents have proven useful for different tasks. However, the context, coordination, governance, and cost management surrounding them have not kept pace.
For many developers, agents are already delivering practical value. At the organizational level, the economics are much harder to prove. The costs surface elsewhere – in review, rework, security, infrastructure, and spend.
Which agents can access company code? Where can data go? Which output requires human review? What happened while an agent was working remotely? Who approved the resulting change, and how was it verified?
Fragmentation at this level isn’t just irritating. It makes software development harder to understand, measure, and govern at exactly the point when more of the work is being delegated.
Code that’s obviously wrong gets caught quickly. That part of the system still works. The harder problem is code that’s almost right: plausible, capable of passing a superficial check, but quietly carrying a bad assumption or architectural inconsistency that won’t surface until it’s expensive.
As agents take on more of the execution, the bottleneck shifts from producing change to understanding, verifying, and owning it. Code becomes cheaper to generate but more expensive to verify. Agent activity becomes easier to start but harder to coordinate, audit, and explain.
And while the work can be delegated, accountability cannot. An agent will not get the call at 3:00 am when something breaks. The responsibility for what ships still belongs to the people and organizations that ship it.
This is why control becomes harder, not easier, as AI improves. A more capable model may produce better output. It does not establish organizational policy, preserve provenance, provide cost visibility, or decide who accepts responsibility for the resulting change.
Multi-vendor support is a foundational design principle of JetBrains Air, shaping how the system is being built from the outset.
We don’t believe this market will consolidate any time soon. Models vary in what they’re good at, and rankings change every few months. Teams inside the same company already make different choices, and they are often right to do so. Standardizing on one AI vendor today means making a multi-year commitment in a market that won’t look the same next quarter.
Keeping the options open is the reasonable thing to do. The problem is what openness currently costs. Every new model, agent, or service an organization adds takes away a little more visibility into its own development work. Context doesn’t carry over between tools. Spend can’t be attributed. Policies have to be rebuilt for each service.
Organizations should not have to choose between using the best available tools and understanding what is happening inside their own engineering. That trade-off exists because nothing in the current stack was built to sit above several vendors at once.
This is the work JetBrains has taken on. We build our own agent, and we intend to make it excellent. But JetBrains Air does not require customers to use ours, and our strategy does not depend on which model provider leads the rankings this quarter. We have no reason to make the ecosystem smaller than it is.
What we can offer instead is one place to run, see, govern, and account for agentic development across every model, agent, and service – for the developer, the team, and the organization.
Supporting multiple models and agents is the floor, not the ceiling. The part that matters is what sits above them: shared context, one set of policies, a single cost view, and a record of what happened, regardless of which vendor produced the change.
Multi-vendor choice solves only part of the problem. Agents also need reliable software intelligence.
JetBrains brings 26 years of engineering intelligence to the problem, helping developers understand the structure and behavior of complex software, not simply generate more of it. That deterministic code intelligence provides a foundation for making agentic work more reliable, efficient, and understandable across different models and agents. We are seeing promising results from giving AI agents access to deterministic code intelligence.
This is an economic advantage as well as a technical one. Agents spend time and money rediscovering information the codebase already contains. An agent that can retrieve that knowledge is cheaper and more accurate than one that has to reconstruct it. Because intelligence does not belong to one model, the benefit can extend across supported agents and services.
We are also going through the same transition as the organizations we build for, adopting agents internally, redesigning workflows, and learning where individual productivity gains translate into better software delivery and where they simply move work elsewhere.
JetBrains Air will develop through a rolling series of releases. We will be explicit about what customers can use now, what is entering preview, and what remains part of our longer-term direction.
Over time, JetBrains Air will extend further into mobile and remote experiences, allowing people to initiate, monitor, review, and continue agentic work as it moves between environments. The goal is not to reproduce the IDE on every surface. We are making the right context and controls available wherever decisions need to be made.
We will also bring JetBrains’ intelligence into more agentic workflows. This includes richer context drawn from code, architecture, repositories, runtime behavior, and organizational knowledge, as well as better ways to route work between developers, models, agents, and services.
More work will be triggered by repository events, schedules, and delivery processes rather than by a developer opening an editor and issuing a prompt. JetBrains Air will provide the intelligence, oversight, and human control these workflows require across surfaces and services.
We will not name future products before their scope and availability are ready to be confirmed. With each release, we will explain what works, how it connects, and what’s still in progress.
The companies that succeed in adopting AI will not necessarily be those that generate the most code or deploy the most agents. They will be those that can expand experimentation without losing quality, context, cost discipline, or human understanding.
JetBrains Air is our commitment to building for that reality. It expands JetBrains from the developer workbench into a system of products connecting developers, agents, teams, and organizations.
The goal is not more code. It is software that developers, teams, and organizations can understand, verify, and stand behind.
You can now publish a BigQuery data agent in Gemini Enterprise by registering the agent with Agent Registry and importing it using default Google-managed credentials. When BigQuery and Gemini Enterprise are in the same Google Cloud project and configured with a matching Agent Gateway region, you don't need to manually copy the Agent-to-Agent (A2A) JSON card or configure OAuth client credentials.
These features are in Preview.
You can use the Google Cloud console to create and manage protobuf schemas (schema bundles) for your Bigtable tables. You can also view schema bundle definitions in Bigtable Studio. This feature is generally available (GA). For more information, see Create and manage protobuf schemas.
google-cloud-sdk Snap package will be deprecated and removed on September 29th, 2026. Please migrate to the google-cloud-cli package. For more information, see https://docs.cloud.google.com/sdk/docs/downloads-snap.gcloud CLI man pages component (gcloud-man-pages) is deprecated and
will be removed in release version 590.0.0 on October 20th, 2026. Please use
the built-in --help flag for full command documentation.gcloud services api-keys create and
gcloud services api-keys update to require --api-target restrictions
across GA and beta.--clear-restrictions flag from gcloud services api-keys update.kpt component from the Google Cloud CLI. Kpt is an open-source project and continues to be actively maintained. To avoid disruptions, please migrate to the standard OSS kpt installation: https://kpt.dev/installation/kpt-cli/.gcloud audit-manager audit-schedules command group, supporting create, list, and update commands.gcloud biglake iceberg catalogs create
--[federated-catalog-type, glue-aws-region, glue-aws-role-arn,
glue-warehouse, namespace-filters, refresh-interval, secret-name,
service-directory-name, snowflake-account-identifier, snowflake-role,
snowflake-warehouse, unity-catalog-name, unity-instance-name,
unity-service-principal-application-id, workday-base-url, workday-tenant].gcloud beta biglake iceberg catalogs create
--[service-directory-name, snowflake-account-identifier, snowflake-role,
snowflake-warehouse, workday-base-url, workday-tenant].gcloud beta biglake iceberg catalogs update
--[service-directory-name, snowflake-role].gcloud biglake iceberg catalogs update --[glue-aws-role-arn,
namespace-filters, refresh-interval, secret-name, service-directory-name,
snowflake-role, unity-service-principal-application-id].create and update methods to gcloud observability buckets
command group.BETA to GA.gcloud domain mappings create, allowing users
to create easy to remember and shareable subdomains of the format
<user-chosen>.cloud.run--clear-key flag to gcloud beta run instances deploy and gcloud
beta run instances update to remove a previously set CMEK key reference.networkTags property to instance configuration flags in gcloud beta
cluster-director clusters create.--nfs, --add-nfs, --remove-nfs,
and existingNfs in --config) in gcloud beta cluster-director clusters
create/update.gcloud compliance-manager framework-deployments update to update framework deployments across organization and project scopes.gcloud compute url-maps test-iam-permissions command to test IAM permissions on a URL map in beta, preview, and GA.--request-body-to-exclude flag of gcloud compute security-policies rules add-preconfig-waf-exclusion and gcloud compute security-policies rules remove-preconfig-waf-exclusion to GA.--request-body-to-exclude flag of
gcloud compute org-security-policies rules add-preconfig-waf-exclusion
and gcloud compute org-security-policies rules
remove-preconfig-waf-exclusion to GA.--preemption-notice-duration flag to gcloud compute instances
in GA.gcloud compute interconnects set-name to beta.--load-parallel-level flag to gcloud database-migration
migration-jobs create and gcloud database-migration migration-jobs update
commands to specify the parallelism level during initial load for MySQL
migrations.gcloud design-center spaces applications recommend-iam-roles command to get recommended IAM roles for a Design Center application.gcloud developer-knowledge commands to GA.gcloud device-run sessions submit xctest to beta.gcloud device-run software-versions list command to list available test software versions.gcloud device-run software-versions describe command to describe a specific software version.--hub, --auto-accept, and --psc-routing-enabled flags of gcloud network-connectivity transports create to GA.gcloud network-security authz-policies import to support DENY_BY_DEFAULT action.gcloud network-security firewall-endpoints wildfire-verdict-change-requests commands to the ALPHA and BETA release tracks.gcloud beta orchestration-pipelines info command to display information about the orchestration pipelines library and supported model version.Subscribe to these release notes at https://groups.google.com/forum/#!forum/google-cloud-sdk-announce.
The following remote Google Cloud MCP servers automatically generate a trace span for
tools/call operations.
These spans can help you understand the behavior of your agentic applications. For more information, see Investigate MCP calls using Trace.
You can use Terraform to configure resources managed by the Observability API. For example, you can use Terraform to create and update observability buckets, create links on datasets, and configure default settings.
For more information, see the following documents:
As of September 15, 2026, NVIDIA P100 (nvidia-tesla-p100 and
nvidia-tesla-p100-vws) GPUs have reached end of support (EOS) and are shut
down. You can no longer create, launch, or access Compute Engine
instances or other Google Cloud resources that use NVIDIA P100 GPUs.
For information about migrating your workloads to supported GPU alternatives such as the G2 (NVIDIA L4) or G4 (NVIDIA RTX PRO 6000) machine series, see NVIDIA P100 end of support.
NVIDIA T4 (nvidia-tesla-t4 and nvidia-tesla-t4-vws) and NVIDIA P4
(nvidia-tesla-p4 and nvidia-tesla-p4-vws) GPUs are deprecated and will reach
end of support (EOS) on August 1, 2027. After August 1, 2027, you won't be able
to create, launch, or access Compute Engine instances or other
Google Cloud resources that run NVIDIA T4 or P4 GPUs. In addition, you can no
longer purchase or renew 3-year committed use discounts (CUDs) for NVIDIA T4 or
P4 GPUs.
To transition your workloads to supported GPU models such as the G2 (NVIDIA L4) or G4 (NVIDIA RTX PRO 6000) machine series before the EOS date, see NVIDIA T4 end of support and NVIDIA P4 end of support.
The Secret Manager API is no longer enabled by default when you enable the Developer Connect API. For Git repository connections, you must enable the Secret Manager API explicitly.
Gemini Enterprise: D&B Risk Analytics data store
The D&B Risk Analytics data store is generally available (GA) in Gemini Enterprise. You can connect D&B Risk Analytics to run third-party and counterparty risk workflows against your D&B Risk Analytics tenant using natural language. Supported workflows include KYB onboarding, counterparty due diligence, sanctions and adverse media screening, and supplier and financial risk assessment. The data store also supports actions, such as creating an entity, starting a screening, and updating tags and custom fields.
For more information, see Connect D&B Risk Analytics.
Anthropic's Claude Opus 5.5
Claude Opus 5.5 is available in Model Garden.
Connectivity Tests analyzes firewall policy rules that use Google Threat Intelligence data.
We're adding support for running GGUF models efficiently in transformers, so you can use checkpoints sized for your laptop's memory through the familiar transformers APIs. Pick a GGUF from the Hub, load it with from_pretrained, and start generating on your own machine.
Running AI models on your laptop has become much easier, and llama.cpp has been a big part of that. Its inference engine powers local AI tools such as Ollama, LM Studio, and Jan. Alongside projects like MLX, it has helped make local inference a practical option for everyday use.
A recent example of what local AI can feel like:
— Julien Chaumond (@julien_c) April 24, 2026This is where we are right now. And i’m not gonna lie it feels pretty magical 🧙♀️
Qwen3.6 27B running inside of Pi coding agent via Llama.cpp on the MacBook Pro
For non-trivial tasks on the @huggingface codebases, this feels very, very close to hitting the latest Opus in Claude… pic.twitter.com/lsIxLoUneU
GGUF, developed by the llama.cpp team, is a widely used format for local inference. The team also shares quantized checkpoints under ggml-org on the Hub. Publishers such as Unsloth, LM Studio Community, and bartowski also provide ready-to-use GGUF checkpoints in a range of quantizations, so users can pick the version that fits their machine. GGUF models have been downloaded millions of times.
We want to make it easier to run these models locally with transformers, too. Compatibility is only useful if the model is pleasant to run. To bring performance close to llama.cpp, we're reusing its underlying ggml kernels through the kernels library, and reducing overhead in generate. Our initial focus is local inference on Apple Silicon, starting with the Qwen3.5 architecture.
GGUF packages model weights and metadata, including tokenizer information and an optional chat template, in one file. It supports different quantization levels, letting you trade some precision for a smaller memory footprint. Variants such as Q4_K_M mix tensor precisions, using mostly 4-bit weights while keeping sensitive tensors at higher precision.
Here's how quantization changes the file size of Unsloth's Qwen3.5-4B:
| GGUF variant | File size | Tradeoff |
|---|---|---|
BF16 |
8.42 GB | Unquantized reference |
Q6_K |
3.53 GB | More precision than the smaller variants |
Q5_K_M |
3.14 GB | A middle ground between size and precision |
Q4_K_M |
2.74 GB | A practical starting point for local inference |
We suggest starting with Q4_K_M, then trying Q5_K_M or Q6_K if you have more memory available. More aggressive quantization can help larger models fit, but the quality tradeoff depends on the model and the task. Evaluate it on the work you actually want the model to do. The Hub's GGUF documentation describes the available quantization types.
To get started, you need:
kernels.pip install -U "git+https://github.com/huggingface/transformers.git" kernels
To load a GGUF model, pass its Hub model_id and filename as gguf_file to from_pretrained.
No extra configuration is needed: when the weights stay packed on Metal, transformers automatically loads the compatible ggml/Metal layer kernels and uses ggml-org/ggml-attn as the attention implementation. If that kernel cannot be fetched, the model falls back to "sdpa" with a warning, and you can always force "sdpa" by passing attn_implementation="sdpa" explicitly. See the GGUF documentation for more loading options.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "unsloth/Qwen3.5-4B-GGUF"
filename = "Qwen3.5-4B-Q4_K_M.gguf"
tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
model = AutoModelForCausalLM.from_pretrained(
model_id,
gguf_file=filename
)
That is the only GGUF-specific step. Everything after it is the standard transformers API:
messages = [{"role": "user", "content": "Explain why the sky is blue in a few sentences."}]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Without a compatible quantization kernel, the loader falls back to dequantizing the model and uses more memory.
You can also use the same checkpoint with transformers serve, which exposes an OpenAI-compatible API:
pip install -U "transformers[serving] @ git+https://github.com/huggingface/transformers.git" kernels
transformers serve "unsloth/Qwen3.5-4B-GGUF:Qwen3.5-4B-Q4_K_M.gguf"
The model argument uses <model_id>:<filename>.gguf: before the colon is the Hub repository (unsloth/Qwen3.5-4B-GGUF), and after it is the file to load (Qwen3.5-4B-Q4_K_M.gguf). This selects a specific quantization from a repository that may contain several.
For models whose chat template supports thinking, add --reasoning off to skip it or --reasoning on to enable it. The default, --reasoning auto, follows the chat template’s default. See the reasoning options for details.
You can connect a client such as Jan or Pi by adding a custom OpenAI-compatible provider with these settings:
| Setting | Value |
|---|---|
| Base URL | http://localhost:8000/v1 |
| Model ID | unsloth/Qwen3.5-4B-GGUF:Qwen3.5-4B-Q4_K_M.gguf |
transformers runs the model on your Mac, while the client provides the conversation interface. The same endpoint can be used by other clients that support this API.
Our reference for local inference performance is llama.cpp. The comparison below focuses on three GGUF checkpoints: a small dense model, a larger dense model, and a mixture-of-experts model.
The llama.cpp column comes from the llama-bench tool (build 5f55650a7, release b10200, Metal backend from ggml 0.18.0), run as llama-bench -m <file> -p 0 -n 128 -r 3, which reports tg128: the token-generation rate over 128 decoded tokens, averaged across three repetitions, with prompt processing excluded. The transformers column is generate producing the same 128 tokens from a 12-token prompt, best of three warmed runs, and it includes prefill.
Measured on a MacBook Pro M2 Max, 32 GB unified memory, macOS 26.6, PyTorch 2.12.1, kernels 0.17.0, plugged in.
The benchmark scriptimport time
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id, filename = "unsloth/Qwen3.5-4B-GGUF", "Qwen3.5-4B-Q4_K_M.gguf"
model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file=filename)
tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
inputs = tokenizer("The capital of France is Paris. The capital of Germany is", return_tensors="pt")
inputs = inputs.to(model.device)
with torch.inference_mode():
model.generate(**inputs, max_new_tokens=8, min_new_tokens=8, do_sample=False) # warm up
torch.mps.synchronize()
for _ in range(3):
time.sleep(90) # let the machine cool: back-to-back runs decay by 10% or more
start = time.perf_counter()
model.generate(**inputs, max_new_tokens=128, min_new_tokens=128, do_sample=False)
torch.mps.synchronize()
print(f"{128 / (time.perf_counter() - start):.1f} tok/s")
For the other column:
llama-bench -hf unsloth/Qwen3.5-4B-GGUF:Q4_K_M -p 0 -n 128 -r 3
Transformers is close to llama.cpp across all three checkpoints. The chart uses the same measurements described above; it does not imply identical benchmark conditions, since the Transformers measurement includes prefill while llama-bench reports decode-only throughput.
When GGML and llama.cpp joined Hugging Face, we described their complementary roles: llama.cpp provides a foundation for local inference, while transformers provides a foundation for model definition. GGUF support brings those two closer together.
llama.cpp remains our recommended engine when your priority is efficient local inference. Its dedicated runtime, memory management, and broad hardware support are built around that goal. This integration gives developers a convenient way to work with the same GGUF checkpoints inside transformers:
generate, or write your own generation loop in Python.For that last case, use GgufConfig(dequantize=True):
import torch
from transformers import AutoModelForCausalLM, GgufConfig
model = AutoModelForCausalLM.from_pretrained(
"unsloth/Qwen3.5-4B-GGUF",
gguf_file="Qwen3.5-4B-Q4_K_M.gguf",
quantization_config=GgufConfig(dequantize=True),
dtype=torch.bfloat16,
)
The bigger opportunity is bringing ggml's performance to models that llama.cpp does not support.
transformers already provides the PyTorch implementations of these architectures. With ggml kernels and quantization schemes available in PyTorch, we can work toward accelerating their supported operations without first implementing the entire model in llama.cpp. This is especially useful for new architectures, research models, and custom variants that may never receive a dedicated llama.cpp implementation.
That opportunity extends beyond the GGUF format itself. A kernel operates on tensors; it does not require the whole model to come from a GGUF file. The same building blocks can be integrated into other transformers models and loading workflows. This also opens a path to other modalities: computer vision models, audio models, and multimodal models could reuse compatible attention, normalization, and matrix multiplication kernels without first having a full implementation in llama.cpp. Each architecture still needs integration and validation; the initial GGUF examples here cover text generation.
We also wanted to show how far we can get while keeping the model and generation loop in Python. With the right kernels and an efficient generation loop, Python and PyTorch can deliver strong local inference performance. The kernels handle the heavy computation, while the generation loop keeps the GPU busy by avoiding unnecessary synchronization.
Our focus was to make eager execution fast without requiring torch.compile. For interactive use, we wanted a quick start and a steady stream of tokens, without compilation pauses or recompilation when input shapes change. The two main pieces of that work are the kernels and generate itself.
A kernel is a small program that performs an operation on the GPU. PyTorch supplies general-purpose implementations; a specialized kernel can do less work, combine several operations, or read quantized weights directly in their stored format.
The kernels library lets us distribute compatible builds of ggml's Metal kernels on the Hub and call them from transformers. That brings ggml's work into the PyTorch model without replacing the model with a separate inference runtime.
| Kernel | What it does |
|---|---|
ggml-quantization |
Reads packed quantized weights for matrix operations, including the selected experts in an MoE model. It avoids expanding the whole weight matrix before each decode operation. |
ggml-norm |
Fuses normalization operations, including the zero-centered RMSNorm used by Qwen3.5 and Qwen3.8. |
ggml-attn |
Provides ggml's Metal flash attention for prompt processing and token decoding. |
ggml-gated-delta-net |
Accelerates the gated delta network used in the linear-attention layers of the Qwen3.5 and Qwen3.8 hybrid architectures. |
topk |
Selects the experts for each token in an MoE model, combining softmax and top-k routing. This is our own Metal implementation. |
The first four packages build on ggml's kernels; the top-k kernel addresses a separate bottleneck in MoE routing. Together they reduce the GPU work needed for each generated token.
To show the contribution of the layer kernels, we compare the same packed GGUF checkpoints with and without them. The quantization kernel stays enabled in both configurations: disabling it would also change how weights are represented and would measure a different tradeoff.
Faster kernels only help if the GPU has work to do. During generation, the CPU schedules GPU operations and controls the loop that produces the next token. Reading a result back from the GPU can force the CPU to wait until queued operations finish. Repeating even a small wait for every token can noticeably reduce throughput.
Two changes address this in generate, which results in improvements for all transformers models (not just when running GGUF files):
generate copies the stopping decision asynchronously and consumes it on the following step. The CPU can keep scheduling work while the GPU runs. Streaming tokens use the same approach, and any extra step past the stopping condition is removed from the result.These changes improve the generation loop around the model, so their usefulness extends beyond GGUF. They complement the kernel work: kernels reduce the cost of an operation, while fewer synchronization points let CPU scheduling and GPU execution overlap.
These measurements keep all layer kernels enabled; the bars isolate the changes to the generation loop.
The initial target is a single interactive conversation on Apple Silicon. There are a few boundaries to keep in mind:
generate_batch on MPS.If you have a GGUF model you would like to use in transformers, open an issue with the checkpoint and your use case. That will help us prioritize support for the models people are running locally.
We would like to thank Arthur Zucker for initiating this work and reviewing all of my PRs, and Cyril Vallez for the generate PRs. We are grateful to Sayak Paul, the llama.cpp team, and Bertrand Chevalier for their help integrating the kernels. We also thank Aritra Roy Gosthipaty and Pedro Cuenca for reviewing this blog post, and Lysandre Debut for overseeing the project.
We are super excited to welcome Jun as our newest team member 🔥. We are completely invested in local AI, and MLX is a central piece of the ecosystem. We are delighted that Jun chose us to set up home and continue contributing to MLX.
MLX is Apple's framework for local AI, especially optimized for Apple Silicon. We are big MLX supporters since it was the Christmas present from Awni and Angelos in 2023, and proud that Hugging Face is the Hub where people find MLX models and contribute their own. Usage of open, local AI is accelerating, and we believe in a healthy ecosystem where people can find the tools that work for them.
Stability, and hopefully faster development! Graduating from a side job to a fully maintained and funded project will allow Jun to better guide the contributors and build for the long-term. oMLX stays Apache 2.0, and Jun keeps leading it as before.
Our end goal is to unblock the community to run local AI in any shape or form, and provide the tools and building blocks to make that happen. We expect oMLX to serve as a testbed for new ideas, while leveraging the foundational work of the dependencies it already relies upon, such as mlx-lm or mlx-vlm. We believe that strong modeling and inference libraries help the community, so we'd love to upstream work to wherever it makes sense. We have been collaborating with many projects mlx-lm, mlx-vlm, LMStudio, and we hope we can strengthen the relationship with Cheng, Prince, Yagil, and their teams to better serve the community together.
Concretely, one focus area is the quick transition from a transformers model definition to a reference MLX implementation that can be consumed by different engines, so each one can focus on the unique features they provide. The transformers library has become the reference for ML model definitions, we want to streamline the process to make new transformers models run on MLX.
We are incredibly excited about the future.
Welcome, Jun! 🙌
The Next.js team has disclosed a critical severity vulnerability in an upstream dependency that can lead to remote code execution when ImageResponse renders untrusted input. It is patched in 15.5.26 and 16.3.6. Applications that do not pass untrusted input into ImageResponse are not expected to be affected. Here’s what Netlify customers need to know.
next/og ImageResponse. Critical. Patched in 15.5.26 and 16.3.6.Netlify sites are affected only if they use ImageResponse and the image it generates includes untrusted input — text, or an image loaded from the request. Sites that don’t use ImageResponse, or that only render trusted content through it, are not affected.
For sites that do, the impact is limited to a crashed function invocation, not code execution. On Netlify, this has minimal impact: our autoscaling serverless architecture means that a malicious request resulting in a crashed function does not affect other requests. However, active exploitation could increase your function costs.
We strongly recommend upgrading as soon as possible to patched releases:
next 15.5.26 or later, or 16.3.6 or later, then redeploy.Until you can upgrade, do not place untrusted input inside elements passed to ImageResponse. Escape it as XML before rendering, or keep it out of the generated image entirely.
Note that any publicly available deploy previews and branch deploys may remain vulnerable until they are automatically deleted. Consider deleting these deploys manually.
Posted on 2026-09-22 by EMS Software Development
Related Proprietary
This is a major release, and the headline feature is our new AI Assistant. We're starting to roll out AI capabilities across our database tools, and SQL Manager for PostgreSQL is the first to get them. It's also available in SQL Management Studio for PostgreSQL, which ships with SQL Manager.
You'll find all the AI Assistant features you'd expect: write a query from a plain-language description, explain existing SQL, fix errors, optimize, make sense of an EXPLAIN plan.
But you can also take on bigger jobs:
Here's the key part: you decide how much of the schema the model sees, down to a single table. You stay in full control of exactly what context goes to the model.
Connect as many providers as you like — all it takes is an API key (stored encrypted), and you can switch models right in the chat. Need to stay completely self-contained? It works with Ollama, so your data never leaves your machine.
Posted on 2026-09-22 by Constructive
Related Open Source
Application code has fast testing loops: runners, fixtures, and red-green feedback in JavaScript, TypeScript, and Python. Postgres can be tested too, but database logic often sits outside those loops. Developers have to provision state, manage transactions, or fall back to a pasted query, a browser refresh, or a manual check. That gap shapes architecture. When rules are easier to test in application code than in Postgres, they tend to end up there, even when Postgres is the better place to enforce them.
Mocks do not close the gap. They test how application code handles a result, not whether Postgres will produce it. A mock does not exercise a foreign key, fire a trigger, evaluate a row-level security policy, or verify the database role under which a query actually runs.
pgsql-test is an MIT-licensed harness that puts a real PostgreSQL database inside those loops. It spins up an ephemeral PostgreSQL database, seeds it once, and rolls every test back to that seeded state. Assertions run in the project’s existing test runner, and Postgres executes the constraints, functions, and policies under test. It is not the first way to test Postgres—pgTAP has long done it in pure SQL. pgsql-test targets the application layer instead, where most developers already work.
Row-level security (RLS) makes access rules enforceable by Postgres. Policies are pure database logic, invisible to mocks, and a wrong one can leak rows. Testing them means checking both what a user can access and what they cannot.
The pgsql-test harness provides an administrative client, pg, for setup and an application client, db, for testing grants and policies. Superusers bypass RLS, so tests need to exercise the database roles and permissions the application actually uses.
Suppose a project defines app.documents with an ownership policy, and its fixtures insert document 101 owned by Alice and document 202 owned by Bob. Each test runs inside a transaction that is rolled back afterwards, so every test starts from the seeded state:
import { getConnections } from 'pgsql-test';
let db, teardown;
beforeAll(async () => {
// create a fresh database and deploy the project's schema
({ db, teardown } = await getConnections());
});
afterAll(() => teardown());
beforeEach(() => db.beforeEach());
afterEach(() => db.afterEach());
test('Alice sees her document and not Bobs', async () => {
db.setContext({
role: 'authenticated',
'jwt.claims.user_id': 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa'
});
const result = await db.query(
'SELECT id FROM app.documents ORDER BY id'
);
expect(result.rows).toEqual([{ id: 101 }]);
});
The query has no ownership filter. The policy must make Alice’s document visible and keep Bob’s out of the result.
setContext() applies the role and identity settings through SET LOCAL and set_config(..., true), scoping them to the transaction. They supply the identity the policies read. Nothing validates a token. Test identities must match what the application’s policies expect. The RLS tutorial covers the setup.
The harness seeds through pgsql-seed, which loads SQL files, programmatic fixtures, CSV, JSON, or migrations.
The same harness runs under other stacks:
supabase-test supplies Supabase roles, schemas, and authentication defaults.
drizzle-orm-test runs Drizzle queries within the managed transaction.
pglite-test uses in-process PGlite, with no external database service when the schema and required extensions are supported.
graphile-test runs GraphQL against a PostGraphile schema backed by the test database. graphile-realtime-test adds subscription testing.
In one supabase-test run, noted by Supabase CEO Paul Copplestone, 246 tests across 44 databases completed in four seconds.
pgpm, Constructive’s package manager for modular PostgreSQL, scaffolds workspaces with pgsql-test, Jest, and GitHub Actions; getConnections() deploys the module’s plan by default. Start with pgpm init workspace, then add a schema change and a test.
Each change ships deploy, verify, and revert scripts. verify checks that the schema landed. The tests check that it behaves.
The scaffolded workspace extends the same feedback loop into CI with safegres. Tests verify that database behavior is correct, and safegres checks the deployed schema for security and performance regressions. Its CI job enforces a security threshold and compares performance findings against a committed baseline, so a change cannot lower the schema’s security grade or add performance findings.
Postgres enforces the rules. pgsql-test brings verification of those rules into the application development loop.
pgsql-test and its integrations are MIT licensed and available on npm and PyPI.
Get started: Tutorials · End-to-end testing course · npm · Source code
Posted on 2026-09-22 by PostgresCompare
Related Proprietary
PostgresCompare is pleased to announce the release of version 2.2.0, the first public release since 1.2.2 and the largest change in the product's history. The application has been rebuilt on a new foundation, and the 2.x series adds database pipelines, ad-hoc comparison, data-change scripting, an MCP server for AI agents, and a fully translated interface.
PostgresCompare connects to two live PostgreSQL databases, detects schema differences across tables, views, functions, indexes, types, and 30+ other object types, and generates a ready-to-run SQL deployment script to synchronise them. It runs on Windows, macOS, and Linux.
Rebuilt from the ground up
New ways to compare
Data comparison and scripting
Data comparison gained a scripting engine. Differences between table contents can be turned into a SQL script, with row-level selection so only the chosen rows are included, and the script panel highlights the row under the cursor as you work through it. Comparison results can also carry notes, so the reason for a decision stays with the comparison.
Automation and AI agents
Under the hood
Availability
PostgresCompare 2.2.0 is available for Windows, macOS (Apple Silicon and Intel) and Linux (AppImage and .deb). The pgc command-line tool ships for all three platforms. Users on 1.2.2 upgrade by downloading 2.2.0 directly; from 2.x onward the application updates itself. A 14-day free trial is available with no credit card required, and when a trial ends, quick compare and connection management remain available without a licence. PostgreSQL versions 9.2 through 18 are supported.
pnpm 12.6 ships with automatic dependency deduplication, relocatable
node_modules, --save-types for installing @types/* packages alongside
their counterparts, package.yaml manifest editing, and file: / link:
protocols in catalogs.
autoDedupe deduplicates
compatible dependency versions during installation
(#7258). Enable it in
pnpm-workspace.yaml:
autoDedupe: true
When a dependency appears at multiple versions and one satisfies every range, pnpm picks that version for the whole workspace. Frozen installs leave the lockfile unchanged.
node_modulespnpm install, pnpm run, and pnpm exec on macOS and Linux now reuse a
node_modules directory and bin shims that moved or were copied together with
their project (#6937). The first
command after the move checks the tree and records the new location, so project
commands in node_modules/.bin keep working.
--save-typespnpm add --save-types saves available @types/*
packages in devDependencies alongside registry dependencies
(#3868):
pnpm add express --save-types
# saves express to dependencies, @types/express to devDependencies
Packages that declare bundled TypeScript types are skipped. Set saveTypes: true in pnpm-workspace.yaml to enable this by default.
package.yaml manifestspackage.yaml manifests can now be updated by pnpm add, pnpm update,
pnpm remove, pnpm pkg, pnpm link, pnpm set-script, and pnpm version
(#2008). Existing comments and key
order are preserved.
file: and link: catalogsCatalog entries can now use the file: and link: protocols
(#8642). A relative path or bare
path in a catalog entry is measured from the directory holding
pnpm-workspace.yaml:
catalog:
my-lib: link:./packages/my-lib
my-tarball: file:./tarballs/foo.tgz
pnpm tasks statuspnpm tasks status lists running and waiting tasks in each concurrency group,
and waiting tasks now take available slots in order of descending priority,
with arrival order used only to break ties
(#15208). If workspaces use
different limits for the same group, a later task can take a free slot that
earlier tasks cannot use. A package script named tasks takes precedence; use
pnpm pm tasks status when that script exists.
macosBackup.excludeModulesDir
and
macosBackup.excludeStoreDir on
macOS can now exclude newly created modules, virtual-store, and package-store
directories from Time Machine
(#6440). Set either to true in
global configuration or through the
PNPM_CONFIG_MACOS_BACKUP_EXCLUDE_MODULES_DIR and
PNPM_CONFIG_MACOS_BACKUP_EXCLUDE_STORE_DIR environment variables.
pnpm add --tilde is now an alias for --save-prefix=~
(#12863). The Yarn -T
shorthand is not supported.progress setting and --no-progress option turn
off dependency and download progress lines
(#14065). Warnings, lifecycle
output, and the dependency summary are still printed.pnpm cache prune now also deletes registry metadata cache directories that
this version of pnpm can no longer read
(#15046). pnpm cache prune --dry-run lists what it would delete without removing anything.cygpath and wslpath from the system default path
on Cygwin, MSYS2, and WSL2 so a dependency cannot redirect another package's
shim (#14866).pnpm install deprecation warnings no longer carry the text of a package's
deprecation notice, naming only the deprecated package and version
(#15099).pnpm install and other commands now warn when environment variables in
project .npmrc credentials are ignored
(#15051).pnpm install --frozen-lockfile now succeeds when an optional dependency was
unresolvable and skipped by the install that wrote the lockfile
(#3960).pnpm install --frozen-lockfile no longer installs dependencies of projects
removed from pnpm-workspace.yaml
(#15248).pnpm ci now empties node_modules before installing in a project that
declares a clean script
(#15276).pnpm install --force now re-imports every package into the virtual store
(#15030) and removes obsolete
dependency links inside virtual-store packages when their dependencies change
(#15039).preinstall script for the root project now runs before dependencies are
resolved and linked
(#3760).pnpm install --prod no longer downloads registry packages that only a
devDependency reaches
(#881).pnpm install no longer hangs when a git dependency is fetched over SSH and
ssh prompts for a passphrase or host key confirmation
(#2227).pnpm install now reuses an in-flight tarball download when another
resolution of the same archive still needs its package.json
(#15037).pnpm install and pnpm update now resolve a dependency range to the newest
matching version that is not deprecated
(#15128).pnpm add <pkg> without a version now uses the catalog entry when the
workspace already catalogs that package
(#14865).pnpm.overrides entries written as a bare path are now measured from the
directory holding pnpm-workspace.yaml
(#11131).pnpm peers check and strict peer dependency checks no longer reject
compatible versions from named registries
(#15225).pnpm outdated and pnpm update --interactive --latest now include
named-registry dependencies and preserve their registry prefix
(#15226).pnpm dedupe and pnpm install in projects with many convergence
overrides by checking overrides concurrently
(#15175).pnpm run signal handling no longer delivers a redundant second SIGINT on
Ctrl+C, and properly forwards termination signals without a terminal
(#7374).pnpm test now forwards --filter arguments to the test script when the
option follows the shortcut
(#15217).pnpm deploy, pnpm rebuild, pnpm rb, and pnpm setup now prefer a
package.json script of the same name
(#14976).pnpm-workspace.yaml edits now preserve scalar YAML anchors and aliases
(#8245).pnpmfile configuration now loads a .js file as CommonJS or an ES module,
following the nearest package.json
(#15141).readPackage hook changes now take added dependencies out of
pnpm-lock.yaml and update dependencies when an existing lockfile is present
(#3735,
#15136).storeDir values loaded from global configuration now expand a leading ~/
to the user's home directory
(#6560).pnpm install on Windows now runs dependency build scripts from long global
virtual store paths
(#15111).pnpm install across projects sharing a global virtual store on Windows no
longer fails with Access is denied or file-exists errors
(#15114,
#15176).pn, pnpx, pnx, and pnpm now run when Git Bash, MSYS2, or Cygwin
launches them through a Windows path
(#14884).pnpm pipeline --watch now resolves Windows short paths so multiple path
representations share the build cache
(#15105).pnpm remove now runs preuninstall, uninstall, and postuninstall
scripts (#3276).pnpm remove -r now fails before modifying manifests if any requested
dependency is absent from all selected projects
(#2319).pnpm update --peer now updates ranges in peerDependencies
(#8081).pnpm update -g no longer reinstalls unchanged packages
(#12002).pnpm publish now allows a detached Git HEAD in CI environments
(#5894).pnpm store prune now removes unreferenced files from the content-addressable
store (#3635) and expired
pnpm dlx cache data
(#15171).pnpm deploy no longer triggers an install when running scripts in a
read-only deployed filesystem
(#11617).pnpm sbom now validates SPDX identifiers before emitting them as CycloneDX
license IDs (#14786).pnpm --filter and pnpm -F shell completion now suggests workspace package
names (#15216).pnpm run shell completion now suggests package scripts
(#15034).pnpm --version no longer creates a temporary file in the project directory
(#15264).See the v12.6.0 release notes for the complete list of changes.
We're opening up the Context7 database.
Context7 indexes documentation from thousands of libraries, frameworks, and APIs, published and maintained by the library owners. Until today, the only way to use it was a two-step API built for looking up one library at a time. Now there's a single search endpoint. You send a question, and Context7 finds the right libraries and returns the best snippets.
With this new API, Context7 can now be used for grounding. Search engines like Exa ground agents on the open web. Context7 Search does the same for coding agents, with results from official docs only. Think of it as Exa for code.
It's a plain GET request, so you can try it right now by clicking this link:
context7.com/api/v3/search?query=what+is+nextjs
From your terminal, it looks just as simple:
curl -G 'https://context7.com/api/v3/search' \
--data-urlencode 'query=How do I stream an OpenAI response from a Next.js route handler?'
No library IDs, no setup, and you don't even need an API key to try it (requests without a key are for demos only and are rate-limited by IP address). You get back ready-to-use documentation, and every snippet comes with its library and source:
Library: /websites/nextjs
### Stream AI responses using AI SDK in route handler
Source: https://nextjs.org/docs/app/api-reference/file-conventions/route
Streams AI-generated content in a Route Handler using the AI SDK with OpenAI.
```typescript
import { openai } from '@ai-sdk/openai'
import { StreamingTextResponse, streamText } from 'ai'
export async function POST(req: Request) {
const { messages } = await req.json()
const result = await streamText({ model: openai('gpt-4-turbo'), messages })
return new StreamingTextResponse(result.toAIStream())
}
```
The question is about two libraries, but the query doesn't name them. Behind that one call, Context7 picks the relevant libraries, finds matching snippets across them, and reranks everything before returning a compact answer.
Here is the main use case. A coding agent gets a question, searches Context7, and writes its answer from the documentation it found. With the Vercel AI SDK, that is one tool definition:
import { generateText, tool, isStepCount } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";
const searchDocs = tool({
description:
"Search official documentation for libraries, frameworks, and APIs. " +
"Use it before answering any question about how to use a library.",
inputSchema: z.object({
query: z.string().describe("The question to search for"),
}),
execute: async ({ query }) => {
const url = new URL("https://context7.com/api/v3/search");
url.searchParams.set("query", query);
const res = await fetch(url, {
headers: { Authorization: `Bearer ${process.env.CONTEXT7_API_KEY}` },
});
return res.text(); // documentation snippets, ready for the model
},
});
const { text } = await generateText({
model: anthropic("claude-sonnet-5"),
tools: { searchDocs },
stopWhen: isStepCount(5),
prompt: "How do I stream an OpenAI response from a Next.js route handler?",
});
console.log(text);
What happens in that call:
searchDocs.The default text response is designed for this. It is already trimmed to the snippets that answer the question, so the tool result goes straight into the model's context without any parsing. If you want to inspect or filter the results first, add type=json and work with codeSnippets and infoSnippets.
The same tool works with streamText, with any model provider the AI SDK supports, and in any agent loop that can call a function. There is nothing Context7-specific in the agent code; the whole integration is one HTTP request.
Any search API can be a grounding tool. What matters is what comes back.
General search engines like Google, or even AI search engines, index everything. When you search for code, you get a mix of official docs, GitHub issues, Stack Overflow threads, Reddit posts, and old blog posts. Most of that is useful. But some are outdated, written for a different version, or just wrong. For a coding agent that pastes whatever it finds into its context, it's a real risk.
Context7 is safe search for code:
codeTokens), so you can budget context before you add it to a prompt.If your agent knows the library or language, pass it as a hint:
curl -G 'https://context7.com/api/v3/search' \
--data-urlencode 'query=How do I stream an OpenAI response from a route handler?' \
--data-urlencode 'library=Next.js' \
--data-urlencode 'library=OpenAI' \
--data 'language=TypeScript' \
--data 'type=json'
library: a library name or Context7 ID. Repeat it for up to four hints.language: prefer examples in a given language. It's a preference, not a filter.version: ask for a specific release, such as version=15.4.0. It requires at least one library hint.type: txt (default) for text you can add directly to a prompt, or json for structured results.In the tool above, you can expose library and language as optional fields in inputSchema and let the model fill them in when it knows the stack.
Search is also available in the Context7 TypeScript SDK as client.search(query, { libraries, version, language }).
Use the Search API for grounding and quick answers: one request, and Context7 picks the libraries and snippets for you. It fits anywhere you need documentation on demand: agent tools, chat apps, IDE plugins, and code review bots.
Use the Context API when you need to go deep: choose the exact library, ask follow-up questions, and combine results from several libraries yourself. Agents doing deep research use this flow. It is also the better choice when you know exactly which library you want to search in.
Search API calls count as regular Context7 API calls. There is no separate price:
See Plans & Pricing for details.
Open this link, change the query, and check the results. No key needed for a quick demo. For anything real, get an API key from context7.com and read the docs.
If you're building a coding agent, drop the searchDocs tool above into it. That's the whole integration.
dbt is the tool many data teams use to manage their SQL transformations: you write each model as a SELECT statement, and dbt works out the order to run them in from the references between models, builds the resulting tables and views in your database, and can test them along the way.
dbt-duckdb, the dbt adapter for DuckDB, received its first pull request on August 27, 2021, and in the meantime has 1.4k stars on GitHub. Since then, dbt users have been able to install one Python package (dbt-duckdb, via pip), point it at a file (a local DuckDB database), and have a working project (models building into tables and views), without having to sign up to (and pay for) servers or warehouses.
When dbt Labs announced the new Rust-based Fusion engine in May 2025, DuckDB initially wasn't supported out of the box. That has changed with dbt v2, which ships with a DuckDB adapter built in. Here is how to set it up and what else is new.
dbt Labs announced the new Rust-based Fusion engine on May 28, 2025. Two days later, a user, ran-codes, opened a GitHub issue asking for a DuckDB adapter:
Quote “There is a huge community utilizing the DuckDB adaptor to run DBT. For me personally, I was able to learn and start using DBT just because of the light-weight setup for the dbt-duckdb workflow and it has allowed me to get over the learning curve to start using DBT.”
At the time of this writing, the issue resulted in 146 ❤️ and 21 👍 reactions. The adapter is now built into dbt v2.
On June 1, 2026, dbt Labs released the first alpha of dbt Core 2.0, built on the same foundations as Fusion, and open-sourced a large part of the Fusion code. That code moved into the dbt-core repository under Apache 2.0, and the dbt-fusion repository was archived. There are two distributions of v2, both free to install locally and both running on the same engine.
dbt 2.0.0 was released on September 14, 2026. That release also renamed the CLI branding from Fusion and dbt-core to dbt (proprietary) and dbt-oss (open source). So “Fusion” is now mostly the name of the engine, and the thing you install is just called dbt.
In dbt v1, an adapter was a standalone Python package. In v2, adapters live inside a Rust monorepo and connect through ADBC drivers.
With v2, dbt automatically downloads and caches the DuckDB driver the first time you run it, so after you install dbt there is nothing else to add. dbt also publishes a DuckDB quickstart guide for getting a project running locally.
A basic profile looks the same as before:
my_project:
target: dev
outputs:
dev:
type: duckdb
path: ./warehouse.duckdb
v2 adds catalog support that the Python adapter doesn't have. dbt's DuckDB docs flag it as "dbt v2 only"; the legacy Python adapter instead attached DuckLake through the profile's attach block.
With catalogs.yml you can configure DuckLake and Iceberg REST catalogs, with catalog-aware materializations. This requires the v2 engine with the use_catalogs_v2 flag enabled and isn't available in the Python adapter. dbt generates and runs the ATTACH statements for you.
A DuckLake catalog is defined in catalogs.yml:
catalogs:
- name: local_lake
type: ducklake
table_format: default
config:
duckdb:
metadata_path: metadata.ducklake
data_path: s3://my-bucket/lake
Enable the flag in dbt_project.yml, then reference the catalog from a model:
flags:
use_catalogs_v2: true
{{ config(materialized = 'table', catalog_name = 'local_lake') }}
select * from {{ ref('customers') }}
The example above follows the DuckDB catalog support documentation.
v2 also writes its metadata as Parquet as an alternative to the large JSON files, and these (as well as the large JSON files) can be queried directly with DuckDB.
dbt calls this the Information Schema, a v2 feature that stores the manifest as Parquet instead of JSON. Running dbt parse --generate-info-schema writes a set of Parquet files to target/info_schema/v1/, so you can list your models without parsing manifest.json.
These are the same artifacts dbt ships as test fixtures, so you can query one straight from the dbt repository using DuckDB without running dbt first:
SELECT name, materialized, schema_name
FROM 'https://raw.githubusercontent.com/dbt-labs/dbt/main/crates/dbt-docs-server/web/src/test/fixtures/parquet/dbt.models.parquet';
For the above, this lists the three models in the fixture, along with how each is materialized and the schema it lands in:
┌─────────────────┬──────────────┬─────────────┐
│ name │ materialized │ schema_name │
│ varchar │ varchar │ varchar │
├─────────────────┼──────────────┼─────────────┤
│ my_second_model │ view │ main │
│ my_third_model │ view │ main │
│ my_first_model │ view │ main │
└─────────────────┴──────────────┴─────────────┘
Why would you do this? On a large project, the JSON manifest.json can grow to hundreds of megabytes, and reading it means loading and parsing the whole file just to answer a simple question. (Although, DuckDB can do this too.) The Parquet files are columnar, so DuckDB reads only the columns you select and can filter them without materializing everything in memory. That makes it practical to ask questions about the project itself: which models are materialized as tables rather than views, which schema each one lands in, or which models are missing tests.
This is useful in a CI check or an audit script, where you want to enforce conventions across a project without standing up dbt or the warehouse. Because the files are located on disk after a dbt parse, you can point DuckDB at them directly and treat your project's metadata as just another dataset to query.
dbt models combine SQL with Jinja templating, which earlier versions compiled into a query string without inspecting the SQL itself. The v2 engine instead has a native understanding of SQL across multiple engine dialects. That means it can catch invalid column references and type mismatches before a query reaches the warehouse, rather than surfacing them only when the model runs against DuckDB.
That same analysis produces column-level lineage locally, without a dbt platform account. Running dbt compile with --generate-info-schema --static-analysis strict writes a dbt.column_lineage file into the Information Schema Parquet directory covered above, so you can trace which upstream columns feed each model with a plain DuckDB query.
v2 is distributed as a compiled Rust binary rather than a set of Python packages, so there is no Python dependency tree to resolve before a run. dbt describes the engine as the foundation for fast builds on large projects, where parsing and compiling happen inside that single native executable.
The dbt VS Code extension builds on the same SQL comprehension. As you edit models, it gives you autocomplete, hover information, and inline errors, so mistakes show up in the editor instead of after a round trip to the warehouse. The extension is published on the VS Code Marketplace.
v2 ships a pinned DuckDB version, rather than relying on whatever version pip resolves for the Python adapter. Pinning the version is what enables the read-write Iceberg REST catalog support described above, which depends on features from that specific DuckDB build.
Pinning a specific DuckDB version also lets dbt push work down into the database. Some adapter logic that used to be a SQL macro is now implemented as a native DuckDB extension function, such as array_except, which is exposed as sf_array_except.
A low-risk first step is to test the v2 parser while still on dbt v1.12, which ships an opt-in v2 parser. dbt's docs describe this as a way to catch compatibility issues early before fully migrating. Run the following command to check whether your project parses:
dbt parse --use-v2-parser
If it does, follow the install guide to switch. The dbt-autofix package handles many of the required changes, and there is an upgrade guide for v2.
The DuckDB adapter is now part of dbt v2 and needs no separate install, and the Python versions of dbt Core remain available if you'd rather not move or not move yet. Either way, running dbt on DuckDB means you develop, test, and publish your models on your own machine.
Beyond removing the separate install, v2 is where DuckDB picks up several new capabilities: catalog support for DuckLake and Iceberg, metadata written as queryable Parquet, native SQL comprehension with column-level lineage, and a pinned DuckDB build.
If you've already been using dbt-duckdb, upgrading to v2 means one less package to install and all of the above to build on. And if you haven't, a single dbt install and a few lines of profile are enough to start building models directly on your laptop, without servers or warehouses.
Modern applications rarely rely on one database. Data is often distributed across regions, business units, shards, and different technology stacks. Bringing that data together in real time should not require users to build and operate a separate integration architecture for every source.
Redis Data Integration (RDI) is evolving to make that experience simpler.
RDI Software and RDI in Redis Cloud now support multi-source pipelines, allowing users to connect multiple source databases to a single pipeline and load captured and transformed data into one Redis target database.
This makes it easier to consolidate data from systems such as Snowflake, MongoDB, Oracle, MySQL, PostgreSQL, RDS, and Aurora into a unified, low-latency data layer in Redis.
Multi-source pipelines are useful when an application needs data that is distributed across multiple systems but must be available together in real time. For example:
By consolidating data before it reaches the application, multi-source pipelines reduce the need to stitch data together in application code, minimize independent integration deployments, and simplify the path to real-time application experiences.
Multi-source pipelines consolidate your sources. Multi-pipeline support consolidates your deployments, so a single RDI install can serve an entire integration architecture rather than one pipeline within it.
With multi-pipeline, users will be able to operate multiple RDI pipelines as part of a broader integration architecture. This will make it easier to model more complex environments, separate workloads and ingestion flows, and scale RDI deployments as data integration requirements grow.
Multi-pipeline support is coming to RDI soon. More details on availability, configuration, and supported deployment options will follow as the capability progresses toward release.
Multi-pipeline support will strengthen RDI’s role as the integration layer between distributed operational data and Redis applications. Users will be able to evolve from a single integration flow to a more flexible architecture without losing the benefits of real-time capture, transformation, and delivery into Redis.
Multi-source pipelines are an important step toward simplifying distributed data integration. Multi-pipeline support is the next evolution, giving users more flexibility as their environments, workloads, and real-time application needs expand.
RDI is making it easier to turn distributed source data into a unified, actionable Redis data layer.
At the end of August, we announced our first Maintainers in Residence, Rust Project contributors who are funded for their upstream contributions and maintenance work from the Rust Foundation Maintainers Fund (RFMF). Since then, the Rust Leadership Council has dedicated more funds from its Project Priorities budget to RFMF, and together with AWS also providing additional funds, this allowed us to open a new full-time Maintainer in Residence (MiR) position to support the Cargo team. We would like to thank the Rust Leadership Council, AWS, and also the Rust Foundation for providing us with this opportunity! If you would like to help us hire more maintainers to improve Rust, consider donating to RFMF.
This post explains why we chose to support the Cargo team specifically, and introduces Scott Schafer, the new Cargo Maintainer in Residence.
The new MiR full-time position is dedicated to helping with the maintenance of Cargo, our build system and package manager. The Cargo project is deeply involved in many new Rust features, improvements, and Project Goals. Combined with its cross-cutting nature, where it has to support many different use-cases and integrate with several other tools, it takes a lot of work just to keep up with its maintenance needs, let alone support so many feature requests and proposed changes.
Because of that, the Cargo team has sometimes struggled with meeting its maintenance demands. You might remember that for several years, it actually held a feature freeze, to reduce Cargo's internal tech debt, perform necessary refactorings, go through the issue and pull request backlog, and come up with scalable internal development and design processes, so that they could eventually go back to even thinking about adding new features.
Recently, some changes occurred within the team, which made it more difficult for them to meet their maintenance baseline. Some members of the team left, while others lost their dedicated funding for working on Cargo maintenance and had to scale down their involvement. The Funding team thus considered it very important to support this team, given that we had an opportunity to do so. And thus we decided to hire a full-time maintainer to work on Cargo for (at least) the next 12 months.
Even though we know that a single full-time maintainer will not completely solve the maintenance struggles of the Cargo team, we hope that it will improve the situation, and provide a bit of a relief for the team.
We are very happy to welcome Scott Schafer (@muscraft) into the Maintainer in Residence role! Scott has joined the Cargo team three years ago, and apart from working on Cargo, he is also the lead of the Rust Docker team, which prepares official Docker images for every Rust version.
Apart from working on general maintenance of Cargo, Scott has implemented Cargo's Workspace inheritance feature, and has also spearheaded a complex multi-year effort to switch the rendering of diagnostics in the Rust compiler to use the annotate-snippets crate. This effort has been completed in the Rust 1.93.0 release. Thanks to it, the same diagnostics interface can now be shared between the compiler and Cargo (and also other tools), which amongst other things unblocked further development of the Cargo linting system, which has now been stabilized and will ship in the Rust 1.100.0 release.
Everyone we talked about was very excited about Scott becoming a Cargo Maintainer in Residence, and we share that feeling. We wish Scott all the best in his new role, and we are very happy that we can support his maintenance work.
Here is what Scott thinks about it:
I am incredibly excited to work on Cargo full-time! There have been so many things that I wish I could've worked on over the years, that I will now be able to get to. I hope that my efforts will bring Cargo into a more maintainable state.
We are incredibly happy that we keep getting more funds for the Rust Foundation Maintainers Fund, which allows us to support Rust Project contributors. The funding team will be working with the supported maintainers, and also the funders, to ensure that they are all happy with the arrangement, so that we can secure stable funding for Rust maintenance for years to come.
If you would like to help us support more Rust maintainers, consider donating to RFMF!
The Helidon team is pleased to announce Helidon 27, the first release under the Tip-and-Tail model used by OpenJDK.
More at https://medium.com/helidon/helidon-27-released-9ce206503e0a.
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability that enables a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives while retaining TensorRT inference optimizations. It is fully supported starting with TensorRT 11.0.
NVIDIA Dynamo-Triton (formerly NVIDIA Triton Inference Server) release 26.07 enables the multi-device inference capability of the TensorRT backend. One Triton KIND_MODEL instance can own multiple GPUs, create per-rank TensorRT execution contexts, CUDA streams, and NCCL communicators, and launch the ranks together for each request. The application calls one named model through a gRPC endpoint instead of coordinating GPU ranks itself.
For organizations deploying generative AI, this closes the gap between multi-GPU acceleration and a consumable inference service. Teams can trade additional GPU resources for shorter request latency, keep the application interface and surrounding workflow stable, package the engine as a versioned Triton model, and keep rank and communicator lifecycle code out of the client. For latency-sensitive generative media workflows, a shorter time to result can reduce user wait time and accelerate review-and-refine cycles.
This post demonstrates the integration using NVIDIA Cosmos 3 Nano video generation, a long-sequence workload featured in the previous post Scaling AI Inference Across Multiple GPUs Using NVIDIA TensorRT with Multi-Device Inference Support. Diffusers continue to orchestrate prompts, latents, classifier-free guidance (CFG), scheduling, VAE decode, and frame postprocessing. Dynamo-Triton serves the 36-layer denoising transformer, and TensorRT multi-device inference uses Ulysses context parallelism to distribute its 44,160 video tokens across as many as eight NVIDIA GPUs.
The distributed Ulysses graph is compiled into each TensorRT plan before deployment. The Dynamo-Triton TensorRT backend loads the versioned plan, creates the multi-rank execution state, and exposes one gRPC model endpoint. The client sends a transformer request to that endpoint; it does not coordinate the participating GPU ranks.
The Cosmos 3 Nano model provides a practical example of this boundary. The transformer accounts for 93.4% of the single-GPU generation time, making it the highest-impact stage to accelerate. Each of 35 denoising steps requires one negative or unconditional prediction and one prompt-conditioned prediction for CFG. The Diffusers proxy therefore makes two sequential Triton calls per step, for 70 transformer RPCs per generation. Each request carries prepared tensors and returns noise_patches to the application workflow.
The distributed graph is compiled into each context-parallel TensorRT plan. Dynamo-Triton configuration activates that plan; it does not convert a single-device engine into a distributed engine. The single-device baseline uses a standard GPU model instance on GPU 0. The two-, four-, and eight-GPU variants use KIND_MODEL, enable the TensorRT backend multi-device path, and identify the participating ranks.
# Excerpt from the generated CP8 config.pbtxt
name: "cosmos3_cp8"
backend: "tensorrt"
max_batch_size: 0
instance_group [
{ kind: KIND_MODEL count: 1 }
]
parameters [
{ key: "enable_multi_device" value: { string_value: "true" } },
{ key: "multi_device_gpus" value: { string_value: "0,1,2,3,4,5,6,7" } }
]
The fixed Cosmos 3 Nano profile for this example produces 44,160 video tokens. At context-parallel size eight (CP8), each rank processes 5,520 video tokens outside attention. The shorter 2,992-token text path remains replicated. Within each of the 36 transformer layers, Ulysses changes the partitioning axis around attention so that every rank processes the full video sequence for a nonoverlapping subset of heads.
The engine is exported from PyTorch and compiled with Torch-TensorRT. Three local converters lower export-carrier operations to the TensorRT public distributed-collective layer: reduce-scatter, all-to-all, and all-gather. Each accepted context-parallel plan contains two initial reduce-scatters, three all-to-alls in each of 36 transformer layers, and one final all-gather. The resulting topology is two reduce-scatters plus 108 all-to-alls plus one all-gather.
All four variants ran on the same healthy eight-GPU NVIDIA system. The single-device baseline used one GPU; CP2, CP4, and CP8 used two, four, and eight ranks. Every run used 1280×720 output, 189 frames at 24 FPS, and 35 denoising steps.
Each result includes one warm-up followed by five measured complete generations. Timing covers prompt work, the 70 Dynamo-Triton calls, CFG and scheduler updates, VAE decode, and frame postprocessing. Note that model loading and mp4 encoding were excluded.
Table 1 compares SD, CP2, CP4, and CP8 Cosmos 3 runs. End-to-end latency drops from 156.595 seconds on one GPU to 34.183 seconds on eight GPUs, while transformer RPC speedup increases to 6.09 times.
| Variant | GPUs | E2E mean | E2E speedup | RPC mean | RPC speedup | RPC share |
|---|---|---|---|---|---|---|
| SD | 1 | 156.595 | 1.00x | 146.192 | 1.00x | 93.4% |
| CP2 | 2 | 87.999 | 1.78x | 77.548 | 1.89x | 88.1% |
| CP4 | 4 | 53.093 | 2.95x | 42.661 | 3.43x | 80.4% |
| CP8 | 8 | 34.183 | 4.58x | 23.993 | 6.09x | 70.2% |
On one GPU, transformer RPCs account for 93.4% of generation time. At CP8, that share falls to 70.2%. Time outside the measured RPC path remains between 10.2 and 10.5 seconds across configurations, so prompt work, scheduler updates, VAE decode, postprocessing, and other client overhead become a larger fraction of the total.
Every variant used the same seed and generation profile. Validation sampled frames 0, 47, 94, 141, and 188, checked format and temporal variation, and compared each context-parallel output with the single-device result. CP2, CP4, and CP8 passed the configured thresholds of mean absolute error (MAE) ≤ 25 and peak signal-to-noise ratio (PSNR) ≥ 18 dB.
The outputs are not claimed to be pixel-identical. CP2 and CP4 measured MAE 12.759 and PSNR 21.111 dB. CP8 measured MAE 16.316 and PSNR 19.400 dB. The contact sheet also shows the same coherent action across the clip: a robot arm cleaning a plate.
For product teams, these results demonstrate a practical option when response time carries more business value than minimizing the GPUs assigned to one request. A complete Cosmos 3 generation that previously took more than two and a half minutes completes in about 34 seconds, while the application continues to use a conventional model-serving interface.
Teams must still decide on the best approach based on a resource-for-latency trade-off. This benchmark does not measure concurrent request throughput, cost per generated video, or total cost of ownership (TCO). Teams should evaluate these metrics against their own SLOs and deployment economics.
To reproduce the results featured in this post in your own environment, download NVIDIA Dynamo-Triton 26.07 from NGC. Then use the TensorRT, Torch-TensorRT, Diffusers, and Cosmos resources linked.
To learn more, check out these related resources:
Today, we are announcing that xAI’s Grok 4.6 is available in Amazon Bedrock, adding a frontier model built for long-running agents, coding, and knowledge work to the Bedrock model catalog. Grok 4.6 launched on Bedrock on August 18, 2026. It offers a 500K token context window and supports configurable reasoning effort at four levels: low, medium, high, and xhigh.
This is xAI’s second model in Amazon Bedrock. When Grok 4.3 became generally available, xAI joined Amazon Bedrock as a model provider and the model was reachable through Bedrock Mantle, the OpenAI-compatible inference engine in Amazon Bedrock. Grok 4.6 widens that surface area considerably: it is available on both the bedrock-mantle and bedrock-runtime endpoints, and it supports the Converse API alongside Chat Completions and Responses.
This post covers what xAI says Grok 4.6 is designed for, how it is packaged on Amazon Bedrock, and how to send your first request.
The capability and training details in this section come from xAI’s launch announcement, Introducing Grok 4.6.
Grok 4.6 builds on Grok 4.5 with a particular focus on long-running agents and more ambitious interactive and visual work. xAI describes the model as staying with complex tasks across many steps, whether that is researching a topic, analyzing information, working across a code base, or turning an idea into a polished application or work artifact.
On training, xAI reports a longer supplemental training run than Grok 4.5, using curated model-generated data for reasoning and advanced technical concepts, high-quality engineering data, and an improved optimizer and training recipe. It then used Grok 4.5 to regenerate the supervised fine-tuning trajectories across reasoning efforts, agent harnesses, and domains including STEM, software engineering, and knowledge work, filtering out problematic traces with model-based checks. The model was then trained on a wide range of agentic reinforcement learning tasks spanning knowledge work, general coding, and domain-specific environments such as kernel optimization, web development, and computer-aided design.
Two behaviors xAI calls out are worth noting for anyone building agents. On longer trajectories, the model began showing more self-testing and verification, checking its own work before moving on. It also produces stronger first passes on visual and interactive projects, establishing the structure and visual language of an application in a single pass, which the team found useful where the fastest route to a good result was to start with something substantial and then iterate.
On safety, xAI states that Grok 4.6’s safeguards have been improved and calibrated in line with the model’s capabilities, backed by what it describes as its widest-ever suite of pre-deployment testing for capabilities and safeguard calibration, plus post-deployment and third-party testing. The company positions its safety stack as maximizing utility and security across legitimate use cases in domains such as vulnerability patching, accelerating the engineering design cycle, and augmenting AI research.
xAI reports that Grok 4.6 achieves frontier intelligence across several agentic coding and knowledge work benchmarks. These are the figures it published for Grok 4.6 High at launch on August 12, 2026:
| Evaluation | Grok 4.6 High |
| AA Intelligence Index | 61 |
| GDPVal-AA v2 | 1753 |
| CursorBench v3.2 | 69.9% |
| DeepSWE v1.1 | 65.9% |
| FrontierCode v1.1 (Extended) | 61.3% |
| APEX-Agents | 57.5% |
| Terminal-Bench v3.0 | 26% |
| APEX-SWE | 56.4% |
| AA-Briefcase | 1577 |
| Harvey LAB (Vals) | 15.8% |
Source: xAI, according to https://x.ai/news/grok-4-6.
Several of those evaluations come from Artificial Analysis, so it helps to know what they measure. According to Artificial Analysis, the Artificial Analysis Intelligence Index v4.1.1 is a composite that incorporates nine evaluations: GDPval-AA v2, τ³-Banking, Terminal-Bench v2.1, SciCode, Humanity’s Last Exam, GPQA Diamond, CritPt, AA-Omniscience, and AA-LCR. Those cover agentic tool use, reasoning and knowledge, knowledge reliability, long context reasoning, and quantitative analysis over spreadsheets and documents. AA-Briefcase is its agentic knowledge work benchmark, where AA-Briefcase Elo aggregates rubric pass rate, analytical quality Elo, and presentation Elo, with higher scores better.
Artificial Analysis also tracks cost and latency alongside intelligence. Its cost-per-task metric is a weighted average cost per Intelligence Index task, derived from input, cache hit, cache write, reasoning, and answer token prices, which is a useful lens if you are sizing a reasoning-heavy agent workload where reasoning tokens are a real line item.
Several Bedrock capabilities are new for this model rather than carried over from the earlier Grok launch.
The bedrock-runtime endpoint. Grok 4.6 is served on bedrock-runtime in addition to bedrock-mantle, so you can reach it with the AWS SDKs and the standard Bedrock control surface rather than only an OpenAI-compatible client.
The Converse API, including streaming. Both converse and converse_stream are available. This is the practical payoff of runtime support: one message shape across models, and streaming through the usual Converse events (messageStart, contentBlockDelta, contentBlockStop, messageStop, metadata) without hand-rolling server-sent events (SSE) parsing.
An xhigh reasoning effort level. Effort runs low, medium, high, xhigh, extending the range at the top end for problems where a deeper pass is worth the tokens. On Converse, set it through additionalModelRequestFields={"reasoning_effort": "xhigh"} rather than a reasoning parameter.
Cross-Region inference. On bedrock-runtime you route through one of two inference profiles rather than pinning to a single Region. us.xai.grok-4.6 keeps traffic within the US geography when you have data residency requirements, and global.xai.grok-4.6 routes worldwide for the widest capacity pool. Global is also the cheaper of the two, at $2.00 per million input tokens against $2.20, so absent a residency constraint it is usually the better default.
Amazon Bedrock Guardrails. Grok 4.6 now supports Guardrails on bedrock-runtime across its APIs, giving you content filters, denied topics, personally identifiable information (PII) redaction, and word policies. You attach a guardrail by ID and version on the request, and the policy is evaluated against both the prompt and the model’s response. For agentic workloads this matters because it puts a consistent policy boundary around a model that might run unattended across many steps.
Invocation logging. With model invocation logging enabled, Grok 4.6 calls are captured as complete Amazon CloudWatch records: request body, response body, token counts including reasoning tokens, and the inference profile used. Useful for auditing agent runs where you need to see what the model was actually asked.
Prompt caching. Cached input is billed at roughly a quarter of the standard input rate, which matters for agents that resend a large system prompt or document on every turn. Caching applies to a repeated prefix, so keep stable content at the front of the request, and read the cached token count in the usage block to confirm the discount is landing before you build it into a cost model.
Tool calling, structured output, image input, response streaming, and encrypted reasoning content are available as well, but those date from the Grok 4.3 launch and are covered in that post.
Grok 4.6 accepts text and image input and returns text. Audio, speech, video, and embedding modalities are not supported, and it does not generate images. The model is reachable through two endpoints, and the model ID differs depending on which one you use:
| Endpoint | Model ID | Base URL |
| bedrock-mantle | xai.grok-4.6 |
https://bedrock-mantle.{region}.api.aws/openai/v1 |
| bedrock-runtime | us.xai.grok-4.6 (Geo) or global.xai.grok-4.6 (Global) |
https://bedrock-runtime.{region}.amazonaws.com/openai/v1 |
On the API side, Grok 4.6 supports the Responses API, the Chat Completions API, and the Converse API. The Invoke API is not supported.
Feature support differs by endpoint, which is the detail most likely to shape your integration choice:
On bedrock-mantle, supported features include client-side tool calling, reasoning, structured outputs, prompt caching, response streaming, projects, and abuse detection.
On bedrock-runtime, supported features include reasoning, prompt caching, response streaming, invocation logs, and projects (default project only). Structured outputs, server-side tool use, intelligent prompt routing, count tokens, and application inference profiles are not supported on that endpoint.
Tool calling works on both endpoints. The model returns a structured function request, your code executes it, and you pass the result back. On bedrock-runtime you can drive that loop through Converse’s toolConfig or the OpenAI-compatible tools parameter, so agents that depend on function calls are not limited to bedrock-mantle.
If your application depends on JSON Schema structured output, that points you at bedrock-mantle. If you want the Converse API or invocation logging, that points you at bedrock-runtime.
Availability differs by endpoint. On bedrock-mantle, Grok 4.6 is available for in-Region inference in US West (Oregon) (us-west-2) . On bedrock-runtime, in-Region inference is not offered. Instead, you invoke the model through cross-Region inference profiles. Geo cross-Region inference is available from the US Regions (us-east-1, us-east-2, us-west-1, and us-west-2), and Global cross-Region inference is available from a considerably longer list spanning the US, Canada, Europe, Asia Pacific, the Middle East, Africa, and South America. Geo cross-Region routes across Regions within a geography while respecting data residency, and Global cross-Region routes anywhere worldwide when there are no residency constraints. The full table runs to more than 30 Regions, so check the model card and the Regional availability by model page for the current list before you pin a Region.
This is a change in shape from the Grok 4.3 launch, where, as noted in the Grok 4.3 post, the model used in-Region inference only and Geo and Global cross-Region inference were not offered.
Grok 4.6 supports three service tiers. Standard is pay-per-token with no commitment, selected by setting "service_tier": "default" or omitting the field. Priority delivers faster, prioritized processing for a premium ("service_tier": "priority"). Flex offers lower-cost access for work that is not time-sensitive ("service_tier": "flex"). For per-token pricing across the tiers, see the Amazon Bedrock pricing page.
The other two tiers are priced as multipliers on those Standard rates: Priority at 1.75x, a 75 percent premium, and Flex at 0.5x, a 50 percent discount. So the same workload that costs $2.20 per million input tokens on Standard in-Region runs $3.85 on Priority and $1.10 on Flex, which makes tier selection a larger cost lever than the Region choice.
For reference, xAI lists Grok 4.6 pricing starting at $2 per million input tokens and $6 per million output tokens, with a fast variant at twice the price. Always confirm current rates on the Amazon Bedrock pricing page, because prices and tiers change.
Before your first call, confirm the model is available to you in the Bedrock console for the Region you plan to use. Grok 4.6 is served through inference profiles rather than on-demand throughput on the bare model ID, which is why requests name us.xai.grok-4.6 or global.xai.grok-4.6 on bedrock-runtime.
Grok 4.6 uses OpenAI-compatible APIs, so the OpenAI SDK works against either endpoint after you set the base URL. Install the SDK, and boto3 if you plan to use the Converse API:
pip install openai
pip install boto3Generate a long-term Amazon Bedrock API key from the Amazon Bedrock console for exploration, then set your environment. For bedrock-mantle:
export OPENAI_API_KEY="<provide your Bedrock API key>"
export OPENAI_BASE_URL="https://bedrock-mantle.us-west-2.api.aws/openai/v1"For bedrock-runtime:
export OPENAI_API_KEY="<provide your Bedrock API key>"
export OPENAI_BASE_URL="https://bedrock-runtime.us-east-1.amazonaws.com/openai/v1"A first request on bedrock-mantle with the Chat Completions API:
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="xai.grok-4.6",
messages=[
{"role": "user", "content": "Can you explain the features of Amazon Bedrock?"}
],
)
print(response)On bedrock-runtime the difference is the model name: you pass a cross-Region inference profile instead of the bare model ID. This example also switches to the Responses API to show that shape:
from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="us.xai.grok-4.6",
input="Can you explain the features of Amazon Bedrock?",
)
print(response)And through the Converse API with boto3. Because reasoning is active, the first content block carries the reasoning and the answer sits in a later block, so search the blocks for the text rather than indexing content[0]:
import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.converse(
modelId="us.xai.grok-4.6",
messages=[
{"role": "user", "content": [{"text": "Can you explain the features of Amazon Bedrock?"}]}
],
inferenceConfig={"maxTokens": 2048},
)
blocks = response["output"]["message"]["content"]
text = next(b["text"] for b in blocks if "text" in b)
print(text)On Converse you set the effort level through additionalModelRequestFields rather than a reasoning parameter:
response = client.converse(
modelId="us.xai.grok-4.6",
messages=[{"role": "user", "content": [{"text": "What is 17*23? Number only."}]}],
inferenceConfig={"maxTokens": 3000},
additionalModelRequestFields={"reasoning_effort": "xhigh"},
)Three operational notes. First, on bedrock-runtime, Grok 4.6 is not available for in-Region inference, so requests must name us.xai.grok-4.6 or global.xai.grok-4.6.
Second, bedrock:InvokeModel is evaluated against three resources: your account’s default project, the inference profile you name, and the underlying foundation model. The foundation model ARN is wildcarded across Regions because cross-Region profiles route outside the calling Region. Bearer-token authentication on the OpenAI-compatible endpoints additionally requires bedrock:CallWithBearerToken, which boto3 and Converse do not need:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "bedrock:InvokeModel",
"Resource": [
"arn:aws:bedrock:{region}:{account-id}:project/default",
"arn:aws:bedrock:{region}:{account-id}:inference-profile/us.xai.grok-4.6",
"arn:aws:bedrock:*::foundation-model/xai.grok-4.6"
]
},
{
"Effect": "Allow",
"Action": "bedrock:CallWithBearerToken",
"Resource": "*"
}
]
}List every inference profile you plan to call. Profiles are scoped individually, so a policy naming us.xai.grok-4.6 does not cover global.xai.grok-4.6.
Third, the two authentication mechanisms cover different code paths. An Amazon Bedrock API key in OPENAI_API_KEY travels as a bearer token and authenticates the OpenAI-compatible calls on both endpoints. The boto3 Converse examples sign with SigV4 instead, drawing on your ordinary AWS credentials from the environment, a profile, or a role. Configure both if you intend to use Converse alongside the OpenAI-compatible APIs.
Treat a long-term API key as an exploration-only credential. For production, the Grok 4.3 launch post recommends short-term bearer tokens generated from your IAM credentials with the aws-bedrock-token-generator package, because they expire automatically and keep access tied to your IAM identity, and that guidance applies equally here.
Reasoning is active on Grok 4.6 by default, and you configure how much of it the model spends through the reasoning parameter with low (the default), medium, high, or xhigh. The xhigh level is new relative to what the Grok 4.3 launch post documented, where the levels were none, low, medium, and high.
Reasoning content is encrypted. You can have it returned by passing include: ["reasoning.encrypted_content"] on a Responses API request, then send that content back on subsequent turns to give the model its own prior reasoning as context in a multi-turn conversation. The Chat Completions API does not return reasoning tokens.
Encrypted reasoning is a Responses API feature, so this example uses the OpenAI client rather than the boto3 client from the Converse examples above:
from openai import OpenAI
client = OpenAI() # OPENAI_BASE_URL points at the bedrock-runtime endpoint
response = client.responses.create(
model="us.xai.grok-4.6",
reasoning={"effort": "high"},
include=["reasoning.encrypted_content"],
input="Explain quantum entanglement simply.",
)
print(response.output_text)Because reasoning is by default and effort is per request, effort level is a real cost and latency control. Run short extraction and classification calls at low, and reserve high or xhigh for planning steps and long agent trajectories where an early mistake compounds. Benchmarking effort levels against your own workload is the fastest way to find where higher reasoning stops earning its token cost.
Grok 4.6 on Amazon Bedrock gives you a model xAI built for long-running agents and ambitious interactive work, with a 500K token context window, four reasoning effort levels, image input, prompt caching, and a choice between the OpenAI-compatible bedrock-mantle endpoint and the bedrock-runtime endpoint with Converse API and cross-Region inference support.
To start building, review the Grok 4.6 model card for the current Region list, feature matrix, and parameter details, and check the Amazon Bedrock pricing page for token rates. If you generated a long-term Amazon Bedrock API key for exploration, delete it from the Amazon Bedrock console when you are finished. A standing credential you no longer need only widens your account’s exposure surface.
Suheel is a Principal Solutions Architect at AWS, specializing in artificial intelligence, machine learning, and generative AI. He helps Foundation Model Provider customers design, build, modernize, and scale their AI/ML and generative AI workloads on AWS. His experience spans the AWS AI/ML and generative AI portfolio, particularly Amazon Bedrock, Amazon Bedrock AgentCore, and Amazon SageMaker AI. In his free time, Suheel enjoys working out and hiking.
Ikenna is a Principal Solutions Architect at AWS specializing in networking, containers, and AI infrastructure. He guides model providers through scaling their training and inference systems while enabling rapid deployment of evolving frontier models on AWS. His work increasingly spans agentic AI – building reliable, cost-efficient multi-agent systems and the inference infrastructure behind them in production.
Fabio is a Senior Customer Solutions Manager at Amazon Web Services (AWS) and strategic advisor guiding foundational model providers in their go-to-market journey. Prior to AWS, he held Product Management, Engineering, Consulting, and Technology Delivery roles across multiple Fortune 500 companies in industries, including retail and consumer goods, oil and gas, financial services, insurance, and aerospace and defense.
Saurabh is a Senior Product Manager for Amazon Bedrock and Amazon SageMaker Inference. He is passionate about working with customers and partners, motivated by the goal of democratizing AI. He focuses on core challenges related to deploying complex AI applications, inference with multi-tenant models, cost optimizations, and making the deployment of generative AI models more accessible. In his spare time, Saurabh enjoys hiking, learning about innovative technologies, following TechCrunch, and spending time with his family.
Anirban is a Principal Engineer at AWS based in Seattle, USA, where he focuses on the design of secure, high-scale model-serving infrastructure for Amazon Bedrock. He has driven the technical work behind several foundation-model launches on the platform. Prior to joining Amazon Bedrock, he was a Principal Engineer on AWS Outposts, building hybrid on-premises cloud infrastructure.
Jev is now available as a judge for evaluations in LangSmith. Jev gives teams a fast, low-cost way to evaluate open-ended agent behavior and turn the results into structured feedback they can track in LangSmith.
Below, we explain why a System One model like Jev is useful for agent evals, share what we found when we tested it, and walk through setting up a Jev-as-a-judge evaluator for online evals.
Try Jev-as-a-judge in LangSmith today by visiting the Evaluators tab in any tracing project.
Back in 2023 when we first started building agents (which we mostly called LLM apps at the time), the primary approach to evals was code-based. Later that year, researchers introduced LLM-as-a-judge, and since then, code-based and LLM-as-a-judge have been the two main ways to evaluate agents.
Code-based evaluators check for specific, deterministic conditions: Did the agent call a tool? Does the output match a pattern? Is a field present? That's fast and reliable, but it only covers the narrow slice of agent behavior you can fully specify before the agent runs. Since agents are non-deterministic, an agent that solves the same problem three different valid ways will fail a code-based check that only expects one of them.
LLM-as-a-judge evaluators fill that gap. You give an LLM judge an agent trace, along with instructions and a rubric on how to grade it, and it reasons through the trace in free text before returning a verdict. However, that flexibility comes at a cost. LLM-as-a-judge evaluators are slower and more expensive to run than a function call, and because they're non-deterministic, the same input can produce a different verdict from one run to the next. On top of that, the step that turns free text into a structured output is itself a source of error, independent of whether the judge's reasoning was correct.
Now, System One models like Jev introduce a third type of agent eval, one that trades some of code-based evaluation's speed for the flexibility to evaluate open-ended agent behavior, at a fraction of the cost of an LLM judge.
Jev isn't a traditional LLM and doesn't generate text. The TypeSafe AI team calls it a System One model:
📖 System One models are a class of AI models built to make fast, structured decisions that software can use directly. A System One model evaluates a state and returns typed answers and probabilities.
For evals, the state can be an agent trace, a single message, or any other context you want evaluated. Questions define the criteria you want to evaluate the state against, like whether a response leaked PII, what the user's intent was, or how frustrated the user seemed. Jev can answer three types of questions: (1) a noul returns a yes/no probability, (2) a choice picks one option from a set, and (3) a score rates the state on an ordered scale. Each answer comes back typed, instead of a block of generated text that gets converted into structured output.
The three question types Jev can answer, using the feedback keys from this post: PII leakage (noul), user intent (choice), and user frustration (score).
Three things about Jev map directly onto pain points in agent evals. According to TypeSafe AI, Jev is up to ~450x cheaper and ~200x faster than comparable LLMs on classification tasks, and it can evaluate multiple questions about the same state in parallel.
Cost is a common reason teams evaluate their agents less than they would like to. Every eval carries a trade-off: score more agent runs, evaluate more criteria, or test more changes, and the cost of testing grows proportionally. With multiple agents and high-volume usage, an LLM judge that costs a few cents per eval gets expensive fast across production traffic, large datasets, and regression tests against every model or prompt change. Teams end up running fewer evals to manage costs, which slows down the feedback loop that building great agents depends on. At a fraction of that cost, a Jev judge can remove that trade-off.
With Jev-as-a-judge, you can score every trace instead of a sample of them, check more criteria per trace, and run the same judgment repeatedly to see how consistent the judge is. The cheaper the judge, the more of your agent's behavior you can afford to evaluate, and the tighter that agent improvement loop becomes.
Speed matters a great deal for online evals, where a judge is scoring live traffic. Being up to ~200x faster than an LLM judge, a Jev judge is better at keeping pace with traffic as it arrives. That matters most for feedback keys that flag security or safety risks, like PII leakage, prompt injection, or toxicity, where you can set an alert on the feedback key that triggers a webhook to automate a response. The faster the judge, the smaller the window between something going wrong and something being done about it.
Parallelization changes how many criteria you can evaluate against a single agent trace. Jev evaluates every question in a request together, so scoring a trace against multiple feedback keys, such as PII leakage, user intent, and user frustration, costs only marginally more than scoring it against one.
💡 System One models evaluate every question in a request in parallel. Adding questions barely changes the response time and costs only the tokens for the extra questions, which are cheap.
An LLM judge, by contrast, either needs a separate call per criterion or has to reason through all of them sequentially in one prompt with output tokens scaling with the number of criteria.
System One models map well onto these pain points, but none of this makes LLM judges obsolete. Fine-tuned and open models can be effective judges at much lower cost than a frontier model, and for open-ended criteria where you want written reasoning alongside a verdict, an LLM judge is still the better tool. Jev is a good fit when the decision you need is narrow and typed and you are making it at volume.
We put Jev to the test in Jev-as-a-Judge for Agent Evals, comparing it against GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6 on accuracy, consistency, speed, and cost. Jev was more accurate, dramatically more consistent, and both faster and cheaper than the LLM judges.
Jev matched a human reviewer on every decision, with 92-913x lower variance than the LLM judges. It averaged 0.44 seconds per call, compared to 2.16-2.83 seconds for the LLM judges. At $0.00035 per call, running the full set of judgments cost $0.34 with Jev, versus $0.39 with GPT-5.6 Luna, $2.90 with GPT-5.6 Terra, and $28.17 with Claude Sonnet 4.6.
This was one test on one agent, but the results are a promising early sign that Jev-as-a-judge is a viable third type of agent eval, alongside code-based and LLM-as-a-judge.
TypeSafe is now a model provider in LangSmith, with Jev available as a model. Setting up a Jev-as-a-judge evaluator follows the same path as an LLM-as-a-judge evaluator. The key difference is that a Jev-as-a-judge evaluator defines a state and a set of typed questions instead of a prompt and evaluation criteria.
Jev-as-a-judge is available in LangSmith today.
Sign in or sign up for LangSmith, then open the Evaluators tab in any tracing project, add an LLM-as-a-Judge evaluator, and select TypeSafe as the provider to try it out. For more details on online evals, including filters and advanced options, see the online evaluators guide.
If you try Jev as a judge on your agents, we want to hear how it holds up, especially against the LLM judges you use today. Share what you find on the forum or tag us on X.
To see how Jev fits into the agent loop beyond evals, including model routing and tool-risk gating, read Building a harness with Jev.
If you want to learn more about building agents with Jev, we're hosting a livestream with the TypeSafe AI team on Tuesday, Sep 22nd: https://events.langchain.com/webinar/building-a-harness-with-jev/
Vercel Connect now includes a managed connector for Microsoft Teams. Creating one gives your organization a Teams bot that your apps and agents run. People can @mention it in channels or message it directly, and your code receives the message and replies as the bot.
As a Vercel Managed Connector, Vercel registers the Entra app and Azure Bot resource in your tenant, so there's no client secret to store. A tenant administrator with an Azure subscription completes setup once. Incoming Teams activities are verified and forwarded to your project as Connect trigger.
Create a connector from the dashboard or Vercel CLI:
Once the connector is set up, your code requests a token only when it needs one. Use it with the @vercel/connect SDK, eve channel or Chat SDK adapter:
Each token is scoped to what you request and refreshed automatically, so there's nothing to rotate by hand. Connectors only work in the environments you attach them to, and you can revoke access at any time with vc connect revoke-tokens.
Read the Vercel Connect documentation, view details about the Teams connector in the Vercel Connect catalog, or create a Teams connector to get started.
The OpenTelemetry project is excited to announce the 2026 OpenTelemetry Governance Committee (GC) election. Nominations are due by 16 October 2026 23:59 AoE. The list of eligible candidates will be shared on 19 October 2026. Voting will take place between 26 October 2026 12:00 UTC and 28 October 2026 end of day AoE (29 October 2026 11:59 UTC), and the final election results will be announced 30 October 2026.
If you are a member of standing in the OpenTelemetry community, we invite you to participate with your vote in this election to ensure that the community is well-represented in the Governance Committee. In this election four people must be elected, each with two-year terms.
If you have made contributions to our ecosystem not measured by the automatic process, you can request an exception before 23:59 AoE on 23 October 2026 to participate in the election. See the voter roll with all members of standing and approved exceptions. Approved exceptions will be added to the roll continuously.
Voting will be open between 26 October 2026 12:00 UTC and 28 October 2026, end of day, Anywhere on Earth (29 October 2026 11:59 UTC) on Helios Voting; voters will need to sign in with their GitHub account.
For more information, see the OpenTelemetry 2026 Governance Committee election.
If you’ve been working on OpenTelemetry and seeing it grow or you’re an end-user who wants to help us make OpenTelemetry better, now’s the time to consider running for a seat on the Governance Committee. You can read about the Governance Committee’s role in this blog post or refer to the charter document. You may nominate yourself (or others!) by submitting a Pull Request against the list of candidates by 16 October 2026 23:59 AoE — see the detailed requirements under nominations for the Governance Committee election.
We would like to thank the GC members whose term expires this year; they have helped grow OpenTelemetry, and invite them to run for re-election if they so choose: Alolita Sharma, Morgan McLean, Pablo Baeyens, and Trask Stalnaker.
For any election related questions, please file an issue on the
community repository
tagging the @open-telemetry/otel-elections team, email us at
cncf-opentelemetry-governance@lists.cncf.io,
or reach out to us on CNCF Slack in
#opentelemetry if you have
an urgent access issue during voting.
See you at the polls!
By default, Render rebuilds your code every time you deploy, even if the code hasn't changed from the previous build. But if you're deploying the same commit to staging then promoting to production, or running multiple services from one repository, you shouldn't have to wait for Render to rebuild identical code.
You can now save time and compute by reusing builds across multiple Render services. Reusing builds also ensures that the correct artifact is promoted between dev, staging, and production environments.
Starting today, we are rolling this feature out in Private Beta to select customers. To request access, fill out this form, and our product team will be in touch when we're ready to onboard you.
If you run two or more services on Render built from the same repository or image (a web service and its workers, multiple services within a monorepo, or the same service deployed across staging and production) reusing builds can usually save you time and money. The benefit increases with the number of services sharing a build and the time each build takes.
Reusing builds when promoting between environments also ensures that those environments don't drift apart because of changes in build-time variables or in how dependencies resolve.
Define a Build Source once by specifying a repository, branch, and build command, and Render produces one immutable build artifact. Any linked service across development, staging, and production can deploy that exact artifact with no rebuild.
For services linked to a Build Source, build-time and runtime variables are now scoped separately, so runtime secrets aren’t available during the build unless you explicitly pass them as build-time variables. This separation makes it safer to promote the same build across environments instead of rebuilding it with a different set of credentials.
Currently, you can reuse builds for web services, private services, and background workers. This allows you to:
See the documentation for current limitations.
During Beta, we plan to add cron job support, a fuller Dashboard experience, and CLI and Terraform support.
Capacity is limited during this phase. To request access, fill out this form. We'll review your request and reach out when we are ready to onboard your team.
Once approved, you’ll:
Your use case and feedback will help shape Build Reuse as we work toward General Availability.
Living in the Netherlands, I spend a fair amount of time on trains, and that is usually where I catch up on what the builder community is writing. Until now, that meant opening a laptop or squinting at a browser tab on my phone. This week I found myself scrolling through trending articles and checking a workshop from the AWS Builder Center mobile app while waiting for a delayed train, and it made those spare twenty minutes very useful. That is why I am glad to open this week with the Builder Center mobile app.
AWS Builder Center is now available as a mobile app on iOS and Android, extending the experience beyond desktop and web. Using your AWS Builder ID, you stay signed in across sessions and can browse trending articles, access 600+ AWS Skill Builder courses, and manage hands-on workshops with free sandbox environments from your mobile device. You can follow AWS Heroes, Community Builders, and User Group Leaders, check Builder Loft event calendars on the go, and receive push notifications for subscribed topics and communities. The app also supports the Wishlist feature for submitting product feedback directly to AWS teams. It is available worldwide on the Apple App Store and Google Play Store.
Builder Center also added two features this week. Polls give you a way to ask the community a question from the Home feed: write a question, add 2 to 5 answer options, set a deadline, and people vote, with results updating live and discussion happening in the comments. Votes are anonymous, and creators see aggregate counts and percentages only. Separately, the Zero to Shipped hackathon is open from September 18 to October 2. You connect your coding agent to AWS, build a real application, and ship it live on AWS for a chance to win a share of a $28,000 prize pool. Five winning projects each receive $5,000 in AWS credits and an AWS Builder swag bundle.
Last week’s launches
Here is what else happened this week.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts you may find useful:
For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
Summer has officially given way to September, but the weather where I am has not quite caught up. The days are still unusually warm, and I suspect these are the last mild afternoons before autumn settles in for good. I am making the most of them while they last. Come back next week for more!
— Esra Posted by Jan Kleinert, Developer Relations Engineer, Android for CarsToday, the games category for Android Auto and cars powered by Android Automotive OS with Google built-in is officially graduating from beta to general availability. Our early access partners have already been bringing games to the parked-only experience for cars, and you can browse these in our latest collections of games for Android Auto and games for Android Automotive OS.
Bringing your game to cars lets you reach users in their vehicles during natural downtime, such as while waiting at a charging station or for a curbside order pickup. Today's milestone means that we're opening up access so developers can now publish games to the open testing and production tracks on Google Play. In this post, we'll cover how to adapt your existing Android game for the car screen, focusing on key technical requirements and publishing criteria.
If you're already following best practices for building adaptive apps, bringing an existing Android game to cars primarily involves configuring your app manifest and ensuring your game respects the vehicle parked state.
To distribute your app in the games category, you need to explicitly declare its category. Add the android:appCategory="game" attribute to the <application> element of your manifest file:
<application ...
android:appCategory="game">
...
</application>Games are supported on Android Auto on devices running Android 15 and higher. To declare that your game supports Android Auto, include this <category> element in the intent filter of an activity in your manifest file:
<activity ...
<intent-filter>
<action android:name="android.intent.action.MAIN" />
...
<category android:name="android.intent.category.CAR_LAUNCHER" />
</intent-filter>
</activity>Generally, the android.intent.category.CAR_LAUNCHER category element is placed in the same intent filter as the android.intent.category.LAUNCHER element, but it can be in another activity's intent filter if you prefer to launch a different activity.
To declare that your game supports Android Automotive OS, include the android.hardware.type.automotive <uses-feature> element in your manifest file.
<manifest ...
...
<uses-feature android:name="android.hardware.type.automotive"
android:required="false" />
...
</manifest>The android:required value has different restrictions depending upon which track you choose to distribute your Android Automotive OS app. If you distribute your Android Automotive OS app on the mobile track, android:required must be set to "false". However, if you distribute on the Android Automotive OS dedicated track, you can set android:required to "true", "false", or leave it unset. Leaving the value unset has the same effect as setting android:required to "true", and means that your app is available only for distribution on Android Automotive OS devices.
Cars introduce a unique physical context with a driving state and a parked state. Certain types of apps, like games, are considered parked apps and aren't permitted to run while the vehicle is in motion to avoid driver distraction. By default, Android Auto and Android Automotive OS block activities from being used or launched when the vehicle is in motion or when user experience (UX) restrictions are active. To make sure your game complies with driver distraction guidelines, don't include the distractionOptimized metadata element in any activity in your manifest. You must also ensure that your game audio stops when the user starts driving and can't be unpaused while the vehicle is in motion.
Additionally, when the user relaunches the app from the home screen, your game must restore the app state as closely as possible to the previous state. Test your game for responsiveness and ensure it doesn't freeze or stutter during gameplay.
Car screens support touch input, but many users prefer playing with a connected gamepad. If your game supports controller input, declare the android.hardware.gamepad feature in your manifest to help boost the visibility of your app in the Google Play Store to users specifically seeking controller-compatible experiences.
<uses-feature android:name="android.hardware.gamepad" android:required="false"/>Set the android:required attribute to false to indicate your app supports controllers, but the use of controllers is optional. Don't set the android:required attribute to true unless a controller is mandatory for your game.
Car displays come in various shapes and aspect ratios, including portrait and wide landscape screens. For a great user experience, make your game fully adaptive to different screen sizes so that it runs full screen without letterboxing or pillarboxing. For Android Auto, refer to the guidance for testing against canonical screen sizes and use bundled hardware profiles when testing with the emulator for Android Automotive OS.
After you've implemented the necessary changes, you can opt in to Android Auto and Android Automotive OS form factors in the Google Play Console. Before submitting to production, test your game against the car app quality guidelines for games.
Use the Desktop Head Unit to test your app's Android Auto compatibility, and use the Android Automotive OS emulator to test the experience on Android Automotive OS. Your game will be reviewed against the car app quality guidelines for the games category before it is approved for open testing or production.
With the games category now generally available, it is the perfect time to optimize your titles for cars. To learn more about implementation details, review the documentation at Build games for cars.
Given a set of objects and a set of bins, how do we assign objects to bins in a way that optimizes specific objectives while meeting certain constraints?
This question arises at all layers of Meta’s infrastructure stack including in
The main challenges to designing a reusable framework for solving problems like these are its usability and scalability. Usability is impeded by practitioners struggling to translate real-life policies into the precise mathematical formulas required by formal optimization methods, while scalability is hampered by NP-hard problems that cannot be solved efficiently by commercial solvers.
Rebalancer addresses both of these challenges by separating a problem’s specification from its solution. Rebalancer provides a language for describing problems using objects, bins, constraints, and objectives, as in the examples above. Once a problem is described in this way, Rebalancer transforms the problem into a directed-acyclic graph called an expression graph. Rebalancer’s solving algorithm uses the expression graph to either design a local search heuristic or to build a mixed integer program (MIP) solvable with either a commercial (FICO Xpress or Gurobi) or open source solver (HiGHS).
Rebalancer’s specification language employs a three-step approach to incrementally elevate the level of abstraction for ease of use.
In the example above, tasks are modeled as objects and servers are modeled as bins into which tasks are to be placed. Servers are physically situated in racks; this grouping is modeled as a scope. Tasks take a certain amount of CPU and storage and servers have a limited amount of each. CPU and storage are modeled as dimensions. The CPU and storage utilization of a server corresponds to the sum of all of the tasks assigned to that server and the server’s utilization limits are modeled using a CapacitySpec. The expression API could be used to change how the utilization is calculated if a simple sum isn’t appropriate.
Further, we model tasks as belonging to jobs. A grouping of objects like this is called a partition and we use a GroupCountSpec to ensure that each rack has only a single job type (partition) assigned to it. A BalanceSpec ensures that each server’s utilization is balanced across both its CPU and storage dimensions.
This example demonstrates how complex assignment problems can be easily and naturally constructed using Rebalancer and how specs provide a way of expressing constraints and goals that can be re-used in many different ways by varying dimensions, scopes, or partitions.
Please check out an exhaustive list of Rebalancer specs in the docs.
Once a problem is specified using the API described above, Rebalancer translates it into an expression graph. The leaf nodes in this graph represent utilization expressions; for example, the memory utilization of server A, as obtained by summing the memory contribution of tasks assigned to server A. These utilization values are then recursively composed using aggregation nodes such as Max and Sum, or transformation nodes such as Square and Abs. Note that the value of each node in the expression graph depends on the current assignment and needs to be updated every time the assignment changes.
Along with the problem objectives and constraints, modelers also provide Rebalancer with an initial assignment and a stopping condition, such as a time limit. Rebalancer will compute an optimized assignment that minimizes the objective value and does not violate any new constraints. The constraints that were violated by the initial assignment become high priority goals and their violation is minimized, ideally to zero.
Rebalancer offers two distinct techniques to solve the assignment problem.
Optimal Solver. In this mode, Rebalancer translates the expression graph into a set of expressions that can be fed into MIP solvers such as FICO Xpress, Gurobi, or HiGHS. During this translation, Rebalancer needs to represent utilization of a bin by a weighted sum of binary decision variables (one per object) that indicate if the object is assigned to the bin; this can lead to very large MIP models! Rebalancer automatically uses techniques such as variable aggregation (compacting similar objects into a single integer variable), interchangeability, and symmetry breaking to reduce model sizes, but the worst case size of the generated MIP model can still be quadratic, i.e. O(|objects| * |bins|). The largest problems we consider are too big for any MIP solver.
Local Search Solver overcomes this limitation by working directly on the expression graph, exploring the local neighborhood around the current assignment by moving some objects to another bin. This neighborhood has a worst-case size of O(|objects|+|bins|), which allows Rebalancer to model even very large problems without hitting memory limitations. Each move creates a new candidate assignment for which Rebalancer evaluates the new values of the objectives and constraints. After all candidates have been evaluated, Rebalancer applies the best candidate assignment; that is, the one that does not violate a constraint and improves the objective by the maximum amount. This process of evaluating and applying moves is repeated until no progress can be made or a stopping condition is reached. Rebalancer’s local search algorithm is heavily optimized and parallelized so that each evaluation is relatively inexpensive (millions of evaluations per second are possible), allowing us to quickly explore the search space. In addition, Rebalancer knows how to prune the search space, cutting down the number of evaluations needed in the first place.
The right solution technique will depend on your needs. At Meta, almost all large-scale problems use local search. Small- to mid-size problems that have moderate solve time requirements often use the optimal solver. It is also common to prototype with the optimal solver and then migrate to local search after a high-quality baseline solution has been identified. Offline, the optimal solver can be used to tune local search.
For the last decade, Rebalancer has been continuously used and improved at Meta. It is used to solve a wide range of infrastructure optimization problems including assigning shards to servers (Shard Manager), servers to services (RAS), routing traffic from globally distributed edge datacenters to main datacenters (Taiji), grouping serverless functions to improve locality, balancing online ML training workloads across regions while considering the priority of ML workloads, and so on. At the time of this writing, Rebalancer is used to solve roughly 40 million assignment problems every day with more than 30 unique problem formulations. The P99 solve time is 12 seconds on a problem with 265k objects and 3.2k bins. For problems with more than 1 million objects and 5k bins, the average solve time is 171 seconds and there are more than 3.4k such runs.
Unsurprisingly, Rebalancer has also been used to solve non-infrastructure problems such as assigning meetings to meeting rooms to minimize travel time, assigning support tickets to engineers, and optimizing desk placements. Beyond Meta, assignment problems arise in many domains such as healthcare, energy and utilities, transportation and logistics, education, and emergency response, and, while we don’t have the expertise to apply Rebalancer to these areas ourselves, we hope that others do and will.
With Rebalancer making it easy to formulate and solve problems, we found that the majority of engineering time for modelers shifted to debugging the solver’s behavior. Without proper tools, such debugging required a deep understanding of the solver’s internals.
Over time we identified common questions and pain points among modelers and built a specialized UI tool for answering them: Rebalancer Explorer.
Explorer accompanies Rebalancer in this open-source release as a Dockerized web UI that facilitates rapid debugging and iteration when solving problems with both local search and optimal solvers. It helps answer questions such as which constraints are binding, what would happen if a constraint were relaxed, and why an object was placed in one bin and not another.
We are always looking to optimize Rebalancer’s performance, add new capabilities, and extend it to support a wider range of assignment problems. Rebalancer is proud to be open-source (Apache 2.0 license) and we invite both systems and optimization experts to try Rebalancer and contribute to the project by identifying performance bottlenecks, adding new solve techniques, extending it to support new sorts of problems, or just by fixing bugs. We look forward to seeing how the systems and optimization communities adopt, build, and contribute to Rebalancer.
Rebalancer was developed by past and present members of the Algorithmic Optimization team at Meta: Pol Mauri Ruiz, Igor Kabiljo, Neeraj Kumar, Vijay Menon, Mayank Pundir, Andrew Newell, Liyuan Wang, Richard Barnes, Sahil Deshpande, Karthik Velakur, Yang Liu, Leart Gjoni, Ravi Surulikamu, Tony Zhang, Raj Rajendran, Aravind Narayanan, Lakshmi Ganesh, and Saranyan Vigraham.
The post Open-Sourcing Rebalancer: A Generic, High-Performance Library for Solving Assignment Problems appeared first on Engineering at Meta.
A resilient software supply chain is the foundation of modern delivery, and securing your continuous integration and continuous delivery (CI/CD) pipeline is what keeps innovation moving safely.
Notable supply chain attacks more than doubled in the first half of 2026 compared to the second half of 2025, according to Wiz’s recent Cloud Threat Highlights report.
It’s crucial that your source code not be the weakest link in your private cloud. To help you better address software supply chain threats, Google Cloud Secure Source Manager (SSM) lets you manage your source and CI/CD systems with unified authentication and authorization mechanisms.
We now offer two new capabilities, both generally available, that can simplify and secure your development and CI/CD workflows:
Unauthorized access to CI/CD systems: Attackers only need to alter a single deployment script to turn your CI/CD pipeline into a vehicle for malware. To help mitigate this risk, from the version control system to the build and artifact systems, to deployment tools, SSM can now block unauthorized access to your CI/CD systems even if your corporate network has been compromised.
Unauthorized changes to code by authorized users: The new Code Owners system manages pull request approver sets at a per-file and per-branch level to help provide more granular identity and access management (IAM). Code Owners helps engineers who need to write, edit, and review code. It adds additional guards to files and directories in your repository at a per-file or per-branch level.
Beginning with source code changes to your CI/CD pipeline, the new code owners feature gives you granular merge guards: Check in CODEOWNERS files to your repository to specify required approvers highly granularly:
Per-path approver sets: Using flexible glob-style path specifiers, you can require that changes to matching files be approved by one or more of given sets of users.
Branch-specific governance: Manage security and deployment rules across branches without friction. You can define different owners for main or dev in the same file, eliminating the merge conflicts that occur with existing CODEOWNERS solutions. See our documentation for more details.
Nestable multi-file ownership: You aren't limited to one giant, 5,000-line root file. You can nest CODEOWNERS files in sub-directories. SSM uses a "more local wins" logic, allowing sub-teams to own their folders while the root admin maintains veto power over the entire repo.
Independent approval sections: Using the [SectionName][count] syntax (e.g., [Security Team][2]), a single pull request (PR) can require independent sign-offs from multiple departments. A PR might be reviewed by a peer, but it won't merge until two members of the security team also approve.
With your source code ready, SSM’s new Developer Connect integration makes it easy to connect your CI/CD system and runtimes securely, even when they are in different private networks.
The private CI/CD blueprint architecture follows a secure path: Secure Source Manager connects to Private Service Connect, which connects to Cloud Build. The repository, the build pools, and the artifact storage all reside in a private network, with VPC Service Controls (VPC-SC) providing defense-in-depth to limit access to proxy endpoints.
To secure your network, follow our new Private Network Integrations guide to connect SSM to Cloud Build with Developer Connect.
To secure your pull request approvals, create a root CODEOWNERS file to replace blunt IAM "Approver" roles with file-specific ownership.
Deployments now show their billable duration and CPU minutes in the dashboard, vc inspect, and the REST API. Use it to understand how each build contributes to your usage.
Billable duration is the build and post-build time combined, rounded up to the next whole minute. CPU minutes are that figure multiplied by the machine's vCPUs.
To see the same breakdown from the CLI, update to version 59.23.1 or later with npm i -g vercel@latest, then run vc inspect:
vc inspect <deployment>
Duration
build duration 5m 9s
post-build duration 3m 21s
billable duration 9m
CPU Minutes Usage 270 minutes (9m x 30vCPU)
Learn more about build machines and CPU minutes in the docs.
We're excited to announce that our modernized Dropbox API documentation is now live at https://docs.dropboxapi.com.
We've rebuilt our API reference to make integrating with Dropbox faster and more intuitive.
Test endpoints directly in the documentation. No more switching between docs and separate testing tools. You can set parameters, issue API calls, and see real responses from the Dropbox API right in your browser.
Get instant help with the embedded AI assistant. Ask questions about endpoints, troubleshoot issues, or get code examples without leaving the page.
Connect compatible AI tools directly to our documentation through MCP (Model Context Protocol) using the MCP server. This gives your AI assistant access to up-to-date Dropbox API documentation right where you work, so it can answer questions, help you explore the API, and assist with integrations using information directly from our docs.
To access the MCP server, use the "Connect to Claude Code" or "Connect to Cursor" actions in the API Reference pages, or connect to the MCP server at: https://docs.dropboxapi.com/_mcp/server
Explore the Dropbox API through a clean, responsive interface that brings endpoints, types, schemas, and related documentation together in one place. Improved navigation and search make it easier to find the right endpoint or parameter, jump between related resources, and discover the information you need without digging through multiple pages.
Access comprehensive information including detailed error responses, type definitions, and full request/response schemas.
This is a major update to how we serve our developer community, and we want to make sure it meets your needs.
Tell us what you think:
Please share your feedback in the Dropbox Developer Forum. Your insights will help us build better tools for the entire developer community.
Visit https://docs.dropboxapi.com to explore the new documentation, try out the new features, and then let us know what you think.
Thanks for building with Dropbox!
The Dropbox API Platform Team
We introduced Python Workers two years ago, providing a way to run Python applications in the Cloudflare Workers runtime. Our goal was to make it as simple to write Workers in Python as it is in TypeScript, and to make the ecosystem of Python packages and frameworks “just work”.
Today, Python Workers are now generally available (GA).
What does GA mean? It means Python is now a first-class, fully supported language on the Cloudflare Developer Platform. You can bring the Python code, libraries, and design patterns you already know and connect them seamlessly to Workers AI, R2, D1, Hyperdrive, Durable Objects, Queues, Workflows, and the rest of the Cloudflare platform. You can also run popular Python frameworks like FastAPI, Django, and Flask inside Python Workers. You can even create a Python Worker inside another Worker using Dynamic Workers.
Bringing Python to Cloudflare Workers was a natural choice. Because Workers has supported WebAssembly since 2018, it gave us the perfect environment to run a Wasm-compiled Python interpreter. By using Pyodide, we were able to quickly support a wide range of Python applications in Cloudflare Workers.
Our goal was to create the first platform for infinitely scalable Python apps, while making it as easy and performant as developing Python apps anywhere else.
The features we are highlighting today are the result of this multi-year effort. Many developers are already building applications within Python Workers; today, we are making these capabilities production-ready for everyone.
Python Workers now natively support Cloudflare Developer Platform bindings. Previously, using these Cloudflare bindings in Python Workers required converting Python objects into TypeScript objects explicitly at the RPC boundary. For example, sending a Python dictionary into a Cloudflare Queue required the following glue code to work:
This required Python developers to keep the JavaScript environment and code in mind while writing Python Workers, and it was a common source of error for both humans and AI agents. To address this, we have encapsulated the entire type conversion process within the Workers runtime and the Python SDK. This allows you to utilize all Cloudflare bindings in a Pythonic way without writing a single line of JavaScript code, making the following just work:
You can now run your favorite Python framework, such as FastAPI, Django, or Flask, to build an API server in Python Workers. We implemented a built-in connector that you can use to easily connect your web application to Python Workers.
Let’s say you have a simple FastAPI web application:
In native environments, you would use a web server such as uvicorn to run this application.
In Python Workers, you can run the same application using the workers.asgi package we provide, just by adding this snippet to your code:
Similarly, you can use workers.wsgi package to run synchronous web applications such as Django.
Python has a standard contract for how web applications should communicate with web servers, known as the Web Server Gateway Interface (WSGI), or its modern asynchronous counterpart, ASGI. This standard allows developers to build applications that are completely server-agnostic. In a traditional deployment, web servers like Uvicorn or Gunicorn are responsible for handling multiple concurrent client connections and threads to scale traffic, while web frameworks like FastAPI can focus purely on the application logic.
In Cloudflare Workers, the Workers platform itself serves as the web server. Since our global network already seamlessly handles load balancing and infinite scaling, we don't need to reinvent the wheel by running a server inside Python Workers.
Instead, our workers.asgi and workers.wsgi connectors act as a thin, optimized bridge. They translate the incoming native JavaScript request into the standard WSGI/ASGI structures that Python applications expect, and seamlessly pipe the response back out with minimal overhead. By doing this, Python developers get the best of both worlds: you can write and organize code using your favorite web frameworks, while letting the Cloudflare Workers platform instantly scale your API across the globe, without ever configuring a server.
These connectors can be used not only with FastAPI, Django, or Flask, but with any Python web framework that uses the WSGI or ASGI interface.
You can find more information about using each web framework in the Python Workers documentation.
If you are building a Python application using relational databases such as PostgreSQL or MySQL, you can now integrate Hyperdrive into Python Workers.
Previously, Python Workers didn’t support TCP sockets, making database drivers unavailable. To understand why this was a blocker, you need to look at how WebAssembly operates. Python database drivers like aiomysql or asyncpg rely on the standard library's socket module to establish connections. In a standard environment, this module makes POSIX system calls to the underlying operating system. Inside a WebAssembly sandbox, those POSIX networking syscalls are normally stubs that always fail. Any attempt to open a standard socket would immediately fail. To solve this problem, we implemented socket system calls using the Workers connect API.
When a database driver attempts to open a TCP connection, it goes through our custom socket syscall implementation. It translates standard Python socket operations like opening a connection and reading bytes into the corresponding JavaScript calls used by the Workers runtime. Because this translation happens at the system call level, your database drivers don't have to know about the underlying implementation at all.
This socket bridge is what makes our Hyperdrive integration possible. To use Hyperdrive in Python Workers, first connect your database with Hyperdrive and set up the binding in the Wrangler config:
Then, connect to Hyperdrive using the database drivers you are familiar with:
You can refer to the Hyperdrive Python Workers documentation to find out how you can use Hyperdrive in Python Workers, and which packages are currently supported.
Because Python Workers run inside a WebAssembly sandbox, any packages with native C/C++/Rust extensions must be cross-compiled to WebAssembly to run in Python Workers. However, previously, there was no standard way to cross-compile any Python packages to WebAssembly. That meant our team had to manually compile and host custom WebAssembly packages. This greatly limited the number of packages you could actually use in Python Workers.
We wanted to fix this and allow users to use a wider variety of packages. However, we didn’t want to merely build packages usable only in Python Workers, which wouldn’t benefit the community. Since Python Workers are built on top of Pyodide, we wanted the ecosystem to evolve in a way that benefits Pyodide and the entire Python-on-WebAssembly community.
To this end, we proposed PEP 783, which standardizes a platform for running Python in the browser runtimes called PyEmscripten. After over a year of discussion and refinement, this proposal was accepted, enabling package maintainers to build and publish packages for the PyEmscripten platform and make them available across all environments that implement PyEmscripten.
We also stabilized the existing Pyodide build toolchain and evolved it into a form that is accessible to all package maintainers, enabling developers to easily build packages for the PyEmscripten platform. Furthermore, we added PyEmscripten platform support to cibuildwheel, to make it easier for others to adopt support for the PyEmscripten platform.
While the ecosystem is still adopting this standard, we hope every Python package will have a wheel that works with WebAssembly in the future. We are also actively working with major package maintainers to add PyEmscripten builds. If you encounter a package that isn’t supported yet, let us know on Discord or GitHub, and our team will work to get it built.
You can also check out our EuroPython 2026 talk: “Python Everywhere: The State of Python on WebAssembly” to see how we made this possible.
The large ecosystem of data science and machine learning packages makes Python the natural choice for building intelligent agents and AI pipelines. But bringing these to Python Workers historically presented a challenge: libraries such as openai and langchain rely on HTTP clients like requests or httpx to communicate with external APIs. However, because of missing low-level socket operations support in Python Workers, these HTTP clients didn’t work properly.
To solve this, we contributed upstream to ensure these HTTP clients can route requests directly through the JavaScript fetch API in WebAssembly environments. Combined with our new support for low-level socket operations as explained in the previous section, this makes the entire networking stack work seamlessly inside Python Workers.
As a result, you can now run AI libraries like openai, langchain, and mcp natively in Python Workers. You can also combine them with Workers AI to run serverless inference on GPUs in Cloudflare’s network, or proxy requests through Cloudflare AI Gateway.
The example below shows a way to run Worker AI models in langchain, using the langchain-cloudflare package:
We have assembled a collection of production-ready patterns in our python-workers-examples repository. Here are some ways you can combine Python Workers with the Cloudflare ecosystem.
Building a full-stack AI application often means connecting multiple services such as storage, queuing, and inference. This example shows how to build an AI-driven image-to-image generator purely in Python Workers. It accepts user requests, drops them into a Cloudflare Queue, and uses Workflows to orchestrate the image generation step via Workers AI, and stores the image to an R2 bucket.
Consuming a firehose of real-time events usually requires a dedicated server to maintain the connection. In this example, we use a Python Worker to connect to the ATProto/Bluesky Jetstream WebSocket. By backing this connection with a Durable Object, the Python Worker can maintain long-lived state, ensuring that the WebSocket connection stays alive.
Build and deploy an MCP server using the official Python MCP package to give your AI assistants access to edge data.
Building a RAG system using Workers AI and Vectorize, Cloudflare’s vector database.
We’ve updated our docs across Cloudflare products to include Python example code. Nearly everywhere where there is a code example showing how to do something in TypeScript, there’s also a code example in Python. We’re committed to continuing to include Python examples across all of our products. You can toggle code snippets between JavaScript, TypeScript, and Python throughout our developer documentation.
Reaching GA is just the start. We have many plans to make Python Workers better, including making Python Workers more performant and memory efficient, as well as supporting more packages.
Keep telling us what you want to build on Python Workers, and we’ll keep pushing the bounds of what is possible. Check out Python Workers documentation and start building your first Python Worker!
During beta, each Function was reachable only at its Neon invocation URL, something like https://br-cool-forest-a1b2c3d4-api.compute.c-2.us-east-2.aws.neon.tech. Now, we support custom domains - you can put it behind api.example.com instead.
PS: There's no separate charge for adding a custom domain. Traffic through your domain is billed like any other Function traffic, and certificates are issued automatically.
You can register a domain from the Neon Console, or with the CLI:
neon functions domains register api.example.com --slug apiThe command returns a CNAME target. Add that record at your DNS provider, then check its status:
neon functions domains list --output jsonOnce the status is active, Neon routes the domain to your Function and provisions its TLS certificate through Let's Encrypt.
You can also declare the domain via neon.ts as you declare the function:
import { defineConfig } from "@neon/config/v1";
export default defineConfig({
functions: {
api: {
name: "API",
source: "./functions/api.ts",
customDomains: ["api.example.com"],
},
},
});A stable, branded hostname is what turns a Function from an internal endpoint into something you can ship to clients and other machines. For example, MCP servers.
Building MCP server was the main use case we had in mind when shipping custom domains. We’ll post more about this soon, but the Neon backend is perfect to build MCPs:
DATABASE_URL injected, so tool calls query your data in the same regionBut that endpoint has to look like yours. Marketplace listings, plugin manifests, and docs all store a URL - a hostname like br-cool-forest-a1b2c3d4-mcp.compute.c-2.us-east-2.aws.neon.tech is not something you put in a ChatGPT plugin or hand to a customer. Without a custom domain, the usual workaround is a reverse proxy on Vercel or Cloudflare in front of the Function, but then the MCP would no longer served from Neon.
Point mcp.yourcompany.com at the Function and the request goes there directly, with TLS included. Keep the frontend wherever you already host it.
Other applications you can now build that need the same kind of hostname:
api.example.comtenant-001.app.example.com at a Function and route by the incoming hostCustom domains already work through the Console, CLI, SDK, and API. Follow our custom domains guide or point your agent to it, and get started.
During the beta phase, the only way to run a Neon Function was to send it an HTTP request. That works well for jobs triggered by your app, but not so much for backend jobs. If you wanted to pull an external API into Postgres every 15 minutes, you needed an external scheduler. Also, using pg_cron meant that scale to zero needed to be disabled for that particular branch.
Now, with Function Triggers, this is much smoother. A Function Trigger is a branch-scoped definition that tells Neon when to invoke a deployed function. You deploy the function as usual; the trigger is what calls it. Today we're discussing the first trigger type we’ve shipped: schedule, a cron expression that is compatible with scale to zero.
A schedule fires your function code, not SQL, so the function can do backend operations you can't do with SQL inside Postgres. Some examples:
embedding column, generate vectors, and write them back to Postgresneon_auth schemaTriggers live on a branch and point to a function on that branch, the same way functions do:
So branching production for a test doesn't fire the parent's cron a second time, and enabling a trigger on the child can't reach back and affect production.
Postgres already has pg_cron, and Neon supports it. But pg_cron runs inside the Postgres compute: if the compute is suspended due to scale to zero, the job does not run. You would have to use it on computes that stay up 24/7 or turn scale to zero off, which is a big disadvantage. Function Triggers keep the timer outside the compute, so you can leave scale to zero on.
Pg_cron and function triggers also run different code:
pg_cron is a SQL statement or a Postgres function| pg_cron | Function Triggers | |
|---|---|---|
| Runs | A SQL statement or Postgres function | Your JavaScript or TypeScript function |
| Where | Inside the Postgres compute | On Neon's compute, next to your data |
| External APIs | No | Yes: HTTP, AI Gateway, Object Storage |
| Compute scaled to zero | Doesn't run | Runs; the invocation starts the function |
Here's a function that checks a URL and records the result. The outbound fetch is the part you can't run from SQL. The handler answers a POST, verifies that the call came from Neon's trigger system, and reads the scheduled time from data in the request body:
import { Hono } from 'hono';
import { neon } from '@neondatabase/serverless';
const app = new Hono();
const sql = neon(process.env.DATABASE_URL!);
app.post('/', async (c) => {
if (!c.req.header('x-neon-trigger-invocation-id')) {
return c.json({ error: 'not a trigger call' }, 403);
}
const { data } = await c.req.json<{ data: { scheduled_at: string } }>();
const scheduledAt = data.scheduled_at;
const started = performance.now();
const res = await fetch('https://example.com', {
signal: AbortSignal.timeout(10_000),
});
const latencyMs = Math.round(performance.now() - started);
await sql`
INSERT INTO checks (scheduled_at, status_code, latency_ms)
VALUES (${scheduledAt}, ${res.status}, ${latencyMs})
ON CONFLICT (scheduled_at) DO NOTHING
`;
return c.json({ ok: true, scheduled_at: scheduledAt, status: res.status });
});
export default appDeploy the function, then create the trigger against your branch with the Neon API:
curl -X POST "$API/projects/$PROJECT_ID/branches/$BRANCH_ID/triggers" \
-H "Authorization: Bearer $NEON_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"type": "schedule",
"function_slug": "uptime",
"name": "uptime-check",
"schedule": { "cron": "*/15 * * * *" },
"function_path": "/",
"enabled": true
}'Neon now invokes the function every 15 minutes, and each run writes a row. The X-Neon-Trigger-Invocation-Id header confirms that the call came from Neon's trigger system. ON CONFLICT ... DO NOTHING keeps a repeated occurrence from creating a duplicate row.
If you've been running an external scheduler to invoke a function over HTTP, you can hand that job to Neon. To set this up with a coding agent, start from this prompt:
Create a Neon Function that <task>, then schedule it with a Function Trigger.
Docs: https://neon.com/docs/compute/functions/triggers/schedule.md
- Add one unauthenticated POST route (scheduled invocations arrive without credentials). Read `data.scheduled_at` from the JSON body; keep the handler idempotent.
- If the task uses Postgres, connect with the injected DATABASE_URL.
- Deploy it, then create a schedule trigger via the Neon API with a five-field UTC cron. Start at `* * * * *` to confirm a run, then PATCH to the real cadence.
- The route and trigger both default to `/`; set `function_path` on both if you want a different path.Access Context Manager supports extended session length for Workforce Identity Federation. This feature is in Preview for Looker (Google Cloud core) customers. For more information, see Configure extended session length for Workforce Identity Federation.
Installed latest packages from upstream dependencies.
JupyterLab now forwards client-side logs (console errors, uncaught exceptions, unhandled promise rejections, and failed network requests) to the instance backend, where they surface in Cloud Logging for easier debugging.
Installed latest packages from upstream dependencies.
JupyterLab now forwards client-side logs (console errors, uncaught exceptions, unhandled promise rejections, and failed network requests) to the instance backend, where they surface in Cloud Logging for easier debugging.
You can now use the AlloyDB Columnar Engine as a read-optimized, in-memory cache for HNSW vector indexes. This feature is generally available (GA). It accelerates vector search performance and increases queries per second (QPS) for vector workloads.
For more information, see Accelerate queries with the Columnar Engine.
On September 21st, 2026, we began maintenance updates of Apigee instances configured for maintenance windows.
If you set a preferred window for maintenance for your instance, and your instance version is below 1-18-0-apigee-4, your instance will be updated to 1-18-0-apigee-4 within the next seven to 21 days. A notification containing the expected date of upgrade will be sent within the next two business days.
Note: Instances that meet either of the following two criteria will not be updated:For more information on participating in scheduled maintenance windows, see Maintenance overview and Manage Apigee instance maintenance windows.
On September 21st, 2026, we released an updated version of Apigee (1-18-0-apigee-5).
Note: Rollouts of this release began today and can take four or more business days to be completed across all Google Cloud zones. Your instances might not have the features and fixes available until the rollout is complete.| Bug ID | Description |
|---|---|
| 560130499 | Security fix for Apigee. Fixed a security issue in the Java Callout policy. |
| 547681234 | Security fix for Apigee. Patched CVE-2026-69247 by upgrading a third-party library used by the Apigee model-security engine. |
| 556568593 | Security fix for Apigee. Patched CVE-2026-84304 by upgrading gRPC. |
| N/A | Security fix for Apigee infrastructure. |
| Bug ID | Description |
|---|---|
| 559009293 | Fixed elevated OAuth and VerifyAPIKey latency and Cassandra read load for AppGroup apps by caching the AppGroup entity in the Message Processor runtime, matching Developer-app behavior. |
| 558888960 | Fixed distributed tracing so that the target URL is included as a span attribute in all scenarios. |
| 556750755 | Fixed EventFlow (Server-Sent Events) dropping or truncating events that follow a large (greater than 16 KB) event under load on the http-adaptor data path. |
| 553931019 | The MCP tools/list method now aggregates tools across all approved API products. |
| 531783017 | Implemented the <Enforce>true</Enforce> element of SSLInfo for a Syslog endpoint, so that the syslog target's TLS server identity is verified. |
| 554114419 | Policies can now change request pseudo-headers (for example, :path and :authority) when HTTP/2 is in use. |
| 548763108 | Blocked outbound HTTP from the Message Processor to Kubernetes-internal targets. |
| 513032450 | Restored a 15-second TCP keep-alive on the Apigee Connect control-plane connection so that a silently dropped connection recovers in seconds rather than approximately two hours. |
| N/A | Updates to infrastructure and libraries. |
The Query results pane in the BigQuery Studio query editor lets you view a short history of recent runs for a query, including multi-statement queries, without having to navigate to the Job history tab. This feature is generally available (GA).
BigQuery generative AI functions
now support the gemini-3.8-flash Gemini model.
The C4 machine series is available for Cloud SQL for MySQL Enterprise Plus instances in the following regions:
asia-east2 — Hong Kongasia-southeast2 — Jakarta, Indonesiaeurope-west3 — Frankfurt, Germanyeurope-west4 — Eemshaven, Netherlandseurope-west8 — Milan, Italyus-east5 — Columbus, Ohio, USAus-south1 — Dallas, Texas, USAus-west4 — Las Vegas, Nevada, USAThe C4 machine series provides the following benefits:
The C4 machine series is available for Cloud SQL for PostgreSQL Enterprise Plus instances in the following regions:
asia-east2 — Hong Kongasia-southeast2 — Jakarta, Indonesiaeurope-west3 — Frankfurt, Germanyeurope-west4 — Eemshaven, Netherlandseurope-west8 — Milan, Italyus-east5 — Columbus, Ohio, USAus-south1 — Dallas, Texas, USAus-west4 — Las Vegas, Nevada, USAThe C4 machine series provides the following benefits:
The C4 machine series
is available for Cloud SQL for SQL Server Enterprise Plus instances in the
following regions:
asia-east2 — Hong Kongasia-southeast2 — Jakarta, Indonesiaeurope-west3 — Frankfurt, Germanyeurope-west4 — Eemshaven, Netherlandseurope-west8 — Milan, Italyus-east5 — Columbus, Ohio, USAus-south1 — Dallas, Texas, USAus-west4 — Las Vegas, Nevada, USAThe C4 machine series provides the following benefits:
The Dataform remote Model Context Protocol (MCP) server now supports pipeline authoring in development workspaces and Git repository operations. AI agents can create and list workspaces, search and edit files, commit changes and push commits to remote Git providers, update repository settings, and organize repositories in folders. For more information, see Use the Dataform remote MCP server and the Dataform MCP reference. This feature is generally available (GA).
Gemini layout parser model
pretrained-layout-parser-v3.1-lite-2026-08-11is in Preview.
Gemini Enterprise: Transfer ownership of shared agents
Administrators can transfer ownership of shared employee-made agents to another user or to themselves in the Google Cloud console. This is useful when reassigning agents created by departing employees or when temporary workers hand over agents to full-time staff.
Key characteristics and requirements include:
roles/discoveryengine.agentspaceAdmin or roles/discoveryengine.admin) can
transfer agent ownership. Agent owners cannot transfer ownership unless they
are also administrators.agentUser role.For more information, see Transfer agent ownership.
CodeMender updates (v0.9.0)
This release introduces updates to CodeMender:
cm report --format html to provide a modern, interactive dashboard featuring severity metric cards, syntax-highlighted code snippets with line numbers, and an inline patch diff viewer. Added the --open (-o) flag to automatically open the generated report in the default browser..cs), Rust (.rs), Kotlin (.kt, .kts), Ruby (.rb), and PHP (.php) to the default discovery configuration and initialization templates.cm stats and session exports to report per-turn latency breakdowns, distinguishing time spent waiting on model inference from local tool execution.For more information, see the CodeMender documentation.
Google Cloud CCaaS prerelease notes 6.15
Here are the pre-release notes for what we expect to be the next version of Google Cloud CCaaS. When we release this version, we expect the new capabilities to be as shown here.
Important: The next version of Google Cloud CCaaS could be greater than 6.15.Remove a user from all teams at once
Using the new Remove from all teams button, you can remove a user from all of the teams that they belong to.
Administrators: There's a new Remove from all teams button in the Teams section of the Edit User dialog.
This release addresses the following issues:
Fixed an issue that led to increased startup latency and errors for mobile and web chat sessions.
Fixed an issue where agents were incorrectly demoted to an Unresponsive status and removed from the routing pool despite successfully receiving call offers.
Fixed an issue where dialed numbers on Twilio BYOC SIP inbound calls were incorrectly formatted with extra digits from the SIP host and port.
Fixed an issue that prevented chat transcripts from being generated and delivered for sessions containing structured message content.
Fixed an issue where the call adapter incorrectly showed a call as on hold after a carrier failed to process the hold request, leaving the audio channel open between the agent and the customer.
Fixed an issue where a failed media download caused the service to restart unexpectedly.
Fixed an issue that caused queue-specific wrap-up and disposition settings to reset to global defaults after changing unrelated fields on the Queue Settings page.
Fixed an issue where machine translation didn't activate for chats that were transferred into a non-English language queue if the session originated with a virtual agent.
Fixed an issue where generative knowledge assist answers that contain long URLs were cut off at the edge of the panel.
Fixed an issue where queued calls were neither routed to available agents nor offered a callback.
Fixed an issue where voicemails were automatically dismissed and marked as read if a playback error occurred.
Fixed an issue where agent call recordings were missing or attached to the wrong call record after a virtual agent deflection.
Fixed an issue where unanswered DCR calls that were routed using Nexmo disconnected the caller instead of requeuing the call.
Fixed an issue that prevented virtual agents from transferring calls to a human-agent queue.
Fixed an issue where calls lacking a carrier hangup reason were incorrectly categorized as "customer abandoned", even when the call center didn't answer the call.
Fixed an issue where the call event API payload for DCR calls contained incorrect virtual agent parameters.
Fixed an issue where custom data from chat interactions wasn't recorded in Salesforce records.
Fixed an issue where Mexico time zones were incorrectly applying daylight saving time adjustments.
Fixed an issue where agents and end-users were joined to separate conferences, preventing audio communication between them.
Fixed an issue where call recording deletion tasks entered an endless loop if the provider didn't return a successful response.
Fixed an issue where IVR voice calls didn't send custom wrap-up events to Dialogflow CX under certain configurations.
Fixed an issue where a trailing slash in the host URL caused the web SDK to unexpectedly re-enable features that had been previously disabled for specific deployments.
You can use System for Cross-domain Identity Management (SCIM) data as the source for both user and group claims in the OAuth sign-in workflows for Looker. You can also use Extended Session Length (ESL) when using SCIM.
This feature is in Preview.
For more information, see the following:
Storage Transfer Service now supports filtering Amazon S3 source objects by storage
class. You can specify a list of storage classes to include when creating or
updating transfer jobs using the Google Cloud console, the gcloud CLI, or the
REST API.
For more information, see Filter source objects by storage class.
Storage Transfer Service now supports filtering source objects using glob patterns
with wildcard characters such as * and ?. Glob filtering is supported for
transfers from Amazon S3 and Microsoft Azure Blob Storage when configuring transfer jobs using
the gcloud CLI or the REST API.
For more information, see Filter source objects using wildcards.
How much better could a coding agent perform if it used the best model for each task?
The best single model, GPT-6 Astra, gets 74.1% of DeepSWE tasks at $6.52 each. Pick the right model for each task and the same eighteen models get 97.6% at $1.88. 23 points better, at under a third of the cost.
That number comes from hindsight. We ran all eighteen models on every task first and picked the winner for each one. What it measures is the capability already sitting in the pool, but it's split across models that nobody uses together.
Putting them together is a router's job. It picks which model handles each task before the work starts, and before is the hard part. Looking back, it's easy to point at a task and name the model that would have done it better. A router has to choose before it sees the outcome, and a wrong choice costs far more than the few dollars it saved.
We analyzed DeepSWE v1.1, an agentic coding benchmark where the unit of work is an engineering task: the agent has to understand an issue, inspect a repository, use tools, edit code, execute it, and get the task to pass.
The policy is deliberately simple. Pick one model at the start of a task and keep it for the whole run, with no switching mid-session.
Then we name the winner for each task by measured pass rate, breaking ties on cost. That's the oracle router. the same method we used in our Kimi K3 and Fable analysis.
The oracle scores on the same 113 tasks it picks from, using four rollouts per model-task pair, and taking a maximum over 18 noisy estimates biases it upward.
The best models score around 70% and spend $6.46 to $13.41 a task getting there:
That's the best a fixed-model policy does. Now pick per task:
The oracle router across all eighteen models reaches 97.6% at $1.88 a task. That is 23 points above GPT-6 Astra, at under a third of its cost. Restrict it to open-weight models only (DeepSeek V4 Flash and Pro, GLM-5.3 and GLM-5.3 Flash, Kimi K3, Qwen3.8 Max), and it still reaches 90.3% at $1.45 a task, which beats every closed model here by 16 points while spending under a quarter of what Astra does.
These results make "open versus closed" a less interesting debate. The emerging race is to move from the theoretical oracle router to building a system of models with collectively better intelligence than any single model. A system of open models can in principle already far surpass the closed frontier.
There is substantially more capability in the pool than any individual model exposes.
In our prior work, we found different models are sufficient (even exceptional) on different tasks. The DeepSWE analysis makes the cost implications concrete. At that 97.6% point, the oracle still sends 94 of the 113 tasks to a model costing under $3.
The three most expensive models in the field, all above $11.50 a task, are the sole best choice on only three tasks.
On 79 of the 113 tasks, at least one of those expensive models ties the top score and loses the task on price alone. A strong general-purpose model can be excellent across a broad distribution without being uniquely necessary on most individual tasks.
A fixed-model policy pays for broad capability on every task. A system can ask a narrower question:
What capability does this task actually require?
How many models does it take to capture the effect?
The best pair adds 13.1 points over the best single model, and the best trio reaches 91.2%. Expanding from three models to all eighteen adds another 6.4 percentage points. The useful object is not a catalog of hundreds of nearly interchangeable models. It is a portfolio with complementary coverage.
The value is capability coverage, not model count.
LLMRouterBench evaluates routing across 33 models and more than 400,000 instances. It finds that a handful of models covers most of what the full set can do, and that bigger pools add little without careful curation.
An oracle is easy to love because it never gets to be wrong. A production router does. We measured it strictly: we use pass@1, the probability that a single attempt passes, rather than a "did this model ever succeed across four attempts" rule. That second rule would make the ceiling look far more impressive while meaning much less.
The gap is a product problem and the literature is blunt about it. LLMRouterBench finds that several recent routing approaches, including commercial ones, fail to reliably beat simple baselines, and traces much of that to model recall: even when a model with the right capability exists in the pool, the router has to recognize when to reach for it.
So sticking with one model you know isn't conservative, it's rational: a stable error distribution beats a router that unpredictably picks the wrong specialist. The bar for a routing system is to make model specialization predictable enough that changing models improves the system without making its behavior less trustworthy.
Routing is usually introduced as a cost optimization: send easy work to a more cost-optimized model, reserve the expensive one for hard work, and keep the difference. At Fireworks, we take a broader view.
If different models are genuinely complementary, then selecting among them moves you up the capability curve, not merely left along the cost curve.
That's what FireRouter is built for. It routes at the task level across both open and closed models, and it's cache aware, so switching models doesn't silently throw away the context you already paid for.
Over four weeks of our own production coding traffic, sessions routed through FireRouter cost $7.42 against $15.81 for Opus 5 alone, a 53% reduction across 2,334 sessions.
The useful unit of AI work is already larger than the single model call. A coding agent is a model inside a harness that supplies context, tools, execution, tests, state, and feedback.
Once several models have complementary strengths, the selection policy becomes a component of the system, alongside context, tools, and tests. Choosing and composing those components is the job. That's what AI engineering is.
Our experiment measures only the simplest version of that system: pick one model at the start of a task and leave it there. The selection policy is the part we can actually build.
We serve every frontier open model in production, which is where a real understanding of each model's strengths comes from. You do not learn what a model is uniquely good at from benchmark averages. You learn it by running all of them, on real work, at scale. That is where FireRouter's model choices come from, and that bar is the one we intend to clear. We will go into our own router and how to hill-climb on your own specialized intelligence in future posts.
Define your frontier on FireRouter.
On costs. All cost figures in this analysis come from the DeepSWE leaderboard's published per-model numbers. The raw cost_usd in the public trials file does not match what the board displays, and for the DeepSeek family it differs by several times over, so each model’s per-task costs are scaled so its mean matches the published figure. Accuracy comes from the four raw rollouts of each task-model pair, cost from the board.
Source: DeepSWE v1.1 trials, refreshed 17 September 2026. 113 tasks, 18 models each at its best available configuration, 2,034 model-task cells.
MiMo V2.6 Pro, MiMo V2.6 Flash, and MiMo V2.6 Pro UltraSpeed from Xiaomi are now available on AI Gateway.
MiMo V2.6 combines coding, reasoning, and tool use with native text, image, audio, and video understanding. Its 1M token context supports long repositories, tool traces, and multi-session agent work, with structured outputs and up to 128K output tokens.
Choose a model based on the workload:
xiaomi/mimo-v2.6-pro is the larger sparse mixture-of-experts checkpoint, with 1.02T total parameters and 42B activated per token, for complex software engineering and long-running agent work.
xiaomi/mimo-v2.6-flash uses 309B total parameters and activates 15B per token, making it the more efficient option for multimodal automation and everyday agent workflows.
xiaomi/mimo-v2.6-pro-ultraspeed serves Pro at up to 20 times its output speed for interactive and latency-sensitive workflows, with the same capabilities.
To use MiMo V2.6 in a coding agent, install the latest Vercel CLI and run:
Then select xiaomi/mimo-v2.6-pro, xiaomi/mimo-v2.6-flash, or xiaomi/mimo-v2.6-pro-ultraspeed in your agent.
Try MiMo V2.6 Pro, Flash, or Pro UltraSpeed in the model playground.
AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, budgets for API keys, routing rules, and more.
You can now call Jev from TypeSafe AI through AI Gateway using an existing TypeSafe client or the HTTP API, in addition to the AI SDK.
TypeSafe client: Point an existing TypeSafe client at AI Gateway without changing its evaluation calls.
HTTP API: Call Jev directly from any language or framework.
AI SDK: Call Jev from a TypeScript application using AI SDK.
Jev is a probabilistic decision model for software. State goes in, and typed answers come out with probabilities attached, so there's no generated text to parse. Requests are billed through AI Gateway on all three paths, so they appear alongside your other model calls in usage and observability.
Change the base URL and API key. Your systemOne calls, noul questions, and response shapes stay exactly as they are.
New integrations name the model as typesafe-ai/jev and ask one of three question types: boolean returns a probability from 0 to 1, choice picks one option from a set you name, and score rates against a scale you define. This example asks whether an agent should keep working after fixing a bug and passing its tests.
With the HTTP API, POST to /v1/evaluate:
With the AI SDK, run the same evaluation through evaluate:
You can also use Jev through eve, a framework for building and deploying agents with sandboxed compute, human approvals, and evaluations already built in. eve uses Jev as the default evaluation model for automatic model selection, typed evaluations, and automated tool approvals.
See the TypeSafe-compatible API guide to migrate a client, or the evaluation guide for the HTTP API or AI SDK.
Grok 4.7 from SpaceXAI is now available on AI Gateway and 40% off through September 27. The discount applies automatically when you call spacexai/grok-4.7.
Grok 4.7 has a 500K token context window and supports low, medium, high, and xhigh reasoning levels, giving you control over the tradeoff between latency and depth.
Use spacexai/grok-4.7 everywhere you call the model. The same ID works with the AI SDK, the OpenAI-compatible Chat Completions API, and coding agents connected to AI Gateway:
To use Grok 4.7 in a coding agent, install the latest Vercel CLI and run setup:
Then select spacexai/grok-4.7 in fx, Cursor, Codex, Amp, OpenCode, or another supported agent. See the coding agents guide for agent-specific instructions.
To create a new eve agent with Grok 4.7 and xhigh reasoning, pass the same model ID to the initializer:
Grok 4.7 supports Zero Data Retention and disallow prompt training. Requests appear in AI Gateway logs, custom reporting, and budgets alongside your other models.
AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, budgets for API keys, routing rules, and more.
AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests.
Try Grok 4.7 in the model playground, or view all language models or check out more promotions on AI Gateway.
September 21, 2026
What's left for us to build?
Five years into the AI-assisted coding experiment, I rarely find myself saying "wow" anymore. We, as an industry, have honed the prompt-to-code pipeline to just about the finest point imaginable. That's not to say we haven't made a huge leap – I recently used a first generation Retrieval-Augmented Generation (RAG) chat coding assistant for the first time in years, and it felt like I was trying to code by writing in the dirt with a rock.
The progress has been so fast, and so massive, I've come to expect the world. However, when a new model drops these days, I can rarely detect a difference in the code. The harness wars just don't feel that exciting anymore, and the fact that they're all competing on new battlegrounds (cloud infrastructure, multi-agent orchestration, extensibility) makes it clear that we've pretty well nailed the prompt to PR (or issue to PR, or plan file to PR, pick your favorite jumping off point) problem.
That doesn't mean the developers of the world can pack up and start their own farms. I still find myself groaning in pain whenever I need to update something non-trivial in our massive Sourcegraph monorepo.
I talk to engineering leaders at large, enterprise companies every week that tell me the same story. I don't know if it's just a context problem anymore; if it's context availability, or context window exhaustion, or low quality retrieval and wasted effort, or a simple mismatch between the coding agent paradigm and the sheer scale of these codebases. Maintaining existing, "brownfield" code remains completely unsolved.
What's more, as the quality of code generated by new models has begun to plateau (at pretty damn good code), I can confidently say that a new model drop isn't going to solve this problem.
It's part context, part infrastructure, part interaction model. It requires a paradigm that looks absolutely nothing like "prompt to PR."
The simplest version of an autonomous agent is a cron job.
"Every Monday morning at 8am analyze our logs and o11y stack for anomalies and let me know what you find."
"Every evening send me a recap of progress against our Q3 roadmap in Linear."
As groundbreaking as a tool built in 1975 can be, these sorts of autonomous workflows have changed the way I work more than any coding agent harness has in the last couple of years. You can still vaguely see that same "prompt to PR" shape in these agents, but the jump they take from human initiated to self-driven clearly sets them apart.
At a high level, I don't want to be an engineering manager. I don't want to have to tell an agent what to do every single time a change is needed. The promised land is a self-maintaining codebase.
The simplest primitives for the system I'm picturing are:
This system would be autonomous, composable, and fully agentic. Yet, it is still more deterministic than what many thought leaders are proposing; it's a simple, directed graph workflow, with purpose-built agents deployed to solve enterprise codebase problems. The system could be recursive, or even self-modifying, but that's not required. The agent harnesses you choose determine how much rope you give it.
I should be clear that this is not a new concept. Every enterprise I talk to is thinking about agentic Software Development Life Cycle (SDLC) automation. Agent-to-Agent (A2A) was defined partly to enable this sort of workflow. Billions of GitHub Actions run per year, a large portion of which likely have a large language model (LLM) step in them! Yet, massive, unsolved problems like identity, authorization, and budget controls remain outstanding.
My belief is that many of these issues are our own creations, and are solvable at the harness level. We've spent four years generalizing harnesses in pursuit of prompt-to-PR perfection: an agent that can take any human instruction and execute against it!
In the coming years, inside of enterprises, we will move in the opposite direction, and see more narrowly scoped and narrowly authorized agents composed into trigger/function workflows that automate codebase maintenance work safely.
That is the promise of the autonomous codebase.
The latest trend in large enterprise agent rollouts is "enterprise knowledge bases." Let me tell you, it's a great time to be a context shovel seller.
However, I want to be clear that this is a very, very positive development in the cycle. Thousands of enterprise dev teams have moved mountains and spent millions of dollars in token contracts to roll out coding agents to every corner of their engineering orgs, in many cases rewarding and even mandating tokenmaxxing.
The result is a tidal wave of absolutely terrible code that then needs to be reviewed, tested, fixed, instrumented, and ultimately trashed or deployed. Agents can do all of that, too (the Anthropic and Cursor sales reps say)!
What they can't do is tell you, before the merge, that the service or library you changed is used by another part of the organization in a different repo, on a different code host. Or that the blast radius of your agent's work was completely underestimated.
I can't blame those sales reps though. Their products are revolutionary, and can turn any prompt into a PR. In the real world, they're being asked to guess what number you have behind your back. Context, as they say (or in this case, retrieval), remains absolutely essential for agents to do good work.
The autonomous codebase system I describe above is beautiful in its simplicity, but deployed against a two-thousand-repo codebase, it simply won't be capable of doing much of anything right. How can an agent investigate a CVE if it literally can't clone and grep every single repo before its sandbox times out, before it goes into context window exhaustion psychosis, or before the LLM just decides "I've done enough, this should be good?"
Everything worth doing in an enterprise codebase starts with universal code visibility and code understanding. Some things never change: context is king.
With Sourcegraph, the code understanding platform for enterprise.
Starting today, Upstash Redis supports the Array data type introduced in Redis 8.8. Array was designed and built by Salvatore Sanfilippo (antirez), the creator of Redis, and all of its commands are available on Upstash now, with support in the TypeScript and Python SDKs.
The first reaction from most Redis users is "Isn't a list already an array?" It is not, and the gap between the two is the reason he decided to build it. This post covers why the array was added, how it differs from a list, what you can build with it that was hard before, and when to use which.
Redis has had a blind spot: no data type where the numeric index is part of the data model.
A list looks like an array from the outside. You push items, you read them back in order, and LINDEX even lets you ask for item 47. But under the hood a list is a double-ended queue. It is built for adding and removing at the head or the tail. Those operations are O(1). Everything else is a walk. Ask for item 47 and Redis walks 47 steps from the nearest end. Ask for item 50,000 in a list of 100,000 and it walks 50,000 steps, every time.
A list also has no idea of a gap. Every position from 0 to the end holds a value. There is no way to say "slot 47 is intentionally empty." And deleting an item renumbers everything after it.
That is fine when insertion order is the meaning. It breaks down when the number itself is the meaning:
Each of these can be forced into an existing type, and each workaround costs something:
The array closes this gap with one contract: if you know the index, you get the value, and everything in between costs nothing.
| List | Array | |
|---|---|---|
| What the index means | Position in insertion order | An address in your domain |
| Read by index | O(N) walk from nearest end | Constant-time lookup |
| Gaps | Impossible, always dense | Free, sparse by design |
| Delete in the middle | Shifts everything after it | Leaves the slot empty, nothing moves |
| Bounded window | RPUSH + LTRIM, two commands | ARRING, one atomic command |
| Search and aggregate | Fetch the range, do it in your app | ARGREP and AROP run on the server |
| Memory per element | Most compact | Slightly more |
The details behind each row:
Direct access. ARGET myarray 47 is a lookup, not a walk. It costs the same at index 47 and at index 47,000,000. For random reads and writes, this makes arrays much faster than lists.
Sparse by design. You can write to index 1,000,000 on an empty key and Redis allocates space for one value, not a million. The index space is split into slices of 4,096 slots, and a slice only exists once something is written into it. An untouched slice costs eight bytes. Gaps are free, so a product ID, a sequence number, or a timestamp bucket can be the index directly.
Stable positions. Deleting index 5 leaves index 5 empty. Nothing shifts. In a list, removing an item renumbers everything after it, which destroys the meaning you were relying on.
A real ring buffer. The classic idiom for "keep the last 200 events" is RPUSH followed by LTRIM. It works, but it is two commands, and between them the list is briefly too long. ARRING does the append and the wrap in one atomic command, at roughly twice the throughput of the list idiom.
Compute on the server. AROP sums, takes the min or max, counts, or applies bitwise ops over an index range. ARGREP searches values with exact match, substring, glob, or regex. Both skip empty regions entirely, so the cost tracks the number of stored elements, not the size of the index space.
This is the part that matters. Each of these was possible before, but only with a scan, a secondary index, or client-side filtering. With an array, each one is a single command.
Load a file into an array, one line per index. A code review tool, a log viewer, or a diff engine can then jump to line 4,821 directly and fetch lines 40 to 55 in one call.
ARSET doc:readme 0 "# Project"
ARSET doc:readme 1 ""
ARSET doc:readme 2 "## Install"
ARGETRANGE doc:readme 0 2
This is also a natural store for AI agent context. An agent can pull a specific section of a Markdown knowledge base by line range instead of retrieving the whole document, and use ARGREP to find the lines that mention a term.
Think ports on a switch, seats in a venue, or parking bays. Most slots are empty, and the empty ones carry information.
ARSET switch:tor-01 47 "10GbE trunk VLAN 200"
ARSET switch:tor-01 48 "10GbE trunk VLAN 200"
ARSET switch:tor-01 96 "1GbE access VLAN 100"
ARGETRANGE switch:tor-01 45 48 # nils for the dark ports
ARSCAN switch:tor-01 24 48 # only the active ports
ARCOUNT switch:tor-01 # 3, in O(1)
Empty slots cost nothing to store and nothing to skip. A hash cannot answer "which ports between 24 and 48 are active" without fetching everything.
Step 0 is "received", step 3 is "under review", step 5 is "approved". Steps 1, 2, and 4 never fired. The gap is the signal that this case was handled differently. With a list you would need sentinel values and application logic to interpret them. With an array, ARSCAN over the step range shows exactly which steps ran.
You have many machines, users, or sensors. For each one you want to keep only the most recent events, say the last 200. Older events should drop off on their own so memory never grows.
With a list, this takes two commands per event: push the new one, then trim the list back to 200. Between those two commands the list is briefly too long, and fetching a specific event by number means walking the list.
With an array, it is one command:
ARRING machine:42:events 200 "cpu 14 online"
ARLASTITEMS machine:42:events 50
Think of a circle with 200 seats. Each new event takes the next seat. When all seats are full, the next event overwrites the oldest one. The size never changes, so the memory cost per machine is fixed and predictable.
You still get direct access. ARLASTITEMS returns the newest 50, and ARGET machine:42:events 47 returns event 47 without walking.
Store log entries at their sequence number, but only the ones that passed a severity filter. Then find every error without pulling the range to your application.
ARSET log 0 "ok:200"
ARSET log 2 "err:timeout"
ARSET log 3 "ok:204"
ARSET log 5 "err:404"
ARGREP log - + GLOB "err:*" WITHVALUES
ARGREP supports exact match, substring, glob, and regex, with AND and OR to combine predicates. Only matching entries cross the wire, and there is no secondary index to keep in sync.
Index by minute, hour, or day bucket. Then ask for the total, the peak, or the number of active buckets in a window.
ARSET metrics:device-01 0 142
ARSET metrics:device-01 1 98
ARSET metrics:device-01 5 201
AROP metrics:device-01 0 8300 SUM # 441
AROP metrics:device-01 0 8300 MAX # 201
AROP metrics:device-01 0 8300 USED # 3
No running counter in a second key, no consistency problem between the two.
Profilers index frames by depth. Import jobs index rows by line number. Version histories index revisions by number. If your data already has a number attached to each item, the array lets that number be the key without any translation layer.
Ask one question: does the index carry meaning in your domain?
ARRING instead of RPUSH + LTRIM when you need both a recency view and access by position, or a fixed memory budget enforced by the data structure.The short version: if you find yourself explaining what index 47 means, you want an array. If the index is an internal detail your app never reasons about, the existing types are still the right tools.
Array commands are available on Upstash Redis today, with support in the TypeScript and Python SDKs. Start with the Array commands overview in our docs.
For more depth, see Redis's deep dive into the array type and the design write-up from its author.
The Rust Security Response Team was notified that Miri stores all environment variables to target/, allowing secrets to persist in caches.
While not necessary a vulnerability in and of itself, when paired with GitHub Actions caching behavior, it is possible for this to expose secrets to PRs.
GitHub Actions makes it possible to cache directories between runs. Typical setups allow CI runs on main (and other branches) to write to cache, and PRs can only read from cache (preventing cache poisoning). Rust projects tend to speed up CI by caching binaries built by cargo install and sometimes the contents of target/.
PR CI can be triggered by anyone who can open PRs on your repository. GitHub requires maintainer approval for the first PR, but future PRs will rerun CI on every push. Anyone who has previously landed a change can trigger a CI run extracting information from cached target/ and then cover their tracks by pushing a second commit to the PR.
GitHub sometimes hides overwritten commits in its UI, making this kind of attack harder to detect. CI run logs and overwritten commits are also deleted after a few months.
When cargo miri is invoked, Miri needs to retain build-relevant environment variables between runs1. The current code to do so achieves this by storing all environment variables to target/. This, of course, persists when target/ is cached.
If your environment contained secrets, these can now be accessed by PRs via the cache.
Our short term fix for this is to make Miri only preserve CARGO_* environment variables (excepting CARGO_*_TOKEN) and OUT_DIR. In the longer term, Miri and cargo may figure out better ways to inform Miri of the relevant list of environment variables. Note that this patch may not be available on nightly yet.
We also performed an ecosystem scan of GitHub repositories and identified 1 repository with this issue and 7 repositories that do not appear to be vulnerable but should be cautious anyway. We have reached out to those maintainers.
It is likely that our scan was imperfect, so we recommend you check your own GitHub Actions setups if you run Miri.
You are vulnerable if:
cargo miri in CIcargo miri has access to secrets as an environment variable:
env for the workflowtarget directory, usually done via actions/cache or swatinem/rust-cachePossible quick fixes include:
Once done, please clear the cache. Consider rotating any secrets that might have leaked.
The Miri release in the upcoming nightly (2026-09-22) will no longer have this problem.
Even if you do not run Miri, ensure jobs that can write to public caches do not have access to secrets. Many tools do not have special handling for secrets, and assume the entire environment can be written to the filesystem.
We consider it bad practice to have a cache that can easily be tainted by secrets.
If caching target/, it is worth making sure that the inputs to processes that create target/ (anything invoking cargo) do not have secrets available. It is generally rare for standard cargo build/test subcommands to need any secrets or tokens2, so this is mostly a matter of being careful about having secrets exposed as environment variables to the entire job.
Cargo/Miri/Rust does not guarantee that environment variables will be safe from being copied into target/. While we are treating this as a security issue and patching it out of an abundance of caution, this is not something you should rely on in general. Beyond official Rust tooling, it is possible for build scripts to be doing things that lead to the environment being stored in compilation artifacts.
Thanks to Predrag Gruevski of OpenAI for reporting this issue to us. Furthermore, the ecosystem scan was performed using Codex access and credits donated by OpenAI, which we also thank them for.
Issue triage and remediation was performed by Manish Goregaokar, Ralf Jung, Ben Kimock, Weihang Lo, Jacob Finkelman, Walter Pearce, Josh Stone, and Mark Rousskov.
Miri is invoked multiple times by cargo miri for complicated reasons ↩
In theory it could come up with build scripts reading from the network ↩
mcp-handler now has experimental support for WebMCP, the proposed web standard for exposing tools to in-browser agents. Add a single script tag to your site, and your existing MCP tools become available there too.
Opt tools in by adding them to the experimental_webMcp object:
Then load the script from your MCP endpoint with the ?webmcp-script parameter:
The script registers those tools with the page and proxies each call back to your MCP server as the signed-in user, so authenticated tools work without a browser-side OAuth flow.
Upgrade to mcp-handler@2.2.0 and read the documentation to get started.
v0 now installs private packages from npm and custom registries using credentials stored as shared environment variables on Vercel.
This makes it easier for teams to build with their existing design systems, component libraries, and internal packages directly in v0.
To get started, add one of the following as a shared environment variable on Vercel, scoped to Development and/or Preview:
Use NPM_TOKEN for private packages hosted on registry.npmjs.org.
Use NPM_RC to configure custom or multiple registries.
NPM_RC supports scoped registries and references to other environment variables. For example, configure an organization scope such as @acme for GitHub Packages, or direct package requests through a private JFrog Artifactory registry.
Credentials can be marked sensitive, and v0 never exposes them to the model or writes them to the sandbox filesystem.
You can view the integration status from Settings → Integrations in v0.
Learn more in the private dependencies docs.
Open-weight models are changing the economics of building and deploying AI at scale. Rapid gains in intelligence and efficiency mean companies can match each workload with the right balance of capability, speed, and cost. AWS is building for a future in which organizations can adopt open-weight innovation with the reliability and security required for production.
Today, Kimi K3 from Moonshot AI is available on Amazon Bedrock, giving you a powerful new option for coding and knowledge work. According to Moonshot AI, Kimi K3 is its most capable model and the first open model to reach 2.8 trillion parameters. It combines native vision capabilities with a 1-million-token context window and delivers an approximate 2.5x improvement in scaling efficiency over Kimi K2. These advances make Kimi K3 well suited to long-running coding and knowledge workflows that require sustained context across large repositories, documents, and images. Kimi K3 is the first open-weight model on Amazon Bedrock to support explicit prompt caching, helping you reduce latency and input costs when reusing context across model calls.
The launch of Kimi K3 reflects the sustained investment by AWS in open-weight models on Amazon Bedrock. Since 2025, Bedrock has added dozens of open-weight models from providers including DeepSeek, Google, MiniMax, Mistral AI, Moonshot AI, NVIDIA, OpenAI, and Qwen. Supporting this expanding selection is continued advancement of the inference technology that serves these models at scale. In 2026, Bedrock added support for tool calling, structured output, reasoning, response streaming, and the Responses and Chat Completions APIs. Because these are platform capabilities rather than per-model integrations, new open-weight models can benefit from them as they become available on Amazon Bedrock.
As with all open-weight models on Amazon Bedrock, you can adopt Kimi K3 without changing your security posture. Your data is processed within the AWS data boundary, is not shared with the model provider, and is not used to train the underlying model. Zero data retention is always enabled for inference requests, while zero operator access prevents even AWS operators from accessing your prompts and completions during inference. Together, these protections let you use open-weight models with confidence while maintaining control of your data.
To try Kimi K3, open the Amazon Bedrock console, go to Test > Playground, and select Kimi K3 as the model. From there, you can test your first prompt.
Programmatically, you can call the model using the bedrock-runtime endpoint, which supports the OpenAI-compatible Responses and Chat Completions APIs, and the Amazon Bedrock Invoke and Converse API APIs.
You can invoke Kimi K3 through a cross-Region inference profile. For workloads without regional restrictions we recommend using the global profile, global.moonshotai.kimi-k3, which routes each request to any supported commercial AWS Region worldwide. Global cross-Region inference costs approximately 10% less than a geographic profile. The US geographic profile, us.moonshotai.kimi-k3, keeps processing within the US geography for data residency requirements.
bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream, and bedrock:CallWithBearerToken.Here is a quick example that uses the OpenAI SDK and the aws-bedrock-token-generator library for Python to generate short-term bearer tokens for authentication to Amazon Bedrock.
from aws_bedrock_token_generator import provide_token
from openai import OpenAI
region = "us-west-2"
oai_client = OpenAI(
api_key=provide_token(region=region),
base_url=f"https://bedrock-runtime.{region}.amazonaws.com/openai/v1",
)
resp = oai_client.responses.create(
input="What is Byte-Pair Encoding, in AI?",
model="global.moonshotai.kimi-k3",
)
print(resp.output_text)Long-running coding and knowledge workflows often resend stable context, such as repository instructions, tool definitions, or reference documents. With explicit prompt caching, you identify reusable prompt prefixes so later requests can use cached content. When a request matches a cached prefix, Amazon Bedrock can reduce response latency and input token costs.
Caching for Kimi K3 on Amazon Bedrock:
prompt_cache_breakpoint to a supported input content.With the OpenAI Python API, explicit caching can be configured as shown in the following example:
resp = oai_client.responses.create(
model="global.moonshotai.kimi-k3",
# Enable explicit caching mode:
extra_body={"prompt_cache_options": {"mode": "explicit"}},
input=[
{
"type": "message",
"role": "system",
"content": [
{
"type": "input_text",
"text": SYSTEM_PROMPT,
# A long, static system prompt is a great target for caching:
"prompt_cache_breakpoint": {"mode": "explicit"},
},
]
},
{
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": USER_INPUT,
# Multiple breakpoints can also be defined, for layered cache:
"prompt_cache_breakpoint": {"mode": "explicit"},
},
],
},
],
)
if resp.usage.input_tokens_details.cached_tokens:
print("Hit cache!")You can explore the Moonshot AI on AWS samples repository for more examples.
In addition to using the APIs directly, you can use Kimi K3 through the wide range of coding assistants, personal agents, and agentic frameworks that support Amazon Bedrock specifically, or OpenAI-compatible model providers in general.
There are several popular coding agents available to builders today, so consider OpenCode as an example. OpenCode is open source, model agnostic, and has a native amazon-bedrock model provider, which uses the Converse API.
To get started, you can configure the amazon-bedrock provider either in your user-level or project-level opencode.json configuration files as shown in the OpenCode documentation. With the provider configured, OpenCode will automatically detect available Amazon Bedrock models which you can select from using the /models command. For example, a minimal ~/.config/opencode.json file could look like:
{
"$schema": "https://opencode.ai/config.json",
"model": "amazon-bedrock/global.moonshotai.kimi-k3",
"provider": {
"amazon-bedrock": {
"options": {
"region": "us-west-2",
"profile": "PLACEHOLDER-YOUR-AWS-PROFILE-NAME"
}
}
}
}Once the Amazon Bedrock provider is set up, you can use the /models command to switch models to global.moonshotai.kimi-k3 and start building.
Kimi K3 can build substantial features and work over long-horizon tasks. In the following video, we try it out building a single-file browser-based game to get started:
Figure 1: Building a browser-based game with Kimi K3 in OpenCode
Beyond coding, Hermes Agent is one example of an open source assistant for general productivity. It can be used through a desktop app or popular messaging apps as well as the terminal, and supports use cases like deep research and task automation where Kimi K3 can also perform well.
As detailed in their documentation, Hermes natively supports models on Amazon Bedrock. To get started:
hermes model from your terminal.global.moonshotai.kimi-k3 as a custom model name.If you use named profiles to manage multiple AWS credentials in your environment, then at the time of writing you need to set the AWS_PROFILE environment variable or use your default profile for Hermes. Alternatively, you can switch to an API key. Follow the open issue here for updates on support for setting AWS profile via the Hermes configuration file.
Once the Amazon Bedrock provider is set up and the model configured, you can start using Kimi K3 for your agentic workflows in Hermes. For example, see the following short video in which we ask the agent to build out a personalized study plan:
Figure 2: Building a personalized study plan with Kimi K3 in Hermes Agent
Kimi K3 is available today on Amazon Bedrock through the US Geo (us.) and Global (global.) cross-Region inference profiles. See Bedrock documentation for the full list of supported Regions. For pricing information, see Amazon Bedrock pricing.
Give Kimi K3 a try in the Amazon Bedrock console, or explore the Moonshot AI on AWS samples repository on GitHub.
Interested in how Amazon Bedrock can support your team? Connect with us to start the conversation.
Alex is an AI Specialist Solutions Architect at AWS, based in Singapore. He focuses on how open source technologies and open weight models can help customers around the world to build innovative AI solutions and tackle AI governance challenges.
Saurabh is a Senior Product Manager for Amazon Bedrock and Amazon SageMaker Inference. He is passionate about working with customers and partners, motivated by the goal of democratizing AI. He focuses on core challenges related to deploying complex AI applications, inference with multi-tenant models, cost optimizations, and making the deployment of generative AI models more accessible. In his spare time, Saurabh enjoys hiking, learning about innovative technologies, following TechCrunch, and spending time with his family.
William is Principal Product Manager for Amazon Bedrock.
Tanvi is a Product Marketing Manager for Amazon Bedrock at Amazon Web Services (AWS), where she helps customers adopt and scale AI applications and agents with Amazon Bedrock.
Sofian is a technology leader with over 12 years of experience building AI solutions, and leading high-performing teams to maximize customer outcomes. He is passionate about empowering diverse talents to drive global impact and achieve their career aspirations.
Introducing composable, module system native and agent friendly command line tools for modern Java development
By Danny Thomas, JVM Ecosystem Team
Recent work on the Java language to pave the on-ramp has made it easier than ever to start a Java program and evolve it using the full language and platform. At the end of that on-ramp lies Java’s mature build and dependency management ecosystem, capable of carrying software to enormous scale and complexity.
That ecosystem reached its maturity by developing strong models for projects, dependencies, and builds. When the Java Module System arrived, those models were already serving developers exceptionally well. The module descriptor consequently became just another description of the project to keep in agreement.
We’re excited to announce a preview of ja and its family of composable tools, that build on the capabilities of the Java Module System to provide a modern command line development experience for Java. We take the module descriptor and make it a complete description of a project, with dependency versions sitting naturally beside its requires directives and module metadata provided through documentation tags:
/**
* @mainClass com.example.application.Main
*/
module com.example.application {
requires com.example.framework; // @1.2.3
}
Combined with command line ergonomics you’re used to in other languages, creating and consuming Java modules has never been easier.
Java developers have long been exceptionally well served by graphical tools. An IDE formats source, navigates between declarations and usages, presents API documentation, and maintains a compiled view of the project. That experience has been so complete that Java has had less need to expose the same capabilities through small, composable command line tools. Those gaps become quickly apparent when coding agents work with the Java language, with agents frequently struggling to locate dependencies, documentation and sources.
ja only provides command line ergonomics and tool orchestration, each feature is underpinned by a standalone tool. You don’t need to adopt ja to get the benefit of these tools, you can compose them in any way you choose:
These projects use the tool discovery and execution capabilities of the platform, and are intended to be installed in your JDK along with the standard tools. They all implement Tool or ToolProvider, allowing them to be run in process.
This is also the tool discovery and execution model for ja. There we use OptionChecker and optional custom metadata to discover which module system options are supported so it can resolve the arguments on behalf of the tool. This provides a seamless transition from your source path modules to the standard JDK tooling such as jdeps, jlink and jshell.
In a recent survey of the 1,000 most popular artifacts on Maven Central, just 232 had explicit module definitions and another 248 declared automatic module names. The remaining 520 expressed no Java module name opinion. The module system also makes no distinction between namespace and module name, so module-first tooling requires a solution to module naming and location in existing repositories.
Fortunately, Maven Central already gives published artifacts a verified namespace. Publishers prove control of reverse domain group IDs, reflecting Sonatype’s long standing case for namespaces in public repositories.
We use these conventions to establish a canonical Maven module coordinate, paring a verifiable DNS namespace with the complete module name, for example pkg:maven/com.netflix/com.netflix.tools.ja. For existing modules, authors choose to publish a single Maven relocation pom at the canonical coordinate, to allow for discovery of the original coordinate.
When neither are available, candidates are walked from the root of the namespace using common Maven artifact conventions inferring coordinates from module names. We also bundle a short list of aliases for the most popular modules that don’t use a reverse DNS module name, but we suggest authors should always namespace their modules. The module proxy in jig presents resolved modules using the filename based conventions for module naming, making even automatic modules without stable names safe when used with these tools.
These conventions and location strategies allow the majority of existing artifacts to be discovered using only the module name and version.
ALL-UNNAMED has become unfortunately common in Java access options, because of the heavy use of the class path. It hides the source of the technical debt that applications are incurring by allowing such access and becomes increasingly consequential as Java moves toward Integrity by Default. For example, Preparing to Make Final Mean Final asks applications to explicitly authorize the modules allowed to mutate final fields.
We allow runtime access requirements to bedeclared as module metadata and carried with the module descriptor throughout the module’s lifecycle. For example a library may record the access it requires:
/**
* @enableFinalFieldMutation com.example.framework
*/
module com.example.framework {
}
However, the consuming application remains in control and must explicitly authorize the framework, for it to be available at runtime:
/**
* @mainClass com.example.application.Main
* @enableFinalFieldMutation com.example.framework
*/
module com.example.application {
requires com.example.framework; // @1.2.3
}
The command line interface for ja allows the dependency and authorization to be added together:
ja require com.example.framework@1.2.3 \
--enable-final-field-mutation com.example.framework
Without that authorization, dependency resolution fails with an unsatisfied access requirement. Native access follows the same model through @enableNativeAccess and qualified exports and opens are also supported.
Module integrity is ensured by persistent hashes of resolved binary dependencies in a module-info.hash file, sequent resolution verifies those hashes and rejects an artifact that has changed.
We also take a step further than the recent improvements to annotation processor security by treating annotation processing as an explicit code generation step. The resulting sources are alongside regular module source, making them visible in code review and allowing a module to be assembled without executing generator code.
We think every Java project should be modular, regardless of the build tool you’re using. If you’re a library author producing automatic modules, we’d encourage you to avoid split packages and produce explicit modules.
You can get started with our tools today with our installation guide.
Leave the Class Path in the Rearview Mirror was originally published in Netflix TechBlog on Medium, where people are continuing the conversation by highlighting and responding to this story.
AI is accelerating software development at an unprecedented pace. But as code generation scales, so do the challenges of securing the code, especially emerging AI-based vulnerability exploitations. To meet these challenges, the Google AI and Infrastructure team is transforming how we approach security. In this article, we discuss new AI-native agentic methods that we’ve developed that systematically embed high-precision, pervasive vulnerability scanning and patching directly into Google’s software development lifecycle. By continuously scanning every code change across hundreds of millions of lines of code that we deploy onto our infrastructure, we are preventing hundreds of vulnerabilities per month from ever reaching our code base or production, defending our global network, AI infrastructure and our users.
Solution architecture and implementation
Pervasive pre-submit agentic scanning: security as part of ongoing software development
Traditionally, the technology industry relies on large one-off security scans that are slow and lack sufficient context. As a result, they often find vulnerabilities too late. Our approach instead focuses on pre-submit scanning, where we evaluate each code check-in (across every layer of the stack) in real-time using AI agents. By integrating the pre-submit scan into the tools developers already use, security becomes a continuous routine process, similar to rule checkers, readability reviews or other software development tools. Also, from an AI perspective, scanning each individual code change requires much less context than performing a large one-off scan, significantly improving the scan’s effectiveness.
For this initiative, we evolved Mantis, our open-source multi-agent review harness, to increase the precision of our security agents by matching them with a cohort of robust localized threat models. Rather than relying on static decoupled documents, the threat models use live codebase metadata. The scanning agent improves its accuracy further using a dependence call graph across packages and libraries to expand and refine its threat model context. Making threat models part of our ongoing vulnerability scanning encourages developers to continuously update threats and dependencies, keeping the models up-to-date. Using localized and precise threat model data translates to dramatic accuracy improvements, bringing our false-positive rates down to 3% in some cases.
Vulnerability scanning as part of code check-in requires it to respond quickly to the developer or agents generating the code, so as not to impede engineering productivity. To get responses with low latency, we run a two-step validation process. First, we run a quick lightweight scan that validates its findings against a specialized triage agent. This agent programmatically checks the actual structure of the code (using abstract syntax tree parsing, call-graph traversal, and pre-indexed domain safety rules) to prove that the vulnerable path is actually reachable by an attacker. This agent gets over 92% precision and completes its work in less than a minute. Then, a post-submit scan as part of nightly integration testing serves as a second layer of defense, using off-peak cycles to test for vulnerabilities that may have been introduced across multiple changes.
Finding vulnerabilities is only half the battle. The last component of our solution is an automated bug-fix agent that uses the scan results and generated proofs (snippet of code that demonstrates how the vulnerability is exercised) to autonomously construct precise fixes that are consistent with our internal coding standards. The agent submits the fixes for human review as part of the original change request’s review, further reducing the time between detection and resolution.
Embedding continuous scanning directly into the software development lifecycle has been a game changer at Google; its suggestions are widely adopted, and it’s prevented a multitude of vulnerabilities from being introduced into the codebase. But any organization wishing to improve security can adopt a similar AI-native approach, following these principles:
Keep systems separate: To prevent bias, keep the harnesses, rules, and context for each of your development, scanning, triage agents separate. Pair lightweight AI scans with deterministic, structural validation to drive down latency and improve accuracy.
Use context wisely: Feed your agents your existing threat models. Precise context is the answer to reducing false positives, and up-to-date threat models set a high floor on a team's security posture by improving the rate of true positives in presubmit scanning.
Build a good harness: While the choice of the underlying model is important, using a multi-agent harness can have substantial impact, by helping compensate for variability in model choice.
Automate the fix: Use agents to also propose human-in-the-loop fixes, to further reduce time-to-resolution.
If you want to get started on your own AI-native security transformation, Mantis is now available as open source for you to use and benefit from. You can also learn more about the fundamentals of cybersecurity and the other platforms that power this agentic pipeline: Google Cloud, Gemini Enterprise and Gemini models running on Trillium and Ironwood TPUs. And you can get inspiration from how agentic vulnerability scanning and remediation defends Google Cloud customers as an integral part of Google Cloud’s secure software development lifecycle (SDLC) effort.
With special recognition to critical team members who made this delivery possible: Stella Voutsina (Lead Program Manager), Yulong Zhang (Senior Staff Security Engineer, Mantis), and Nick Galloway (Staff Security Engineer, Mantis).
Agents are no longer experiments. They process claims, write and review code, coordinate across systems, and run for hours without supervision. As agents take on more complex, longer-running work, the infrastructure underneath them must evolve just as fast.
We built Amazon Bedrock AgentCore to help developers build, connect, and optimize agents securely at scale. AgentCore runtime, a capability of Amazon Bedrock AgentCore, is the managed compute layer that gives developers a fully managed environment to deploy and run agents without building or maintaining infrastructure.
Since launch, thousands of teams have used it to run production agents. Every conversation with those teams teaches us something about what agents need next: faster responsiveness as workloads scale, finer control over resource allocation, and economics that track actual usage precisely.
Today, we are announcing the new AgentCore runtime, purpose-built for the speed, flexibility, and cost efficiency that production agents demand.
It brings better memory management, reclaiming memory as a session releases it instead of holding it at the peak. It also delivers consistent cold start times regardless of container size or agent concurrency. You get the serverless model you already liked, now more elastic. Memory is released back the instant a session ends, startup times stay consistent regardless of size or concurrency, and the bill tracks the work your agent does.
Many agents started as chat bots: you asked, it answered, and the exchange ended in seconds. Then came coding agents that work for minutes to hours, holding context across many steps, running while you watch or step away. Now agents are becoming ambient, always on, triggered by events, running unattended, surfacing only when a job finishes or hits a decision that needs a person. And there are far more of them: no longer novelties but running everywhere. They are embedded in products, behind everyday features, and increasingly launched by other agents.
The first version of AgentCore runtime built a strong foundation for this spectrum of agents: serverless, session isolation, scale to zero, and pay only for what you use. Today’s launch of the new runtime extends that foundation across the full spectrum, staying fast and consistent for interactive agents, and durable and affordable for long-running, more autonomous agents.
With AgentCore runtime, you can focus on the agent instead of worrying about the scalable infrastructure needed underneath it. Two things make that possible, and they’re the reasons customers reach for it:
Together they make it cheap to keep many agents idle most of the time and even cheap to run one that stays busy. The consumption model bends to the workload instead of forcing the workload to bend to it.
As agents move from short question-and-answer sessions to ambient, always-on work, that same model runs into two challenges.
Memory is expensive, and today you pay the peak. A session holds on to memory from the moment it allocates it until the session ends, because nothing reclaims it along the way. This works when the allocated memory is used to serve subsequent resources without incurring the latency to fetch it again. However, a long-running or bursty agent keeps paying for its high point the whole time it runs, well after it has stopped using that memory. For an agent that spikes now and then but sits idle most of the day, that is the gap between paying for the peak around the clock and paying for the real usage.
Startup times vary. Every new session has to start before it can do any work, so fast, predictable startup is central to a good experience. It matters most when a person is waiting on an agent that paused for input and needs to resume. The catch is the hardware-enforced isolation these sessions depend on: a session that lands on an already-initialized environment starts in under 100 milliseconds, but keeping environments hot enough to guarantee that means holding compute in reserve. So most sessions begin with a cold start: booting a fresh environment, pulling the image, and initializing the agent before the first request runs. That latency penalty grows with image size and concurrency, and it’s worst under bursty traffic, exactly when most sessions arrive and the fewest ready environments remain. That inconsistency is what a waiting user feels.
The workarounds are heavy. To cover both challenges, customers often build the machinery themselves: holding spare environments ready so requests avoid a cold start, optimizing memory allocation, and tearing it all down again to keep the bill in check. Keeping capacity ready ahead of demand is costly and complex for anyone to run. It reserves scarce compute whether or not that compute is working, and it still gives way when a burst outruns what was set aside. This is undifferentiated work, and none of it is the agent itself.
The enhanced AgentCore runtime takes care of both challenges for you, starting with lower memory consumption tracked to what you use. The new runtime now starts each session from a small, efficient memory profile rather than a full provisioned footprint. Additional memory is allocated and paged in on demand as the workload needs it. Based on an analysis of allocation patterns across billions of sessions, we tuned the new runtime to reclaim memory when it goes cold and is unlikely to be accessed again. It no longer holds that memory until the session ends. With the original runtime, allocated memory remained held even if it wasn’t used by subsequent requests, so the usage tracked the high watermark. With the new runtime, memory that is released or goes cold is reclaimed, and the bill tracks those changes over the lifetime of the session.
Faster, more consistent cold starts come as a direct benefit of smaller profiles at startup. The enhanced runtime prepares the environment once, snapshots it, and restores that snapshot for each new instance. Because the snapshot stays small and consistent, so do the starts, no matter the image size or how much concurrency you run. Rather than repeating the boot-and-initialize work on every cold start, the platform restores an environment that is already up. The runtime now delivers consistent starts in a tight, predictable range.
What we measured. To isolate what the platform itself adds to a cold start, we tested an empty echo agent that returns its input and calls no model and no tools. The timing reflects the runtime’s start path rather than any application work. A Python client on an Amazon Elastic Compute Cloud (Amazon EC2) instance in us-west-2 called agents in us-east-1 over the public internet with no virtual private cloud (VPC) peering, using the boto3 SDK. These are client-side numbers, so each one includes the round trip between the two AWS Regions on top of the platform’s own start time. We sent 5,000 cold invocations per agent across both versions and five image sizes, within default account quotas.
Measured this way, the new runtime delivers a P75 cold start latency of about 2 seconds from a 200 MB image all the way to 2 GB, because image size has no effect on it. The original runtime’s latency, by contrast, rises with image size, from roughly 5.4 seconds to nearly 30 seconds.
To put this latency in perspective, it helps to separate cold start latency from what a user waits on. Start time is how long it takes to get a ready environment before your agent code handles its first request. It is not the time the agent spends working. In a production agent, most of the wall-clock time a user experiences comes from the agent loop and its model calls, often several seconds each. In our echo test, the agent’s own code ran in about 34 milliseconds at P75, so nearly everything here is platform start time. The new runtime makes the platform’s portion of the start time fast and predictable, which matters most when a person is waiting on an interactive agent.
A practical tip for interactive agents. You can hide the start time almost entirely by beginning the session as soon as the user engages, for example when they open a chat, even before they type in the input box, rather than waiting for them to submit. The session warms while they are greeted and while they type their first request, so by the time they send that message, the environment is ready.
The next generation of the runtime reworks how sessions use memory, how agents load, and what you pay for.
Beyond what we shipped today, several capabilities are on the way to give you more choice over pricing, compute, compatibility, and control.
Committed baseline discounts. Today’s consumption-based pricing stays and works well for spiky and scale-to-zero workloads. Alongside it, the new runtime will add a baseline pricing option: you reserve a memory floor for a session and burst above it on demand. Baseline pricing suits steady, always-active agent sessions that want predictable cost, while consumption pricing continues to provide greater elasticity.
Larger compute and storage. Expand your agent’s environment with more RAM, vCPU, and session storage.
x86 support. Run the agent, tool, or environment you already have with x86 microVMs. Teams whose code or dependencies target x86 can move an agent, a tool, or an execution environment to AgentCore as-is.
Greater lifecycle control. Suspend and resume sessions with memory snapshotting. Attach to runtime hooks to serialize state before an active session terminates, so sessions can resume indefinitely.
Scoped identity for unattended agents. Unattended agents raise a question a chat turn never did: what is this agent allowed to do when no one is watching it act? Session context keys will give each session its own scoped identity, so an unattended agent, tool, or environment acts with exactly the permissions defined for it and nothing more.
To get started with the new runtime, set the platformVersion parameter to V2 when you create or update a runtime. See the AgentCore Developer Guide for more details on using the runtime.
You can find samples in the AgentCore GitHub samples repo. An accompanying load test example shows the new runtime’s consistent cold start latency in your own AWS account.
Evandro is a Sr. Data Scientist working on Amazon Web Services. He is part of the Global GTM team that helps AWS customers overcome business challenges related to AI/ML on top of AWS, mainly on Amazon Bedrock AgentCore and Strands Agents. He has more than 18 years of experience working with technology, from software development, infrastructure, serverless, to machine learning. In his free time, Evandro enjoys playing with his son, mainly building some funny Lego bricks.
Mark is a Principal AI Architect for AWS, helping customers design and build agentic AI solutions. Mark’s work covers a wide range of use cases, with a primary interest in AI agents at enterprise scale. He is a worldwide tech lead for Agentic AI, including Bedrock AgentCore. Mark has helped companies in insurance, financial services, media and entertainment, healthcare, utilities, and manufacturing. Prior to joining AWS, Mark was an architect, developer, and technology leader for over 25 years, including 19 years in financial services.
Shishir is a Principal Engineer in AWS, currently building Amazon Bedrock AgentCore Runtime. His experience spans the full agentic stack, drawing on deep work across AI systems, from developing conversational agents in Alexa and LLM post-training and customization to recommender systems in Prime Video. He now focuses on making the infrastructure that powers production agentic systems more reliable, efficient, and scalable.
Abhishek is a Senior Software Development Engineer at AWS on the Bedrock AgentCore team. He is the tech lead for AgentCore Runtime and has led the design and development of multiple AgentCore services from the ground up, including Runtime, Code Interpreter, and Browser. He has 12 years of experience building distributed systems, previously on Bedrock and SageMaker. Outside of work, he likes playing soccer and tennis, and spending quality time with family.
Aniketh is a Software Development Engineer at AWS on the Amazon Bedrock AgentCore team, working on AgentCore Runtime with a focus on the performance and efficiency of agent execution at scale. He has over five years of experience building large-scale distributed systems at Amazon, previously on Amazon SageMaker, and now works on making the infrastructure behind production agentic systems faster and more reliable as it scales to meet growing demand. Outside of work, he enjoys hiking, watching movies, and playing cricket.
Rahul is a Software Development Engineer at AWS, where he builds AgentCore Runtime systems that enable AI agents to run reliably at scale. He is passionate about building distributed systems and optimizing infrastructure to simplify the lifecycle of AI agents. Outside of work, he plays semi professional cricket and enjoys exploring the outdoors.
Amazon Bedrock continues to expand its open weight model portfolio with the same security and governance that customers rely on. Today, Kimi K3 from Moonshot AI is generally available on Amazon Bedrock, giving you a powerful new option for coding and knowledge work.
According to Moonshot AI, Kimi K3 is its most capable model and the first open model to reach 2.8 trillion parameters. It combines native vision capabilities with a 1-million-token context window, making it well suited to long-running coding sessions across large repositories, multi-document analysis including scanned pages and screenshots, and extended agent workflows. Moonshot AI reports an approximate 2.5x improvement in scaling efficiency over Kimi K2. On Amazon Bedrock, Kimi K3 runs within the same security boundary as proprietary models, and the same controls for access, encryption, and auditing across your model portfolio. Kimi K3 is the first open weight model on Amazon Bedrock to support explicit prompt caching, helping reduce latency and input costs when reusing context across model calls.
Kimi K3 is available in all AWS Regions where Amazon Bedrock is available through cross-Region inferencing. To get started, visit the Amazon Bedrock Console. To learn more, see the Amazon Bedrock documentation and read the launch blog post.
We dive into these questions and other AI hot takes on the latest episode of the GitHub Podcast.
Hot takes turn complicated topics into one confident sentence. That makes them great for engagement, but not necessarily for understanding.
At the surface level, they do not matter much. You agree, disagree, repost, argue for a few minutes, and move on. Sometimes the take is directionally right. Sometimes it is complete nonsense.
The value of hot takes is in what happens when you stop reacting and start pulling them apart. Under what conditions is this true? What context is missing? What assumptions does it make? What changes when you apply it to real work?
That is where the depth is. A good hot take gives you something sharp enough to question. The questions are where you find the useful ideas.
We explore all this and more in the latest episode of the GitHub Podcast!
Not ready to dive in yet? Here are a few of the common AI hot takes we discussed and what we can get from them.
Yes, you do. You are still responsible for the code.
But that does not mean every generated line needs the same level of attention.
A production authentication refactor deserves a different review process than a CSS experiment. A codebase you have maintained for 10 years steers your instincts differently than one you opened this morning. Pretending every change carries the same risk is not rigor. It is just a bad use of time.
A simple rule: review until you can explain and own the outcome.
Sometimes that work starts before the agent writes anything. You read the current implementation, map the dependencies, identify edge cases, and make a plan. By the time the first implementation exists, you already understand what it should do and where it could go wrong.
Other times, the generated code itself needs most of your attention. You inspect the error handling, permissions, data access, performance, accessibility, and tests.
AI moves the effort around. It does not make the work disappear.
The actual skill is knowing where the risk lives.
The reality is a little more nuanced. More teams are asking candidates how they use AI. That makes sense. These tools are becoming part of software development.
But no one thinks every developer needs the same workflow, the same tools, or the same level of enthusiasm.
The stronger signal is judgment.
Can you explain when you use AI and when you work manually? Can you describe how you review generated code? Can you talk honestly about speed, quality, security, and maintainability? Can you change your process as the tools change?
If a company is building AI products or uses AI heavily in its engineering workflow, refusing to touch AI may make you a bad fit. That is not controversial. But total dependence and total refusal are rarely good answers.
The better answer is a clear explanation of how you work, what you trust the tools to do, and where you keep yourself in the loop.
That kind of fluency is becoming part of the craft.
No. They solve different problems.
The Model Context Protocol gives agents a standard way to connect to tools and data. That standard matters when you want systems to work together reliably. Agents need structured ways to call tools, fetch context, and take action.
Skills are closer to packaged expertise. A skill can explain how a team works, how a project should be changed, how a tool should be used, or which conventions matter. Since skills are often written in Markdown, people can read them too. That readability is part of their value.
MCP can provide access. Skills can explain how to use that access well.
You do not need to pick a winner. Use standards for shared interfaces. Use skills for context, process, and best practices.
The combination is much more interesting than the argument.
RAG is not dead. It is just not the newest thing people want to post about.
Retrieval-augmented generation gives an AI system relevant information outside the model’s training data. That can include documentation, support history, product details, internal knowledge, or codebase context.
Without good retrieval, the model has to rely on what it already knows or spend extra time searching for context. That wastes tokens, slows down the work, and makes incomplete answers more likely.
Good retrieval helps the model start closer to the answer. It narrows the search space and grounds the response in information that actually matters.
Agents, skills, MCP, and RAG can all exist in the same workflow. An agent might use MCP to access a tool, follow a skill for project-specific instructions, and use retrieval to find the right supporting context.
These things are not fighting each other. Treating them like they are misses how people actually build with AI.
There are valid reasons to fine-tune a model. Still, modern models have seen a huge number of common frameworks, patterns, naming conventions, and architectures. If a model cannot make sense of your codebase, there is a decent chance a new teammate will struggle too.
AI is becoming another pressure test for maintainability, alongside code review, testing, onboarding, and the poor person debugging this six months from now.
Clear structure helps. Consistent naming helps. Readable tests, useful abstractions, and current documentation help.
Those things make a codebase easier for an agent to understand, but more importantly, they make it easier for a person to review, debug, and extend.
AI-assisted development rewards codebases that make their intent obvious.
That is a good thing.
AI will keep producing strong opinions because the tools are changing quickly, and we are all still figuring out our workflows.
You do not need to pick a permanent side in every debate.
The better response to an interesting take is not another take. Test the idea. Build something. Document what happened. Give everyone something real to learn from.
Pollinations AI is doing that by experimenting with a generative AI platform where contributors can earn credits, called pollen, by improving the project. People can open and solve issues, contribute models, and complete quests. The project raises real questions about incentives, quality, scale, and what open source contribution could look like when AI lowers the barrier to participation.
Avian Visitors is doing it in a completely different way. It is a build log for a bird-listening e-ink display that turns birds visiting an apartment balcony into changing wall art. It combines a microphone, Raspberry Pi, e-ink screen, 3D-printed parts, generated bird images, and thoughtful documentation.
These projects do not settle every AI debate. They do something more useful: they create evidence, expose tradeoffs, and give other people a place to start.
Read enough code to own the result. Build enough AI fluency to explain how you work. Use MCP when a standard interface helps. Use skills when context and process matter. Keep RAG when grounded information makes the system better. If your code confuses both people and models, treat that as a maintainability problem.
Most importantly, do something with what you learn.
Subscribe to the GitHub Podcast so you never miss an episode!
GPS is a Senior Developer Experience Advocate at GitHub. She helps make GitHub better for developers through community conversations, conference talks, hands-on workshops, useful demos, and a healthy number of memes. In her free time, she builds popular cloud engineering courseware at learntocloud.guide.
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Your email address
We are living in genuinely interesting times. AI is disrupting software development at a pace where new models, tools, and practices appear almost daily. Many teams’ natural first instinct is to spend ever more time chasing updates.
After almost two years of AI product and market research at JetBrains, we’ve come to a different conclusion: the speed of change is not a problem, as long as you can see the bigger picture. We deliberately don’t try to track everything that happens. Instead, we try to understand where everything we observe comes from – and where it is ultimately going. That gives us a prism to look through, a filter that separates signal from noise. It’s also what saves us from change fatigue.
This post is about that prism. But before we get to the framework itself, let’s start where the webinar started: with what we actually see on the market today. Because you can’t build a useful model of the future without first building an honest model of the present.
Looking through our research, three things stand out.
First, AI is already a common part of the developer’s life. People know about it and use it not only at home but at their companies – including the big ones, which are traditionally the slowest to adopt anything new. We no longer question whether AI in software development “is a thing.” It’s here, and it’s staying.
Second, agentic coding is gradually becoming the new normal. More and more developers use AI coding agents that go beyond automated code edits – they actually delegate coding to agents. This fundamentally changes the development loop from “code → validate → fix” in an editor, to “plan → execute → review” in an agentic chat. The biggest push here came from Anthropic’s Claude Code, which by our estimates is used by around 8.5 million coding professionals, earns roughly $7B in yearly revenue, and is broadly considered the best AI coding tool across all categories. Its release also kicked off the race of IDE-agnostic CLI coding agents – with similar offerings now from OpenAI, Google, and a wave of niche players.
Third, AI agents have started moving to the cloud. Tools like Devin have existed for a while, but only now is this trend starting to actually mean something. With more capable models, more powerful agents, and developers better aware of what AI can and cannot do, developers are making a more conscious decision to delegate work to cloud agents, which are more autonomous by design. They still have heavy limitations, but they can already handle simple, low-effort “garbage tasks”, like fixing a linting error found during a CI run.
And yet, here is the paradox: even though everybody uses AI, we can’t say AI is used everywhere. In reality, AI is mostly used for just two main development activities: brainstorming and coding. But development is more than just coding. Many parts of the software development lifecycle remain largely untouched, creating enormous room for further adoption. AI use is growing steadily, but unevenly.
The question everyone wants answered is: what’s the next big thing? But before jumping there, let’s take a small step back and look at the past. We have to build a proper model of reality first, and only then look at the future through it..
What has the evolution of AI in software development looked like so far?
This reads less like a list of features and more like a trajectory. We can draw a line through these points and ask ourselves: what does this line actually mean? Why did all these embodiments of AI in developer tools show up in this particular order? And if we extend the line into the future, where does it lead?
In early 2025, we were asked to collect insights to evaluate our AI strategy. While working on this, we were inspired to create a “theory of everything”: one that explains not only the current state of the field, but what is fundamentally possible. That’s how we ultimately arrived at our own theory of everything for AI development tools. We called it the Artificial Intelligent Development Environments Framework, or the AIDEs Framework.
Like any piece of theory, we started with definitions and assumptions. Definitions let us abstract away from current jargon and narrowed thinking; assumptions draw boundaries around the problem, making it possible to reason about it systematically. This is standard practice in any rigorous discipline, and it’s remarkable how rarely it’s applied to thinking about developer tools.
Artificial Intelligent System (AIS): Any computer system created by humans that demonstrates the traits of “intelligence” while helping users achieve their goals (their Jobs-To-Be-Done). The key insight: people want to feel intelligence from their tools – but that intelligence doesn’t have to come from LLMs. Our IDEs were always considered “intelligent,” yet the core of their capabilities is built on deterministic heuristics. So the principle is: target the user experience of intelligence, not “AI everywhere.”
Artificial Intelligent Development Environment (AIDE): Simply put, an AIS for creating software. There is a huge set of tools used to create software, applied at particular stages of the process and at specific levels of work delegation. In other words: there is a big world outside of IDEs, full of opportunities we might not have considered yet.
Principal and Agent: Terms borrowed from economics and sociology to describe the relationship between two parties in a delegation. The principal is the party whose interests or objectives are being served, while the agent is the party entrusted to act on the principal’s behalf. But keep in mind that both the principal and the agent can be either a human or an AIS. That means we can consider scenarios where an AI principal delegates work to an AI agent, and even where an AI principal delegates work to a human agent.
Software Creation: We use this term instead of “software development,” as the latter might suggest that software is mostly about writing code. In reality, software creation involves many different roles. These roles can be understood as relationships of delegation: a product team may delegate implementation to software engineers, frontend developers may delegate UI design to UX designers, and so on.
The direction of delegation depends on your perspective. A software engineer may see a UX designer as someone they depend on for a particular activity, but from the perspective of the broader product team, both may simply be contributors to a larger process. In this sense, organizational responsibility is relative to the level and perspective from which you view the work. Adding AI does not fundamentally change this structure; it introduces another kind of actor that can participate in these relationships.
We started with four foundational assumptions:
1. Whatever the future becomes, people will still have the goal of creating software. We don’t believe demand for software will decrease or that humanity will find a completely different technology to replace it. On the contrary, digitalization will continue to be the primary driver of both productivity gains and personal evolution, so demand for software will actually increase. And at least in the mid-term, the basic principles of software development will remain the same.
2. The primary driver of change on the market will be the gradual delegation of software creation activities to artificial intelligent systems. Let’s be honest – we’re all a little lazy, and we’d gladly hand off the work we see as routine. All of human history supports this, from the division of labor, to automation, to digitalization – all of it was, at its core, delegation. Delegation is already present on today’s market. At higher levels, humans delegate to other humans (the most comprehensive IT solutions are still created collaboratively), and at lower levels we delegate to artificial systems through process automation. As AISs develop further, they will become essential actors in the division of labor itself – and the rising level of delegation to AIS will become the ultimate metric of their real capabilities and impact.
3. AIS will never fully replace humans, who will retain two key jobs: task specification and oversight. (The article “AI as Normal Technology” dives deeply into this subject.) AI will not “kill” the developer profession, but it will transform what the profession means. Today, high-level task specification and oversight among developer roles is typically done by architects, a senior grade earned over years. In the future, we might see the emergence of junior architects – a new category that would require rethinking not just roles, but the entire system of CS education.
4. With higher levels of delegation to AIS, personal “immersion” into specific development activities will decrease. Simply put: if you’re not the one doing the job, you’ll always know less about it than if you’d done it yourself. This is exactly what happens between human principals and human agents today. As developers delegate activities with lower added value (like code authoring) and focus on higher-value ones (like requirements formulation), their awareness shifts to a “higher level” of the project. This does not mean everyone goes full “vibe coding” (after all, current tools don’t offer solutions for high-level context communication and management). Future developers should be aware of their projects the way development leads are aware of the projects their teams deliver. Solving this “loss of immersion” problem is a prerequisite for elevating delegation – and this is why context abstraction and management of uncertainty matter so much in the framework.
Our framework operates in three dimensions: stages of the software creation process, levels of delegation, and organizational context of development.
The first dimension is a reworked take on the traditional software development lifecycle, focused on outcomes rather than process. We map 35 high-level activities grouped into 5 activity groups — from “Ideation and Conceptualization” to “Delivery, Maintenance, and Feedback Collection.” Any developer will recognize these immediately, so we won’t dwell on them here. Explore the interactive figure below.
During this stage software creators ideate on original problem and potential solution, explore and come up with vision and high level concepts of what they want to create, identify a valuable opportunity and decide whether it’s worth pursuing.
During this stage software creators “operationalise” the initial ideas and concepts into the design of “engineering solution” – a more specific definition of what should be done from the perspective of system and software engineering. After this stage the developer (who will write code) should understand well what should be done, how it should be done and what are the acceptance criteria (“definition of done”).
During this stage software creators create a codebase and related artifacts that realize the design and pass initial validation. In addition any activities that are required to create and validate this codebase / artifacts are also performed here (e.g. setting up DB, working with external services and / or creating custom tools).
During this stage the created codebase is getting verified and validated against initial requirements, acceptance criteria and quality standards. The end of this stage means the software has passed QA – all critical defects are fixed, and stakeholders are confident in the product’s correctness and stability.
During this stage the created codebase is getting delivered to the end users either via deployment (web production environment) or distribution (application stores, file storages, package repositories). In addition, this stage covers the “operational” state of the software solution, which includes maintenance (making sure the software is still available to end users) and feedback collection (for future improvements).
Regarding our methodology: The taxonomy is designed to cover all types of development involving any roles within software teams (not just developers), yet is not so granular that we lose homogeneous groups of activities. The stages look like a linear workflow, but in reality developers jump between stages and between activities within a stage. These activities can also serve as a foundation for formulating high-level developer Jobs-To-Be-Done.
This is the more novel dimension. Here we define the distribution of roles between principal and agent, along with 10 attributes of delegation – autonomy, level of planning, proactivity, and others. Different combinations of roles and attribute values define five levels of delegation:
L1 – Tool. Delegation of very limited, scoped actions. Code completion is the canonical example: you let AI finish writing what you’ve already started.
L2 – Assistant. Delegation of a well-defined sequence of actions – a “task” with very specific boundaries. One example might be generating a unit test for a specific function. Simple, well-defined, and minimal context – but it’s a task with a series of steps, not just one action. It’s like having a third hand: it’s doing the work, but it’s still your hand.
L3 – General-purpose Executor. This is where focus starts shifting from the process to the deliverables. An L3 agent can execute any task, but requires expert input from the principal, who acts as a “consultant” on more complex topics. Current agentic coding sits roughly here: we believe agents like Claude Code and Codex are well capable at code writing and low-level solution engineering, but we still don’t trust them with decisions about what should actually be built – that requires deeper knowledge of the business domain. So we fully delegate execution, but retain task setting and review.
L4 – Supervised Executor. Here we move beyond the individual space to the organizational perspective, because the agent is now responsible for an entire development function, like managing the backend implementation of your full-stack web application. It is “supervised” because the principal’s role narrows to approving key decisions; everything else the agent decides itself. This is also where we run out of real-world examples, except perhaps some usage patterns of vibe-coding platforms like Lovable or Replit.
L5 – Competence Center. Imagine you’re the CEO of a startup with an engineering team at your side. You define what the company wants to achieve, how you’ll do it, what the key metrics are, and whether you’re performing well. Your engineering team exists to execute your strategy and make your vision a reality. You don’t care what stack they use, what API structure the app has, or whether it’s hosted in Azure VMs or Docker containers on managed Kubernetes in GCP – you delegate those decisions to the team. That kind of delegation is L5.
You may have noticed something conspicuously missing here: there’s nothing about the raw capability of AI or how “smart” it can be. This omission is deliberate, for two reasons.
First, benchmark performance does not automatically translate into real-world delegation. AI models can achieve remarkable results on standardized tests and still struggle to earn enough trust from people to perform even relatively simple tasks autonomously. Thus we might see an AI model having top-notch benchmark results but surprisingly little economic impact. Conversely, a deterministic system that effectively orchestrates a set of less capable agents can potentially produce more useful work than a single super-smart AGI.
Second – and this is the deeper point – everything we’ve described is not an attribute of the agent, but an attribute of the relationship between the principal and agent. The level of delegation is a decision made by the principal, based on their personal perception of the agent. A developer may delegate code writing to Junie at L3 and let it execute a task end-to-end, but for more critical cases they’ll switch to L2, put Junie “on a leash,” and feed it much narrower tasks. Even if the agent is capable of L3, there will be scenarios where the principal chooses to delegate less. The level of delegation is not an attribute of Junie – it’s an attribute of the “agentic contract” between the two, and the principal is the one who sets its terms.
Even when AI is technically capable of doing the job, it’s humans who decide how much control to let go of.
The third dimension describes the organizational context in which development happens. We differentiate three contexts:
These contexts define different constraint types and different complexity of organizational dynamics – which are later reflected in the complexity of development decisions and, ultimately, the codebase itself. We added this dimension primarily so we never forget this aspect – and we already see certain things becoming relevant specifically at the scale of large organizations.
Now, remember our “timeline” picture from earlier? Through the lens of the framework, it becomes obvious that the line running through it is, at its core, the level of delegation dimension. But since the model is richer than a single line, we can also track how AI penetration grows across SDLC activities and how it differs across organizational contexts.
In our regular surveys on AI usage, we have a dedicated section on exactly this, which lets us build what we call AIDEs maps.
Continued at the source.
We want local coding agents to be smart and fast, with the ability to understand a codebase, do useful work, and finish tasks without long waits. This Junie Local update makes it practical to use a more capable model on your own machine.
In the first release, we had to choose between two versions of the same model. With reasoning disabled, Qwen3.6 was fast enough to be usable on a laptop. Qwen3.8 completed more tasks, but it needed reasoning enabled to work reliably, and that made tasks take roughly four times longer. We picked speed.
This update is our attempt to remove the need to choose. We built Qwen3.8-3.6-27B-blend by merging the two in equal proportions. In our coding evaluation, it completed more tasks than Qwen3.6 while generating 71% fewer output tokens than Qwen3.8.
In this post, we’ll show where the new model improves coding results, how we made it run efficiently, and what we learned while testing it. We’re also bringing Junie Local to more machines with experimental NVIDIA support on Windows.
In our 100-task internal coding benchmark, the new model completed 37 tasks, compared with 34 for Qwen3.6 with reasoning disabled. It came close to Qwen3.8’s 39 solves while generating 71% fewer output tokens.
One possible explanation for the token savings was just that the blend model spends fewer tokens when it gets stuck. To test that hypothesis, we compared token use for the 30 tasks that were completed by both Qwen3.8 and the blend model. On these tasks, the blend generated about 70% fewer tokens – 279K for the blend versus 935K for Qwen3.8. It used fewer tokens on 29 of those 30 tasks, further proving its token efficiency.
We started with a simple experiment. Since Qwen3.8-27B is based on Qwen3.6-27B, and they both share the same architecture, we simply merged their weights in equal proportions. This produces a single 27B model without any additional post-training.
However, this simple blend was already a surprisingly useful improvement. The early results were better than we expected, so we focused on evaluating this model across more benchmarks and tasks. That evaluation gave us enough confidence to make it the model for this release while the other experiments continue.
There are many ways to reduce reasoning times, including distillation, reinforcement learning, and more elaborate model merging methods. We are continuing a wider set of model and runtime experiments, and more of that work will appear in future Junie Local releases.
To see how the new model performs beyond our agentic coding tasks, we evaluated it on multiple public benchmarks. Repeating the evaluation runs lets us see which tasks are consistently completed, how much variance there is between runs, and whether a result depends on one favorable sample.
Across four LiveCodeBench runs, the blend model averaged 85.47% correct answers, compared with 83.29% for Qwen3.8, at a similar output cost. Qwen3.6’s four complete passes averaged 67.87% and used about 24.1 million output tokens per pass, versus approximately 6.14 million for the blend model.
The visual benchmarks expose a different tradeoff. The blend model used substantially fewer tokens than Qwen3.6 with thinking enabled, but more than Qwen3.8. We checked identical questions, images, and generation settings, and we found that the extra tokens were almost entirely due to the blend model spending more time on reasoning.
The blend can still overthink when it struggles to find a solution. If Junie keeps revisiting the same approach without new evidence or useful tool results, we recommend interrupting it and restarting it with a narrower goal.
There is also room to make successful reasoning more efficient. Across four identical benchmark runs, the length of CoT varied significantly. Picking the shorter correct trace would have cut token use by 24.5%, which suggests that shorter successful paths exist, and we could potentially teach the model to take those paths with zero performance loss.
The model determines how much text Junie generates, while the runtime determines how quickly that text reaches you and how much memory it needs. Our goal is to improve both.
Junie Local already uses multi-token prediction (MTP). A small subnetwork called the MTP head proposes multiple tokens that the main model checks in parallel. Correct proposals result in more output tokens per pass. We want to make more correct proposals, but this also adds GPU work, so it does not always mean faster generation.
On the M5 MacBook Pro, proposing two tokens per round made decoding 60% faster than running without MTP. Increasing that to four brought the speedup down to 36%, because the extra GPU work of drafting and checking proposals outweighed the benefit of accepting more tokens.
We compared how a four-bit MTP head (Q4) and an eight-bit one (Q8) performed on real-world coding trajectories at five context sizes, from 16K to 128K, with three seeds each. Q4 accepted 63.0% of proposals, and Q8 accepted 63.6%:
The acceptance rate tells us how often the guesses are useful, while decode speed tells us whether they save time.
We found no consistent speed advantage for the Q8 MTP head, so we kept Q4 to save memory.
To understand why MTP slows down with longer context, we profiled the GPU load during the token verification process. Calculating attention accounted for most of the increase: Its time rose from 8.4 to 40.2 ms per round, while feed-forward and Gated DeltaNet computations stayed nearly flat.
This MTP limitation results in slower responses as Junie works through a long coding session, even when its predictions remain accurate. We are researching how to reduce this verification cost and keep Junie responsive as sessions go on.
During the early stages of development, our internal evaluations showed performance degradations that we were unable to reproduce when actually using Junie Local. The reason was a setting we had introduced to make evals reproducible: Every request received the same random seed. This caused numeric instability, as reusing the seed gave the same tokens the same random advantage each time the sampler generated a token. When the model’s predictions stayed similar, it could be steered back toward an unsuccessful action even after the prompt changed. Notably, Qwen3.8 was more affected by this instability than the other models we tested.
We corrected the setup by advancing the seed with each agent step and reflection attempt, allowing subsequent attempts to take a different path while keeping the tests reproducible.
Apple M5 users can already try the new model via Junie:
junie
Run /local and install Qwen3.8-3.6-27B-blend to switch Junie Local over to it. Make sure Junie is updated to the latest version.
For Windows users the nightly build of Junie now includes experimental RTX support, covering all NVIDIA RTX cards based on Ampere or newer architectures with at least 24 GB of VRAM.
junie --channel=nightly
This early preview lets you try Junie Local on Windows and help shape its development with your feedback.
You can find Qwen3.8-3.6-27B-blend on Hugging Face.
Qwen3.8-3.6-27B-blend is just one result of our broader model and runtime research. We are continuing that work, and you will see more of its results in future Junie Local releases.
Get started
Models
Core capabilities
Agents
Tools
Live API
Optimization
Guides
Resources
Policies
This page documents updates to the Gemini API.
Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS generally available
(GA): Released our next-generation text-to-speech (TTS) audio models and
the Gemini API Voices endpoint (/v1beta/voices):
gemini-3.8-flash-tts): Flagship creative TTS model engineered for
studio-grade voice fidelity, nuanced acting, regional dialects, and
long-form multi-turn stability.gemini-3.8-flash-lite-tts): Fast, cost-efficient TTS model built to
replace gemini-3.1-flash-tts-preview for high-throughput production
and real-time voice agent cascades.See the Text-to-speech guide to get started.
Antigravity Agent 09-2026: Released antigravity-preview-09-2026,
which replaces and deprecates antigravity-preview-05-2026.
If you run on a remote sandbox (environment: "remote") and read only
output_text or model_output steps, update the agent string and nothing
else changes.
If you run tools locally (local_environment) or parse function_call
steps, the built-in tools changed. Parameters use PascalCase instead of
snake_case, and file edits use line-range replacements instead of full
rewrites.
| Capability | 05-2026 | 09-2026 |
|---|---|---|
| File creation | write_file(path, content) |
write_to_file(TargetFile, CodeContent, Overwrite, Description) |
| File editing | write_file(path, content), full rewrite |
replace_file_content(TargetFile, StartLine, EndLine, TargetContent, ReplacementContent) |
| File reading | read_file(path, offset, limit), byte offsets |
view_file(AbsolutePath, StartLine, EndLine, ContentOffset) |
| Directory listing | list_files(path) |
list_dir(DirectoryPath) |
| File and code search | None, agents used shell commands | find_by_name(SearchDirectory, Pattern, MaxDepth) and grep_search(SearchPath, Query, IsRegex) |
| Shell execution | code_execution(command, timeout_seconds) |
Unchanged |
| Web search | google_search(queries) |
Unchanged |
See the Antigravity Agent guide.
antigravity-preview-05-2026 shuts down on October 5, 2026, tracked on the
deprecations page.
Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking generally available (GA): Released two new audio-to-audio models for real-time voice applications using the Live API:
gemini-3.8-live): The default option
for most low-latency voice agent experiences and real-time dialogue
without reasoning delays. Features interleaved reasoning, default
asynchronous function calling, and full session client content updates.gemini-3.8-live-extended-thinking): High-reasoning
audio-to-audio model supporting background reasoning during live audio
interactions, recommended when higher background reasoning is required.To get started, see the Live API guide, the Capabilities guide, and the Thinking guide.
Lyria 3.5 generally available (GA): Released the next generation of Google's music generation model:
lyria-3.5:
Full-length song generation with improved musical coherence, natural vocals,
and fine-grained duration and structural control.The model supports text and image inputs and generates high-fidelity 44.1 kHz stereo audio. See the Music generation guide for details and code samples.
Gemini 3.8 Flash generally available (GA): Released
gemini-3.8-flash, our most intelligent Flash model, engineered for
long-horizon software engineering, autonomous agents, and complex enterprise
workflows.
To get started, see the Gemini 3.8 Flash model page and the Latest model guide.
Agentic video understanding: Released agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite across the Interactions and GenerateContent APIs. The model dynamically navigates video timelines, requesting transcripts, frames, or audio tracks on demand. This approach uses up to 88% fewer tokens for long-form content compared to static processing.
To get started, see the Agentic video understanding guide.
Gemini Omni Flash generally available (GA): Released
gemini-omni-1.1-flash, the GA version of our fast, conversational video
generation and editing model. This release includes significant new
capabilities:
extend task or directly
with a prompt.image_to_video task with up to 2 images.resolution parameter in video_config
supports 360p, 720p (default), 1080p, and 4k outputs.
1080p and 4K outputs are generated using upscaling.The existing gemini-omni-flash-preview endpoint will be deprecated on
September 30, 2026.
To get started, see the Gemini Omni Flash model page and the omni guide.
Gemini 3.5 Transcribe generally available (GA): Released two dedicated speech-to-text models based on Gemini's audio understanding:
gemini-3.5-transcribe): High-accuracy,
low-latency non-streaming speech-to-text with utterance-based language
detection across 85+ languages, speaker diarization, word-level
timestamps, and custom vocabulary biasing (up to 1,000 terms).gemini-3.5-transcribe-live):
Low-latency, bidirectional streaming speech-to-text over WebSockets using
the Live API, supporting interim and finalized transcription events,
Smart transcription mode, and multiple Voice Activity Detection (VAD)
strategies.To get started, see the Audio transcription guide, the Live transcription guide, and the Gemini 3.5 Transcribe model page.
Gemini 3.7 Flash generally available (GA): Released our most intelligent workhorse model yet for coding and agents:
gemini-3.7-flash): Substantial improvements
across software engineering, web development, and agentic workflows,
available at an introductory price through December 31, 2026.To get started, see the Gemini 3.7 Flash model page and the Latest model guide.
Gemini Robotics ER 2 in public preview: Released two new embodied reasoning model endpoints for robotics:
gemini-robotics-er-2-preview: Advanced spatial reasoning, agentic
code execution, multi-step tool orchestration, video moment finding,
progress classification, and multi-robot coordination.gemini-robotics-er-2-streaming-preview: Optimized for real-time
text streaming using the Live API, enabling low-latency robot agents with
bidirectional audio and video input.Both model endpoints accept text, image, video, and audio inputs and support function calling with blocking behavior for physical robot actions. To get started, see the Gemini Robotics ER overview. For real-time streaming use cases, see Robotics with streaming.
Deprecation announcement: The gemini-robotics-er-1.6-preview model
will be shut down on August 31, 2026.
Gemini 3.6 Flash and Gemini 3.5 Flash-Lite generally available (GA): Released stable, production-ready versions of our latest 3.x Flash models:
gemini-3.6-flash): Features improved token
efficiency and code/agentic planning capabilities at a lower price point
than 3.5 Flash, resolving developer feedback around output verbosity.gemini-3.5-flash-lite): Offers a
low-latency, highly cost-effective subagent option designed for
high-volume automation.To learn more, see the Latest Gemini model guide.
Deprecated parameters: The sampling parameters temperature, top_p
and top_k are now deprecated. See the
Latest Gemini Model
for details.
Gemini Omni Flash in public preview: Released gemini-omni-flash-preview,
a high-performance multimodal model designed for high-speed video generation
and conversational video editing. Using the Interactions API,
you can generate 3–10 second videos at 720p from text descriptions or animate still images,
and then conversationally edit and refine the outputs. To get started, see the
Gemini Omni Flash guide and the
Gemini Omni Flash model card.
Released gemini-3.1-flash-lite-image (Nano Banana 2 Lite) to general
availability (GA), our built-in multimodal model optimized for ultra-low
latency and cost-effective image generation and editing. See the Gemini 3.1
Flash Lite Image model
card and the Image generation guide.
streamGenerateContent
(and stream: true in the Interactions API) is now supported for the
gemini-3.1-flash-tts-preview model. To learn more, see the
Text-to-Speech guide.Deprecation announcement: The following image generation models are being deprecated and will be shut down on August 17, 2026:
imagen-4.0-generate-001imagen-4.0-ultra-generate-001imagen-4.0-fast-generate-001To migrate your code to newer stable or preview endpoints, refer to the Gemini deprecations page.
Deprecation announcement: The following video generation models are being deprecated and will be shut down on June 30, 2026:
veo-2.0-generate-001veo-3.0-generate-001veo-3.0-fast-generate-001Update your integration to either use the Veo 3.1 preview model IDs
(veo-3.1-generate-preview, veo-3.1-fast-generate-preview) or the
3.1 GA models available through the
Gemini Enterprise Agent Platform
to avoid service interruptions.
Deprecation announcement: The experimental GMP Contextual View tool (a fixed interface for Grounding with Google Maps outputs) will shut down on June 15, 2026:
The following Gemini 2.0 models are now shut down:
gemini-2.0-flashgemini-2.0-flash-001gemini-2.0-flash-litegemini-2.0-flash-lite-001Use gemini-3.5-flash or
gemini-3.1-flash-lite
instead.
Released gemini-3.1-flash-image (Nano Banana 2) and gemini-3-pro-image
(Nano Banana Pro), the generally available (GA) versions of our native
visual models, Gemini 3.1 Flash Image
and Gemini 3 Pro Image.
Video-to-image generation support: You can now pass a video file (via
direct upload or as a public YouTube URL) as multimodal context alongside a
text prompt to generate high-quality thumbnails, cinematic movie posters, or
summary infographics. This feature is supported exclusively on the
gemini-3.1-flash-image model. To learn more, see the
Video-to-image generation
guide.
Deprecation announcement: The gemini-3.1-flash-image-preview and
gemini-3-pro-image-preview models are deprecated
and will be shut down on June 25, 2026.
gemini-3.1-flash-lite-preview model has been
shut down. Use
gemini-3.1-flash-lite instead.Released gemini-3.5-flash, the generally available (GA) version of
Gemini 3.5 Flash,
our most intelligent model for sustained frontier performance on
agentic and coding tasks. This is now the model behind gemini-flash-latest.
Launched the Managed Agents in the Gemini API in public preview. This enables developers to build and deploy autonomous, stateful agents that run in secure, isolated Google-hosted Linux sandbox environments. To learn more, see the Agents overview page and the Quickstart.
Released the general-purpose Antigravity Agent managed agent,
antigravity-preview-05-2026, in public preview.
The Antigravity agent can autonomously plan, reason, write and execute code,
manage files, and browse the web inside its sandbox container. See the
Antigravity Agent guide for code
samples and specifications.
Released gemini-3.1-flash-lite, the generally available (GA) version of
Gemini 3.1 Flash-Lite,
optimized for speed, scale, and cost efficiency.
Deprecation announcement: The gemini-3.1-flash-lite-preview model is
deprecating on 5/11/26 and will be
shut down on May 25, 2026.
outputs → steps) and output format
configuration (response_format) are changing. The new schema becomes the
default on May 26 and the legacy schema will be removed on June 8.
See the
migration guide
for details.gemini-embedding-2 model.
Grounding metadata now includes media_id for visual citations and
page_numbers that indicate where information is found. To learn
more, see the File Search guide.gemini-robotics-er-1.5-preview model has been
shut down. Use
gemini-robotics-er-1.6-preview instead.gemini-embedding-2 as generally available
(GA). To learn more, see the Embeddings page.Released new versions of the Deep Research agent with collaborative planning, visualization support, MCP server integration, and File Search:
deep-research-preview-04-2026: Designed for
speed and efficiency, ideal to be streamed back to a client UI.deep-research-max-preview-04-2026: Maximum
comprehensiveness for automated context gathering and synthesis.Released gemini-robotics-er-1.6-preview, our updated robotics model.
It now has new capabilities like instrument reading, improved spatial and
physical reasoning capabilities. To learn more, see
Gemini Robotics ER page and the
blog.
Deprecation announcement: The gemini-robotics-er-1.5-preview model
will be shut down on April 30, 2026 at 9AM
PST.
gemma-4-26b-a4b-it and gemma-4-31b-it, available on
AI Studio and through the Gemini API,
as part of the Gemma 4 launch.Launched Veo 3.1 Lite Preview, veo-3.1-lite-generate-preview, our most
cost-efficient video generation model, designed
for rapid iteration and building high-volume applications.
The gemini-2.5-flash-lite-preview-09-2025 model has been shut down. Use
gemini-3.1-flash-lite-preview instead.
gemini-3.1-flash-live-preview, the latest
audio-to-audio (A2A) model designed for real-time dialogue and voice-first
AI applications. Read the Live API docs to get
started.lyria-3-clip-preview
(30-second clips) and lyria-3-pro-preview
(full-length songs). Both models accept text and image inputs and generate
high-quality, 48kHz stereo audio. See the
Music generation guide for details and
code samples.Released gemini-embedding-2-preview, our first multimodal embedding model.
It supports text, image, video, audio, and PDF inputs,
mapping all modalities into a unified embedding space. To learn more, see
Embeddings.
Deprecation announcement: The gemini-2.5-flash-lite-preview-09-2025 model
will be shut down on March 31, 2026.
gemini-3-pro-preview now points to
gemini-3.1-pro-preview.Launched Nano Banana 2, Gemini 3.1 Flash Image Preview, a high-efficiency model optimized for speed and high-volume use cases.
Deprecation announcement: Gemini 3 Pro Preview (gemini-3-pro-preview)
will be shut down March 9, 2026.
gemini-3.1-pro-preview-customtools, which is
better at prioritizing custom tools, for users building with a mix of bash
and tools.Deprecation announcement: The following models will be shut down June 1, 2026:
gemini-2.0-flashgemini-2.0-flash-001gemini-2.0-flash-litegemini-2.0-flash-lite-001The following models are shut down:
gemini-2.5-flash-preview-09-25imagen-4.0-generate-preview-06-06imagen-4.0-ultra-generate-preview-06-06gemini-3-pro-preview and
gemini-3-flash-preview.Changed the latest aliases:
gemini-pro-latest switched to gemini-3-pro-previewgemini-flash-latest switched to gemini-3-flash-previewDeprecation announcement: The following models will be shut down February 17, 2026:
gemini-2.5-flash-preview-09-25imagen-4.0-generate-preview-06-06imagen-4.0-ultra-generate-preview-06-06The gemini-2.5-flash-image-preview model has been shut down.
text-embedding-004 model has been shut down.Launched model lifecycle feature. Some models will now specify the lifecycle stage and deprecation timeline. See the following documentation for more information:
total_reasoning_tokens field has been renamed to
total_thought_tokens to better align with the concept of "thoughts" in
thinking models.Launched Gemini 3 Flash Preview, gemini-3-flash-preview, delivering fast
frontier-class performance that rivals larger models at a fraction of the
cost. With upgraded visual and spatial reasoning, and agentic coding
capabilities. Read the documentation on some new features, including:
gemini-2.5-flash-native-audio-preview-12-2025,
a new native audio model for the Live API. This update improves the model's
ability to handle complex workflows. To learn more, see the
Live API guide and
Gemini 2.5 Flash Native Audio.Launched the Interactions API. This API provides a unified interface for interacting with Gemini models and agents. To learn more, see the Interactions API guide.
Launched the Gemini Deep Research agent in preview. It can autonomously plan, execute, and synthesize results for multi-step research tasks. See the Deep Research guide for details.
gemini-2.0-flash-live-001gemini-live-2.5-flash-previewgemini-2.5-flash-image-preview model will be
shut down January 15, 2026.text-embedding-004 model will be shut down
January 14, 2026.gemini-3-pro-image-preview, the
next iteration to the Nano Banana model. Read the Image generation page for more details.Launched the first Gemini 3 series model, gemini-3-pro-preview, our
state-of-the-art reasoning and multimodal understanding model with powerful
agentic and coding capabilities.
In addition to improvements in intelligence and performance, Gemini 3 Pro Preview introduces new behavior around:
Read the Gemini 3 Developer Guide for migration, new features, and specs.
Deprecation announcement: The following models will be shut down:
November 12:
veo-3.0-fast-generate-previewveo-3.0-generate-previewNovember 14:
gemini-2.0-flash-exp-image-generationgemini-2.0-flash-preview-image-generationThe following model is shut down:
imagen-3.0-generate-002Use Imagen 4 instead. Refer to the Gemini deprecations table for more details.
For Gemini 2.5 Flash Image, the input token count for images has been reduced from 1290 to 258, lowering the cost of image editing.
Deprecation announcement: The following models will be shut down:
November 18th:
gemini-2.5-flash-lite-preview-06-17gemini-2.5-flash-preview-05-20December 2nd:
gemini-2.0-flash-thinking-expgemini-2.0-flash-thinking-exp-01-21gemini-2.0-flash-thinking-exp-1219gemini-2.5-pro-preview-03-25gemini-2.5-pro-preview-05-06gemini-2.5-pro-preview-06-05December 9th:
gemini-2.0-flash-lite-previewgemini-2.0-flash-lite-preview-02-05gemini-2.0-flash-expgemini-2.0-pro-expgemini-2.0-pro-exp-02-05The following Gemini Live API models are now shut down:
gemini-2.5-flash-preview-native-audio-dialoggemini-2.5-flash-exp-native-audio-thinking-dialogYou can use gemini-2.5-flash-native-audio-preview-09-2025 instead.
Deprecation announcement: Shut down for gemini-2.0-flash-live-001 and
gemini-live-2.5-flash-preview coming December 09, 2025.
Released Veo 3.1 and 3.1 Fast models in public preview, with new features including:
This launch also added more options for Veo 3 output video durations: 4, 6, and 8 seconds.
Deprecation announcement: Shut down for veo-3.0-generate-preview and
veo-3.0-fast-generate-preview coming November 12, 2025.
gemini-1.5-progemini-1.5-flash-8bgemini-1.5-flashReleased Gemini Robotics ER 1.5 model in preview. See the Robotics overview to learn about how to use the model for your robotics application.
Launched following preview models:
gemini-2.5-flash-preview-09-2025gemini-2.5-flash-lite-preview-09-2025See the Models page for details.
gemini-2.5-flash-native-audio-preview-09-2025,
a new native audio model for the Live API with improved function calling
and speech cut off handling. To learn more, see the
Live API guide and
Gemini 2.5 Flash Native Audio.Deprecation announcement: The following models will be shut down in October 2025:
embedding-001embedding-gecko-001gemini-embedding-exp-03-07 (gemini-embedding-exp)See the Embeddings page for details on the latest embeddings model.
gemini-2.0-flash model
(available during experimental release) will be discontinued in one week.allow_adult setting in Image to Video generation are now available in
restricted regions. See the
Veo
page for details.gemini-2.5-flash-lite, our fast, low-cost, high-performance Gemini
2.5 model. To learn more, see Gemini 2.5
Flash-Lite.Launched veo-3.0-generate-preview, the latest update to Veo introducing
video with audio generation. To learn more about Veo 3, visit the Veo page.
Increased rate limits for Imagen 4 Standard and Ultra. Visit the Rate limits page for more details.
gemini-embedding-001, the stable version of our
text embedding model. To learn more, see
embeddings. The gemini-embedding-exp-03-07
model will be deprecated on August 14, 2025.The preview models gemini-2.5-pro-preview-05-06 and
gemini-2.5-pro-preview-03-25 are now redirecting to
the latest stable version gemini-2.5-pro.
gemini-2.5-pro-exp-03-25 is shut down.
gemini-2.5-pro, the stable version of our most powerful
model, now with adaptive thinking. To learn more, see
Gemini 2.5 Pro
and Thinking. gemini-2.5-pro-preview-05-06
will be redirected to gemini-2.5-pro on June 26, 2025.gemini-2.5-flash, our first stable 2.5 Flash model. To learn
more, see Gemini 2.5 Flash.
gemini-2.5-flash-preview-04-17 will be deprecated on July 15, 2025.gemini-2.5-flash-lite-preview-06-17, a low-cost, high-performance
Gemini 2.5 model. To learn more, see Gemini 2.5 Flash-Lite
Preview.gemini-2.5-pro-preview-06-05, a new version of our most powerful
model, now with adaptive thinking. To learn more, see
Gemini 2.5 Pro Preview
and Thinking.
gemini-2.5-pro-preview-05-06 will be redirected to gemini-2.5-pro on
June 26, 2025.API updates:
generateContent request.Model updates:
gemini-2.5-flash-preview-05-20, a Gemini
preview model optimized for
price-performance and adaptive thinking. To learn more, see
Gemini 2.5 Flash Preview
and Thinking.gemini-2.5-pro-preview-tts
and
gemini-2.5-flash-preview-tts
models, which are capable of
generating speech with one or two
speakers.lyria-realtime-exp model, which
generates music in real time.gemini-2.5-flash-preview-native-audio-dialog and
gemini-2.5-flash-exp-native-audio-thinking-dialog,
new Gemini models for the Live API with native audio output capabilities. To
learn more, see the
Live API guide and
Gemini 2.5 Flash Native Audio.gemma-3n-e4b-it preview, available on
AI Studio and through the Gemini API,
as part of the Gemma 3n launch.gemini-2.0-flash-preview-image-generation, a preview model for
generating and editing images. To learn more, see Image
generation and
Gemini 2.0 Flash Preview Image
Generation.gemini-2.5-pro-preview-05-06, a new version of our most powerful
model, with improvements on code and function calling. gemini-2.5-pro-preview-03-25
will automatically point to the new version of the model.gemini-2.5-flash-preview-04-17, a Gemini
preview model optimized for
price-performance and adaptive thinking. To learn more, see
Gemini 2.5 Flash Preview
and Thinking.Model updates:
veo-2.0-generate-001, a generally available (GA) text- and
image-to-video model, capable of generating detailed and artistically
nuanced videos. To learn more, see the Veo docs.Released gemini-2.0-flash-live-001, a public preview version of the
Live API model with billing enabled.
Enhanced Session Management and Reliability
GoAway server
message indicating when a connection is about to close, allowing for
graceful handling before termination.More Control over Interaction Dynamics
Configurable Voice Activity Detection (VAD): Choose sensitivity
levels or disable automatic VAD entirely and use new client events
(activityStart, activityEnd) for manual turn control.
Configurable Interruption Handling: Decide whether user input should interrupt the model's response.
Configurable Turn Coverage: Choose whether the API processes all audio and video input continuously or only captures it when the end-user is detected speaking.
Configurable Media Resolution: Optimize for quality or token usage by selecting the resolution for input media.
Richer Output and Features
Expanded Voice & Language Options: Choose from two new voices and
30 new languages for audio output. The output language is now
configurable within speechConfig.
Text Streaming: Receive text responses incrementally as they are generated, enabling faster display to the user.
Token Usage Reporting: Gain insights into usage with detailed
token counts provided in the usageMetadata field of server messages,
broken down by modality and prompt or response phases.
gemini-2.5-pro-preview-03-25, a public preview Gemini 2.5 Pro version
with billing enabled. You can continue to use gemini-2.5-pro-exp-03-25 on
the free tier.gemini-2.5-pro-exp-03-25, a public experimental Gemini model
with thinking mode always on by default.
To learn more, see
Gemini 2.5 Pro Experimental.Model updates:
gemma-3-27b-it, available on
AI Studio and through the Gemini API,
as part of the Gemma 3 launch.API updates:
SDK updates:
Model updates:
gemini-embedding-exp-03-07, an
experimental
Gemini-based embeddings model in public preview.API updates:
gemini-2.0-pro-exp-02-05, an experimental model based on
Gemini 2.0 Pro.Model updates:
gemini-2.0-flash-lite, a generally available (GA) version of
Gemini 2.0 Flash-Lite,
which is optimized for speed, scale, and cost efficiency.AI Studio updates:
API updates:
Model updates:
API updates:
Model updates:
imagen-3.0-generate-002, a generally available (GA) version of
Imagen 3 in the Gemini API.SDK updates:
Model updates:
gemini-2.0-flash-001, a generally available (GA) version of
Gemini 2.0 Flash that
supports text-only output.gemini-2.0-pro-exp-02-05,
an experimental public
preview version of Gemini 2.0 Pro.gemini-2.0-flash-lite-preview-02-05, an experimental public
preview model
optimized for cost efficiency.API updates:
SDK updates:
Model updates:
gemini-2.0-flash-thinking-exp-01-21, the latest preview version of
the model behind the
Gemini 2.0 Flash Thinking Model.Model updates:
Released Gemini 2.0 Flash Thinking Mode for public preview. Thinking Mode is a test-time compute model that lets you see the model's thought process while it generates a response, and produces responses with stronger reasoning capabilities.
Read more about Gemini 2.0 Flash Thinking Mode in our overview page.
Model updates:
Read more about Gemini 2.0 Flash in our overview page.
Model updates:
gemini-exp-1121, an even more powerful experimental Gemini API model.Model updates:
gemini-1.5-flash-latest and gemini-1.5-flash model aliases
to use gemini-1.5-flash-002.
top_k parameter: The gemini-1.5-flash-002
model supports top_k values between 1 and 41 (exclusive).
Values greater than 40 will be changed to 40.Model updates:
gemini-exp-1114, a powerful experimental Gemini API model.API updates:
API updates:
Model updates:
gemini-1.5-flash-8b-001, a stable version of our smallest Gemini
API model.Model updates:
gemini-1.5-pro-002 and gemini-1.5-flash-002, two new stable
versions of Gemini 1.5 Pro and 1.5 Flash, for general availability.gemini-1.5-pro-latest model code to use gemini-1.5-pro-002
and the gemini-1.5-flash-latest model code to use gemini-1.5-flash-002.gemini-1.5-flash-8b-exp-0924 to replace gemini-1.5-flash-8b-exp-0827.frequencyPenalty and
presencePenalty.AI Studio updates:
API updates:
AI Studio updates:
AI Studio updates:
Model updates:
Model updates:
gemini-1.5-pro-exp-0827gemini-1.5-flash-exp-0827gemini-1.5-flash-8b-exp-0827API updates:
Model updates:
Model updates:
gemini-1.5-pro-exp-0801, a new experimental version of
Gemini 1.5 Pro.Model updates:
Model updates:
API updates:
API updates:
Model updates:
Model updates:
gemini-1.5-pro-001) is generally available (GA).gemini-1.5-flash-001) is generally available (GA).API updates:
Model updates:
gemini-1.5-flash-latest) in preview.Model updates:
gemini-1.5-pro-latest) in preview.text-embeddings-004, which supports
elastic embedding
sizes under 768.API updates:
response_mime_type configuration option, which lets
you request responses in
JSON format.Model updates:
Model updates:
See Gemini models for more details.
API version updates:
See the API versions topic for more details.
API updates:
GenerateContent is a single unified endpoint for chat and text.StreamGenerateContent method.Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2026-09-23 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-09-23 UTC."],[],[]]
You have the change ready and the tests are green. Now someone has to launch the app, find the right screen, and check the flow. Often, that someone is still you, even when an agent helped write the code.
You should be able to delegate that part too.
Junie /demo is a new mode in Junie CLI. Describe what you want to check, and Junie builds and launches your app, interacts with its UI, and records what happens. You get an HTML report, screenshots, and a video you can review or share.
The useful part is getting the routine clicking off your plate while keeping the result open to inspection. You decide whether the change is ready to ship.
Let’s use a small issue tracker as our example. You have added bulk status updates: select two issues, mark them “Done”, and see the counters change. You also want to check that the update survives a reload.
First time in this repository? Start Docker and ask Junie to set up /demo. It analyzes your project and proposes a build and launch plan. Once you confirm the plan, Junie fills in the configuration for you. Review the generated files, then run:
/demo
Choose the changes from your branch, session, working tree, or last commit. For a specific check, enter a request in the prompt field:
Reset the sample data. Select PB-101 and PB-102 and mark them Done.Check that Open drops from 3 to 1 and Done rises from 1 to 3.Reload the page and verify that both issues are still Done.
Review the prompt and let the agent work:
You can watch the live run as it moves through the UI and inspect what it actually does:
A request with an expected result gives the run a clear target. “Check the feature” leaves more room for interpretation than naming the action, the expected state, and the condition that should survive a reload.
A screen recording is much easier to review when you know what you are looking at. Each demo video starts with a slide introducing the demonstration. If the run covers several scenarios, each gets its own introductory slide. A final slide sums up the results.
A model helps prepare that structure. During post-processing, it examines the captured screenshots, identifies the scenarios, and writes the explanatory slides. These are added to the recording as the final video is assembled.
The video also has explanatory subtitles, which you can turn on or off in the player. Voice-over may follow in a future update.
The HTML report brings together the request, the result, the video, and the screenshots. You can inspect the steps that ran and see which checks passed, failed, or remained incomplete.
That is useful for a reviewer, a QA engineer, or a teammate asking how a feature works. We are also experimenting with this in Junie Live, our Slack agent, to answer suitable feature questions with a demonstration.
A diff explains the code change. A demo adds the behavior you can see: which screen opens, what changes after a click, and whether the flow reaches the expected result.
Inside JetBrains, we connected the demo agent to GitHub Actions. In our agent repository, we have run it for more than 1,500 unique PRs and created over 2,100 demo videos.
The first workflow example follows the same idea. It checks whether a PR contains behavior worth demonstrating, runs the demo when it does, and adds a comment linking to the available artifacts. The prompts are inside the YAML, so you can read and adapt the whole example in one file.
You can read the full demo-pr-changes.yml and adapt it to your repository.
This is most useful when a change has an interface to exercise. A backend change may also be demonstrated through an existing Swagger UI, for example. The value depends on what the run can actually observe.
We also use the demo agent for release smoke tests. Our internal workflow runs 22 scenarios on pushes to release branches and keeps a result and video for each. Across our internal release branches, we have used the agent for more than 1,300 smoke tests.
The second example starts small: two independent scenarios, triggered by a push or a manual run. Replace the prompts with your own steps and expected results. A commented schedule shows how to add regular runs.
You can read the full demo-release-tests.yml and replace the scenarios with your own.
There is one detail worth keeping: a completed agent process does not tell you whether a check passed. In this example, the prompt asks Junie to write an explicit verdict. Only PASS passes the result check. FAIL, PARTIAL, and missing or invalid results fail it. Other scenarios can still finish and upload their evidence.
Both examples use GitHub Artifacts, so there is no separate video hosting service to configure.
The demo environment is a Docker container based on Debian Bookworm. The base image includes Chromium, Node.js, xterm, a virtual desktop provided by Xvfb and a window manager, plus screenshot tools, xdotool, and ffmpeg.
A model with Computer Use support drives the app through clicks, keystrokes, and screenshots. Your Dockerfile adds the project’s dependencies; .junie/demo.md describes its build and launch steps.
A complex repository can have several VM templates. For a monorepo with a backend and several frontends, each environment can have its own Dockerfile under .junie/vms/ and its own launch settings. Describe which template to use, which services it needs, and how to start them in .junie/demo.md. Junie can then choose the right environment for the requested demo.
Junie keeps your active model if it supports Computer Use and is available. Otherwise, it selects the first available model in this order: GPT-5.6 SOL, GPT-6 Astra, GPT-5.5, then GPT-5.4. All models run with High reasoning effort in /demo, regardless of your selected effort level. The run cannot start without a supported model. The Junie /demo documentation covers the environment and configuration in detail.
In CI, the same mode is available through --demo:
junie --auth="$JUNIE_API_KEY" --demo -p . \ --task "Open the app and demonstrate the bulk status update."
In our internal 22-case comparison, GPT-5.6 SOL had the lowest average time and cost among the three models we measured.
The full set cost $19.94 on SOL. In the subscription conversion used for these figures, $1 equals one AI Credit. These are internal measurements on our scenarios, so your app, build steps, and prompts will affect the result. Budget for CI runner usage separately.
The team also found SOL faster in these runs without a noticeable drop in observed quality. That observation comes from our own workloads and helps explain the model preference.
A run still takes minutes. The benefit is that you can hand over the routine interaction and come back to something you can inspect.
Set up /demo once in your repository, check the generated configuration, and start with a small feature or fix. For CI, commit that configuration and add a JUNIE_API_KEY repository secret before copying either workflow.
Pick the change you were about to click through yourself. Ask Junie to demonstrate it, watch the output, and decide what needs a closer look.
Ktor 3.6.0 is here! This release is full of new experimental features, including typed authentication capabilities with specialized support for OpenID Connect and HTTP/3 support for the Netty engine. There are also a few quality-of-life improvements for routing and request handling, more convenient defaults for Kotlin Multiplatform clients, and more. Check out What’s new in Ktor 3.6.0 on our website for the full list of changes, or review the release notes.
Ready to explore Ktor 3.6.0? Start your next project with the interactive project generator at start.ktor.io. Your feedback and contributions are always welcome!
Until now, Ktor’s authentication has relied on implicit typing to bridge configuration to the routes. In this module, you get new types to guarantee full type safety when working with complex authentication. It also supports role-based access and anonymous users. By leveraging context parameters, we were able to ensure even more elegant syntax. Read the type-safe authentication documentation for setup, role checks, and failure handling.
val jwtAuth = jwt<User>("my-jwt") {
verifier(jwkProvider, issuer)
validate { credential ->
val payload = credential.payload
User(
id = payload.subject,
email = payload.getClaim("email").asString()
)
}
}
routing {
authenticateWith(jwtAuth) {
get("/profile") {
val user = call.principal
call.respondText(user)
}
}
}
The new OpenID Connect (Oidc) plugin aims to reduce complexity when securing your service through OpenID Connect Providers. The Oidc plugin allows you to create typed authentication providers that support all OpenID Connect features in a typed way. There is also support for sessions, a browser login interface with auto-refreshing tokens, and more. For the full documentation, check out the Ktor website – here.
suspend fun Application.module() {
val oidc = install(Oidc)
val auth0 = oidc.identityProvider("auth0") {
issuer = "https://my-tenant.auth0.com"
bearer {
audience = setOf("https://api.example.com")
}
}
routing {
authenticateWith(auth0.jwtBearer) {
get("/orders") {
val subject = call.principal.claims.subject
call.respondText("Hello $subject")
}
}
}
}
The Netty server engine now has experimental HTTP/3 support over QUIC. To enable it, configure an SSL connector, then opt in with enableHttp3 { }:
embeddedServer(Netty, environment, {
sslConnector(
keyStore = keyStore,
keyAlias = "server",
keyStorePassword = { "changeit".toCharArray() },
privateKeyPassword = { "changeit".toCharArray() }
) { port = 8443 }
enableHttp3 { quicMaxIdleTimeout = 30.seconds }
}) { /* application */ }.start(wait = true)
The enableHttp3 {} block also lets you tune QUIC-specific settings, such as flow-control limits and UDP socket configuration. It is still experimental, so we would love your feedback if you decide to try it.
A Netty server can now also serve h2c on one connector and HTTP/2 over TLS on another. Enable both with enableH2c = true and enableHttp2 = true.
embeddedServer(Netty, configure = {
connector { port = 8080 }
sslConnector(...) { port = 8443 }
enableHttp2 = true
enableH2c = true
}) { /* application */ }.start(wait = true)
Request-parameter conversion now supports Kotlin’s Uuid, Byte, and unsigned numeric types. ApplicationCall.receive() now also accepts nullable types, making the route contract explicit and deprecating receiveNullable().
put("/users/{id}") {
val id: Uuid by call.parameters
val preferences = call.receive<NotificationPreferences?>()
if (preferences == null) {
preferenceService.clear(id)
} else {
preferenceService.update(id, preferences)
}
call.respond(HttpStatusCode.NoContent)
}
We have also added respondHtmlPartial, which replaces the deprecated respondHtmlFragment. The new function uses TagConsumer<Appendable>, so it can respond with unrestricted partial HTML – with all elements supported by FlowContent.
get("/status") {
call.respondHtmlPartial(HttpStatusCode.OK) {
td { +"Ready" }
}
}
ContentNegotiationThe client ContentNegotiation plugin used to merge its registered content types into every Accept header. That is usually helpful, but not when an API expects the header you set on a request to remain exactly as it is.
With ContentTypeMergeStrategy.SkipIfPresent, an explicit Accept header wins. When a request has no Accept header, the plugin continues to add the registered content types as usual:
install(ContentNegotiation) {
register(ContentType.Application.Json, noOpJsonConverter)
acceptHeaderMergeStrategy = ContentTypeMergeStrategy.SkipIfPresent
}
Ktor 3.6.0 introduces ktor-client-engine-defaults: a curated set of client engines for Kotlin Multiplatform projects. Add it to commonMain, and create an HttpClient() without choosing an engine in shared code. Ktor selects the appropriate available engine for each target.
The HTTP cache has moved in the same direction. File-based cache storage now uses the Path of kotlinx-io, so persistent HttpCache storage is no longer limited to JVM java.io.File APIs. Together, these improvements make setting up a KMP client with a simple cache significantly simpler:
// build.gradle.kts
kotlin {
sourceSets {
commonMain {
dependencies {
api("io.ktor:ktor-client-engine-defaults:3.6.0")
}
}
}
}
// Main.kt
val client = HttpClient() {
install(HttpCache) {
publicStorage(FileStorage(Path("build/cache")))
}
}
This gives Ktor projects a more natural common-code setup while retaining the option to choose and configure a specific engine whenever a platform needs it.
For the full list of 3.6.0 changes, including WebRTC support for JVM, asynchronous DNS resolution for CIO, OpenAPI tag descriptions, duplicate-cookie parsing, and JavaScript fetch() overrides, see What’s New in Ktor 3.6.0.
Thank you to everyone in the community for the feedback, issue reports, and contributions that help make every Ktor release better. A special thank-you to the external contributors whose work is included in the release: kdelay, Rafa Ruiz, and solo.
Start building your next project at start.ktor.io. Your suggestions and contributions are always welcome!
👉 Get Started With Ktor | 💬 Join the community on Slack
Last week, I spent three days in the Netherlands and gave two talks at two conferences: a lightning talk at PGDay Lowlands in Utrecht on Thursday, September 10, and a session at Percona Live in Amsterdam on Friday, September 11. In this blog post, I’m going to share my notes from both.
As often happens with conferences (or any big events, really), there was a minor hurdle to overcome before we could get there. On Wednesday, September 9, just one day before PGDay Lowlands, a nationwide 24-hour public transport strike stopped trains, buses, trams and metros across the whole country. Not the ideal warm-up for a conference that draws people from all over the world, but by Thursday morning everything was moving again and the day went ahead as planned. Yay!
PGDay Lowlands is a one-day Dutch PostgreSQL conference (although all the talks are in English), organized by PostgreSQL Europe. This was its third edition, and the event moves around: last year, it was held at Blijdorp Zoo in Rotterdam; this year, it took place at TivoliVredenburg, a music venue in the center of Utrecht, with the main track in a hall called Cloud Nine.
Last year I gave a full 45-minute talk, my now famous Anatomy of Table-Level Locks in PostgreSQL (the recording is on YouTube). This year, I went for the other end of the spectrum: a five-minute lightning talk. It’s the format I struggle with the most, but I tried anyway.
Five minutes is not a lot of time, so I kept it to two open-source extensions (pg_clickhouse and pg_stat_ch) we maintain at ClickHouse, both Apache 2.0 licensed. The idea behind both is that you keep Postgres as your front door and your system of record, and let ClickHouse do the analytical heavy lifting behind it.
On a personal note, this was my first talk as a new ClickHouse employee 😀 Photo credit: Tom
pg_clickhouse is a foreign data wrapper. You CREATE SERVER pointing at ClickHouse, add a USER MAPPING with the credentials, and IMPORT FOREIGN SCHEMA: the ClickHouse tables show up as foreign tables in a Postgres schema of your choice, with the same column names and ClickHouse types mapped to Postgres types. Change search_path to that schema and existing read queries, ORMs and dashboards run unmodified. Where the query is pushable, the Postgres planner sends the whole thing to ClickHouse as ClickHouse SQL and gets back the aggregated result; otherwise it pushes down what it can and finishes the rest locally.
The point of pg_clickhouse is simple: moving data to ClickHouse is easy, but rewriting years’ worth of dashboard and ORM-generated SQL is hard. The extension lets existing PostgreSQL queries run against ClickHouse, so improving query pushdown is the top roadmap priority. Today, 15 of the 22 TPC-H queries at scale factor 1 are fully pushed down.
The main slide from the lightning talk: Analytics Without Leaving Postgres
pg_stat_ch goes in the opposite direction. Postgres hooks capture every query execution as a raw event (timing, buffers, WAL, CPU, errors, application, client), write it into a shared-memory ring buffer, and a background worker drains batches to ClickHouse over the native protocol, where the aggregation happens. It uses the same query_id as pg_stat_statements, so the two correlate, but you get per-query history you can slice by time and application, with real percentiles and error tracking. pg_stat_statements cannot give you that because it only keeps cumulative counters. There is no back-pressure by design: if ClickHouse is slow or unreachable, events are dropped and counted, and Postgres never waits.
Lightning talks are so much fun to watch, so I stayed for the whole block.
In the audience during the lightning talks. Look how happy I am 😀 Photo credit: Tom
Cornelia Biacsics opened with My Lightning Talk Disaster, looking back on her first speaking experience exactly one year later. It was also a reminder that the five-minute format is sold as the easy way in for new speakers, but is not risk-free, especially for introverts. Speaking as an extrovert, I can confirm that it is THE hardest format for me too, as I mentioned above. Ellert van Koperen showed a real-life case where partitioning, the default answer to "the table keeps growing", had a knock-on effect with serious consequences, and the simple fix that resolved it. Jan Wieremjewicz gave a status update on pg_tde, what works today, what is still open, and how to get involved. And Dave Pitts closed the block with something completely different: the story behind the PGDay Lowlands conference songs, produced with digital instruments and an actual piano keyboard rather than generated by AI. Yes, this conference has its own soundtrack!
The whole day was live streamed and recorded, and the individual talks will be available to watch later.
Before lunch I attended Michael Banck's talk, Optimizer Hints in PostgreSQL, and I liked it a lot. Postgres has famously refused to add optimizer hints for decades, on the grounds that planner problems are bugs to fix. Michael walked through what you can do today: the enable_* parameters (reworked in PostgreSQL 18 so disabled node types are counted rather than penalized with a huge cost) and pg_hint_plan with its /*+ ... */ comments and hints table keyed by query ID.
The part I found most interesting was the two new PostgreSQL 19 contrib modules by Robert Haas, pg_plan_advice and pg_stash_advice. They are aimed at plan stabilization rather than hints in the classic sense.
EXPLAIN (PLAN_ADVICE) prints a compact "advice string" describing the plan you got (join order, join methods, scan methods, parallelism). You can feed that string back via pg_plan_advice.advice to pin the plan, and pg_stash_advice stores advice per query ID in shared memory, so it is applied automatically and survives reconnects and restarts.
The implementation works by constraining the planner rather than replacing it, so you can only ever get a plan that the planner would have considered anyway. Michael's argument was that plan flips are the real problem, and stable plans are often worth a little lost performance. His slides are worth a read.
From Utrecht, I went straight to Amsterdam for the Percona Live speaker dinner on Thursday evening. It was a nice way to arrive at a conference (I was attending for the first time): meet the other speakers over dinner first, then show up the next morning already knowing a few faces.
Percona Live speaker dinner at De Bekeerde Suster—spot me listening carefully to Alastair Turner 🙂
Percona Live 2026 ran from September 9 to 11 at the Mövenpick Hotel Amsterdam City Centre. It is a multi-database conference, with MySQL, PostgreSQL, MongoDB, and Valkey tracks side by side, which makes for a broader audience than at a PGDay. I was only there for the final day.
The final morning opened with a fireside chat called The Columnstore Revolution, moderated by Percona founder Peter Zaitsev, with Alexey Milovidov, CTO of ClickHouse, and Hannes Mühleisen, co-founder of DuckDB, discussing the resurgence of column-oriented databases and what it means for modern data workloads.
I didn’t know that Alexey Milovidov, our CTO, would be there until Peter Zaitsev told me at the speaker dinner, so that was a nice surprise too.
My session was a 30-minute version of the talk I'll give at PGConf.EU in October.
I grouped the changes into five parts:
log_lock_waits is now on by default, log_min_messages accepts different log levels for each process type, autoanalyze logging is split from autovacuum with log_autoanalyze_min_duration, and messages from remote servers, through replication, postgres_fdw, or dblink are now formatted like local ones.wal_fpi_bytes counter appears in pg_stat_wal, per-backend statistics, VACUUM and ANALYZE log lines, and EXPLAIN (ANALYZE, WAL). COPY TO / FROM files, pipes and programs now has its own wait events.WAIT FOR: a new command for read-your-writes semantics on asynchronous standbys, with wait events for the written, flushed, and replayed stages of WAL.pg_stat_lock, pg_stat_recovery, and pg_stat_autovacuum_scores.pg_get_multixact_stats(), and the XID wraparound warning threshold moving from 40 million to 100 million transactions.I closed with what is already committed for PostgreSQL 20 (pg_stat_get_backend_lock(), which gives you pg_stat_lock per backend). I also covered wait-event statistics, where the discussion on the hackers list keeps moving towards sampling rather than counters.
If you want the long version, I have already written most of it in three posts: What's New with Monitoring in PostgreSQL 19, Read your writes: WAIT FOR in PostgreSQL 19 and New system views in PostgreSQL 19.
Both events will be back in 2027 with dates and locations to follow.
Thanks to the people who made PGDay Lowlands happen: Floor Drees, Derk van Veen, Teresa Lopes, Boriss Mejías, Sarah Conway, Stacy Raspopina, Jos van Schouten, Chelsea Dole, Stefan Fercot and Ellert van Koperen. Thanks also to Peter Zaitsev, Alastair Turner, Jan Wieremjewicz and Kai Wagner from the Percona team for having me, and to everyone who came to my talks. See you in Valencia!
Installed latest packages from upstream dependencies.
Installed latest packages from upstream dependencies.
JupyterLab now forwards client-side logs (console errors, uncaught exceptions, unhandled promise rejections, and failed network requests) to the instance backend, where they surface in Cloud Logging for easier debugging.
JupyterLab now forwards client-side logs (console errors, uncaught exceptions, unhandled promise rejections, and failed network requests) to the instance backend, where they surface in Cloud Logging for easier debugging.
Installed latest packages from upstream dependencies.
JupyterLab now forwards client-side logs (console errors, uncaught exceptions, unhandled promise rejections, and failed network requests) to the instance backend, where they surface in Cloud Logging for easier debugging.
Managed workload identity for backend mTLS is generally available for the following Application Load Balancers:
The key benefits are as follows:
Streamline certificate management: Automated certificate and trust management for backend mTLS through seamless integration with Certificate Authority Service and Certificate Manager.
Eliminate operational toil: Certificates are automatically rotated based on the workload identity pool's configuration, removing the complexity and manual bottleneck of private key provisioning and maintenance.
Improve visibility and governance: Gain visibility into communication between distributed services and proactively apply governance to workloads across environments.
For more information, see Backend mTLS with managed workload identity overview
Cloud Scheduler is available in the following location:
asia-southeast3 (Bangkok, Thailand)Config Controller now uses the following versions of its included products:
Gemini Enterprise: Support for new actions (Public Preview)
Support for new actions is available in Public Preview for the following data stores:
For more information, see Connect a third-party data source.
xAI's Grok 4.6 is generally available
Grok 4.6 is now generally available (GA) and available for production use on the global endpoint and the US multi-region endpoint.
Agent Platform SDK for Python version 2.0.1 is available
Version 2.0.1 of the Agent Platform SDK for Python (google-cloud-agentplatform) is now available. This release migrates generative AI modules to the Google Gen AI SDK, decouples the agent surface from google-cloud-aiplatform into a dedicated package, and introduces restructured namespaces.
For details and migration instructions from google-cloud-aiplatform, see the Agent Platform SDK for Python version 2.0.1 migration guide.
Google Cloud CCaaS 6.13
We've released version 6.13 of Google Cloud CCaaS.
The timing of the update to your instance depends on the deployment schedule that you have chosen. For more information, see Deployment schedules.
This release addresses the following issues:
Fixed an issue where session metadata and data feed files were missing from external storage for chats that ended before the first message from the end-user.
Fixed an issue with Kustomer integrations where the caller's information didn't appear on the Incoming call page of the call adapter for direct-line inbound calls.
Fixed an issue with inbound mobile calls where the end-user leg of the call
failed, returning Unknown error, while the agent leg connected normally.
Fixed an agent desktop issue where live call and chat data were lost.
Fixed an issue that occurred when the receiving agent in an agent-to-agent transfer didn't answer the call. The receiving agent was marked as active on the call indefinitely, even after the call ended.
Fixed an issue where the Dismiss button remained active after an agent sent a message, resulting in a 409 error when clicked.
Fixed an issue where duplicate "chat finished" events were reported when the end-user left a chat session at nearly the same time that the agent ended the chat session.
Fixed an issue where deflected calls were missing from the All Call History and Voice Inbound (IVR) History reports.
Fixed an issue that occurred when a direct inbound call was deflected to the agent's overcapacity queue, then that queue redirected to a SIP URI. The SIP redirect didn't include the custom SIP headers.
Fixed an issue where an in-queue announcement interval of several minutes for inbound IVR calls was incorrectly reduced to approximately 60 seconds.
Fixed an issue where calls that agents were unable to answer due to microphone failures were incorrectly reported as "picked up" in the Agent Activity Timeline report.
Fixed an issue where the system incorrectly marked agents as still being on a call after it ended, which either prevented them from changing their status to Available or silently blocked them from receiving new calls.
Fixed an issue where processing delays for ended calls caused timeout errors.
Fixed an issue where a sudden spike in calls bypassed capacity limits, causing agent availability to drop below required minimums.
Fixed an issue where the Agent Activity Timeline report incorrectly attributed manual agent logins and logouts to System instead of the appropriate agents.
Fixed an issue where calls with a missed offer became permanently stuck in the queue, preventing them from being routed to other available agents. This occurred with queues configured with multicast fallback disabled.
Fixed an issue where manual or cascade outbound calls that were canceled before connecting were missing from team-filtered Call History reports.
Fixed an issue that prevented over-capacity deflection from triggering when an agent warm-transferred an outbound call to a queue.
Fixed an issue where calls weren't correctly routed to the top-ranked agent when using agent priority overrides.
Fixed an issue where escalated voice calls were incorrectly reported as both answered and abandoned.
Fixed an issue where calls were missing from the All Call History and Voice Inbound History reports if the caller hung up before leaving a voicemail.
Fixed an issue where Salesforce click-to-dial outbound calls were incorrectly associated with the most recent open case instead of the case from which the call was initiated.
Fixed an issue where email accounts remained disconnected indefinitely after a temporary authentication failure.
Fixed an issue in Agent Assist where long periods of silence during calls caused connection timeouts, triggering false-positive error alerts.
Fixed an issue where the arrow-down-icon and arrow-up-icon arrows on the Agents > Filter Settings page were rendered at an incorrect scale.
Fixed an issue where incoming calls incorrectly created duplicate Salesforce accounts instead of linking to existing accounts.
Fixed an issue where the outbound call queue list displayed stale information, potentially causing calls to be placed in a queue that didn't match the agent's selected language.
Fixed an issue where the menus for transferring calls and forwarding calls to voicemail appeared in English instead of the agent's selected language.
Fixed an issue where the wrap-up disposition panel froze after a network reconnection even though the submission had completed successfully.
Fixed an issue where outbound, click-to-dial calls initiated in Salesforce incorrectly linked to and reassigned ownership of other cases associated with the same phone number.
Fixed an issue where the agent adapter went blank and prevented new calls from reaching the agent if an end-user hung up immediately after the agent received the call notification.
Fixed an issue where calls that failed to connect got stuck in a silent 'connecting' state in the call adapter.
Fixed an issue where Salesforce CRM connections dropped for organizations enforcing OAuth Refresh Token Rotation.
Fixed an issue where part of an agent's audio was dropped from recordings when a virtual task assistant ran in the middle of a call.
Fixed a web SDK issue where menus in the pre-chat and chat screens didn't comply with WAI-ARIA keyboard navigation standards.
Fixed a web SDK issue where screen readers couldn't identify the purpose of the Text size options for the chat screen.
Advanced reporting dashboards 6.4
We've released version 6.4 of the advanced reporting dashboards.
Real-time Agent Monitoring dashboard: new Active call ID(s) column
The Real-time Agent Monitoring dashboard now has an Active Call ID(s) column in the Live Agent Data table. The column displays the call ID(s) for any call in a connecting, connected, or reconnecting state for the agent. If an agent is handling multiple concurrent calls, the call IDs appear in a comma-separated list. The Active Call ID(s) column reduces the number of steps required for supervisors to identify active calls during live monitoring.
Improved filtering by team
We made the following changes to team-based filtering:
Renamed the Teams filter to Agent Teams to clarify that it filters by the agent team handling the interactions. This change is in the Real-time Queue Monitoring - Calls, Real-time Queue Monitoring - Chats, Real-time Connected - Calls, and Real-time Connected - Chats dashboards. For more information, see Queue monitoring dashboards, Real-time Connected - Calls dashboard, and Real-time Connected - Chats dashboard.
Added a Queue Teams filter to the Real-time Queued - Calls and Real-time Queued - Chats dashboards. This lets you filter queued interactions by the team assigned to the queue.
Improved the Real-time Calls and Real-time Chats dashboards
We made the following dashboard improvements:
Real-time Calls - Calls Connected dashboard. Added the following columns to the Connected Calls table:
Total Consumer Talk Time. Total time since the call first connected to a virtual agent or a human agent.
Total Hold Time. Total time the call has spent on hold so far, including a hold currently in progress.
Real-time Chats - Chats Connected dashboard. Added the following column to the Connected Chats table:
Real-time Calls - Calls Queued dashboard: new Projecting column
The Real-time Calls - Calls Queued dashboard has a new Projecting column in the Call Queued table. Indicates whether the routing engine (deltacast) is currently projecting this queued call to an available agent.
Advanced reporting available in French Canadian
All advanced reporting dashboards and Explores are now available in French Canadian. When you select French Canadian as your profile language in the CCAI Platform portal, these dashboards and Explores display in that language.
Administrators: There's a new Français (CAN) option when you click Admin > Change Language in the CCAI Platform portal.
This release addresses the following issues:
Fixed an issue where the formatting of numeric values was inconsistent across tiles.
Fixed an issue where column headers, filter labels, and tile titles didn't immediately switch to a newly selected language.
Fixed an issue where the Productive Agents column in the tables of the Queue Group Performance - All dashboard didn't display values appropriate to the queue group settings.
Fixed an issue in the Call Queue Metrics (Historical) Explore where filtering by Agent Name without including it as a visible column resulted in zero rows being returned.
Fixed an issue that affected calls to a sub-menu that were deflected using Custom After Hours Deflection to a message. These calls were incorrectly attributed to the parent menu in the All Queued Interactions report.
Fixed the effectiveness of the Direction filter in the following dashboards:
Agent Performance. The Agent Productivity Detailed – Calls and Agent Productivity Detailed – Chats tables correctly reflect the filter setting.
Real-time Agent Monitoring. The Agent Performance table and historical metrics tiles correctly reflect the filter setting.
All Interactions – Calls and All Interactions – Chats. The IVR Interactions (calls only) and Virtual Agent Interactions tables correctly reflect the filter setting.
Fixed an issue with the Queue Performance - Calls dashboard when short abandons were present in the specified date range. The Avg Queue Time column in the Queue Summary table incorrectly displayed the raw sum of queue durations instead of a true average.
Fixed an issue where team filters didn't apply correctly when generating the Individual Call History Report and the Individual Chat History Report. This resulted in the inclusion of data from unmanaged queues.
Fixed an issue where French Canadian translations for several dashboard metrics and labels were incorrect, incomplete, or missing.
Fixed the following issues with the Real-time Calls - Calls Queued dashboard:
The Total Queued Now metric didn't include callers who were returned to the queue after an automated-answer detection miss.
The Current Max Queue Wait Time (H:M:S) and Current Avg Queue Wait Time (H:M:S) metrics mistakenly measured from a caller's original entry into the queue, rather than from their most recent return to the queue.
Fixed an issue where a gray bar appeared at the bottom of the advanced reporting dashboards, preventing a full view of the dashboards.
Resizable side panels in the Investigation Management experience
You can now dynamically resize the Case preview and Alert and detection preview side panels in the revamped Investigation Management experience in Google SecOps. You can adjust the panel width using your mouse or keyboard shortcuts to view detailed telemetry, parsed UDM records, and raw logs without navigating away from your main case queue.
To explore the complete triage workflow, see Investigation and case management overview.
The basic token-based authentication feature is generally available.
Filter version v4 is available and set as the default for the Latest alias.
Filter version v3 is promoted to the Stable alias in all supported regions
except the following:
asia-northeast3, v1 remains the Stable version.australia-southeast2, v3 becomes the Stable version on
September 25, 2026.If your templates use the Stable alias, they automatically upgrade to v3
when v3 becomes Stable in that region.
Filter versions v1 (except in asia-northeast3, and starting
September 25, 2026 in australia-southeast2) and v2 transition to Legacy
status and retire on December 17, 2026. If your templates are explicitly
configured with v1 or v2 in regions where those versions are in Legacy
status, you must migrate them to v3 or the Stable alias before December 17,
2026.
For more information, see Version release timeline and Model Armor filter version history.
Spanner supports automatic parameterization of SQL query literals to improve query performance, reduce latency, and lower CPU costs. Spanner converts literal values hardcoded in CRUD-style queries, such as primary key lookups, index lookups, and primary key joins, into query parameters, allowing execution plans to be cached and reused to reduce latency and CPU costs.
For more information, see Use query parameters and statement hints (GoogleSQL and PostgreSQL).
GLM 5.3 FlashX is now available on AI Gateway.
GLM 5.3 FlashX is a high-speed serving option for Z.ai's multimodal coding model, delivering inference at ~200 tokens per second for faster streamed responses.
The higher serving speed is useful for coding agents, tool loops, and interactive applications where users wait on generated output.
Use zai/glm-5.3-flashx across API formats and in coding agents:
To use it in a coding agent, see the coding agents guide, then run vercel ai-gateway setup to create a key and configure your supported agents. Select zai/glm-5.3-flashx inside the agent.
AI Gateway provides a unified API for calling models, tracking usage and cost, and configuring retries, failover, and performance optimizations for higher-than-provider uptime. It includes built-in custom reporting, budgets for API keys, routing rules, and more.
AI Gateway reflects provider pricing with no markup and does not charge a platform fee on inference, including on Bring Your Own Key (BYOK) requests.
Try GLM-5.3-FlashX in the model playground, or view all language models available on AI Gateway.
On September 17, 2026, GitLab 19.4 was released with the following features.
Jimmy contributed across the GitLab codebase, client-go, and the Terraform
provider to ensure that tokens, service accounts, and push mirrors can be
managed end to end through infrastructure as code.
Previously, you could only apply AI agent tool governance rules to internal GitLab Duo Agent Platform tools. Tools available to both GitLab Duo Agent Platform and third-party agents through the GitLab MCP server followed fixed rules that could not be changed.
You can now govern GitLab MCP server tools from the same place as internal GitLab Duo Agent Platform tools. They appear alongside internal tools in your group and project GitLab Duo settings, where you can set a mode for each tool:
You can now restrict access to MCP (Model Context Protocol) servers by allowing or denying access to:
This feature gives you assurance that AI agents within Duo Agent Platform are operating within governed boundaries and can only access MCP tools that are within their scope to perform their activities, sessions, and tasks.
These controls apply consistently wherever AI agents run, including:
This feature is currently in beta and we welcome your feedback in issue #628378.
Use the Vulnerability Context Flow to produce context to triage vulnerabilities more efficiently and intelligently.
The flow produces context in the following three categories:
Advanced SAST now scans Kotlin, Dart, and Scala codebases with the same deep taint analysis that covers Java, Python, and other supported languages, all delivered through the Software Factory architecture with per-language front-ends and framework-aware rule gating.
All three additions are verified using deliberately vulnerable real-code repositories, with findings reported as code flows from source to sink.
GitLab license data now carries SPDX license expressions, including compound declarations
such as MIT OR Apache-2.0 or GPL-2.0-only WITH Classpath-exception-2.0.
Previously these were reported as unknown in the dependency list and were invisible to
license approval policies.
Composite licenses now appear in the dependency list with their operator (AND, OR,
WITH), and license approval policies can allow or deny them the same way they handle
single-license dependencies.
Expressions declared in a CycloneDX SBOM have been supported since GitLab 19.3. This release adds them to the license data GitLab synchronizes. Offline instances receive expressions only after downloading the v3 license data.
GitLab Duo CLI now includes a /goal slash command that delegates open-ended objectives to a
governed, goal-driven flow that runs locally.
You describe a goal and GitLab Duo handles implementation and verification, using an independent judge to decide when you have achieved your goal or reached the iteration limit. You stay in control the whole time: pause, update the goal, or redirect the agent at any time.
The /goal slash command requires GitLab 19.3 and later, and GitLab Duo CLI 9.17.0 and later.
To get started, run /goal <task>.
For example:
/goal Fix the failing tests in spec/models/user_spec.rbYou can now invoke GitLab Duo agent flows directly from Slack, without switching to the GitLab UI.
With the GitLab Duo Slack integration, you can mention GitLab with @GitLab in any Slack channel or thread. Mention GitLab to trigger agent flows, get answers from your codebase, and create GitLab issues from conversations. GitLab Duo streams its progress back into the Slack thread in real time, and includes thumbs-up and thumbs-down feedback buttons so you can rate responses without leaving Slack.
This integration is available as an experiment. To share your feedback, add a comment to issue 624364.
Build custom flows for your GitLab projects with the GitLab flow builder, a new visual editor for AI-native workflows in the GitLab for VS Code extension. Compose a flow visually from components (Agent, Custom tool, and AI task), or edit the underlying YAML directly.
To start, open your flow’s YAML file in VS Code and select Open GitLab Flow Builder. Test your flow with the Run button, which opens an execution console. When your flow is ready, select Publish to publish it to the AI Catalog.
The flow builder is available as a beta feature in GitLab for VS Code 6.87.0 and later. To get started, enable the gitlab.featureFlags.flowBuilder setting in VS Code.
New CI/CD tools let agents trigger, inspect, and control CI/CD from any MCP client:
save_pipeline runs, retries, or cancels a pipeline without switching tools.get_job returns job metadata together with the job trace, so an agent can
read the log of a failed build and diagnose the problem on its own.Previously, agents had no way to trigger or inspect pipelines through MCP.
Merge request tools let agents run the full merge request loop through the GitLab MCP server:
save_merge_request opens and updates an MR.get_merge_request inspects an MR in depth, with new diffs, conflicts, and
approvals facets.list_merge_requests now works at group scope.save_merge_request_review leaves line-level review comments, with batched
diff comments and a summary in a single call.accept_merge_request merges an MR once checks pass, and can also approve
or unapprove it.New project and user tools give agents the context they need to target work correctly through the GitLab MCP server:
get_project and list_projects find and read project details.list_project_members enumerates members and their roles.get_user looks up user details for assignment and mentions.Previously, agents had no way to discover project membership or user information through the GitLab MCP server.
New repository tools let agents browse a project’s structure, read its commit history, and propose changes through the GitLab MCP server:
list_repository_tree explores the file tree.list_branches and list_tags enumerate refs.list_releases inspects published releases.get_commit retrieves a commit’s metadata, diff, or notes.list_commits pages through a branch’s history.add_commit commits one or more file actions in a single call, optionally to a
new branch from a specific starting ref or source project.fork_repository forks a project, so an agent can go from exploring an
upstream repository to proposing changes without leaving its client.semantic_code_search is now semantic_search. The tool finds code by meaning
rather than by exact symbol or filename, which is unchanged from earlier
releases. The rename adds a scope parameter so that additional indexed content
types can fold into the same tool in future releases. Today scope accepts
code only.
The GitLab MCP server now exposes work item tools, so agents and MCP clients can search, read, create, and update issues, epics, tasks, incidents, objectives, and key results.
Use get_work_item to read a single item in depth, list_work_items to search across a group or project, and save_work_item to create or update any work item type.
Because issues and epics are work item types, get_work_item and save_work_item cover what get_issue and create_issue do today.
save_note lets an agent comment on a work item or merge request and reply inside an existing discussion thread. The introduction of this tool renames existing create_merge_request_note and create_workitem_note.
In previous versions of GitLab, the Merge request trigger event type only supported the Approved, Marked ready, and Merge conflict actions. You had no way to run a flow or external agent the moment someone opened a merge request without using a tool outside GitLab.
You can now select Created as a trigger action. When someone opens a merge request in draft or ready state, and GitLab generates the diff, your flow or external agent runs. Use this for a first-pass review, or to add context from related issues.
To configure this trigger, go to AI > Triggers in your project, or select it when you enable a flow.
Finding the details that matter about an agent session used to mean hunting through a cluttered panel. Now, the session details panel surfaces what you need at a glance: status, timestamps, and the triggering user appear in an overview bar, while the right rail organizes identity, execution, and supplemental details into clearly labeled groups.
A new Linked items section separates what started the session from what it produced, including merge requests, work items, jobs, and comments. In the GitLab Duo side panel, session details now live in a collapsible bar pinned to the bottom, so they stay accessible without getting in your way.
The GitLab Duo Agent Platform now supports independent model selection for the Developer Flow. As an administrator, you can select a specific AI model for the Developer Flow separately from other GitLab Duo Agent Platform features, giving teams greater control over model selection.
The GitLab Duo Agent Platform now supports three open-weight models: GLM 5.3, Kimi K3, and MiniMax M3.
In GitLab Duo Agentic Chat, you can select any of these models for your own conversations. Users with the Owner role for a group and administrators can also set them as the default for Agentic Chat and for other agents, flows, and features.
In previous versions of GitLab, the only way to stop a trigger from automatically starting a flow was to delete it entirely. Deleting a trigger meant losing any complex filter configuration you had set up.
Now you can turn a flow trigger off and retain its configuration. Use the new toggle to turn it back on at any time.
To manage triggers, go to AI > Triggers.
In previous versions of GitLab, you turned on SAST false positive detection, GitLab Duo Vulnerability Resolution, secret detection false positive detection, and dependency scanning auto-remediation for each project individually. Now you can apply an Automated Triage and Remediation profile to a group or project, setting severities and run modes in one action. Start with a preset, or configure each flow yourself:
Profiles are available only with the GraphQL API, and require GitLab Duo Agent Platform with foundational flows turned on for the top-level group. Most flows consume GitLab Credits.
When a file is locked, you now see who locked it and what your options are, without leaving the blob viewer.
Previously, only a Locked label appeared, with no way to tell who locked the file or whether you could unlock it yourself. Now, a popover next to the label shows who locked it. If you have permission to unlock the file, the popover includes an unlock action. If you don’t, it explains why. For locked directories, the popover links you directly to the specific file that’s blocking your changes.
Security teams can use the bulkSetVulnerabilityFindingsDueDates
GraphQL mutation to assign, update, or remove due dates for
vulnerability findings in bulk. Each request supports up to
1,000 finding UUIDs and returns counts for assigned,
removed, and skipped updates, along with structured
errors. Teams can use this information to
connect vulnerability remediation
timelines with existing service-level agreement (SLA) and workflow automation.
When you apply filters to the Vulnerability Report, exported CSV reports will respect those filters. Rows that are not included in the Vulnerability Report UI after filtering will not appear in the CSV file export either.
You can now view scanner coverage for an entire group hierarchy from one page. In previous versions of GitLab, the Security Inventory showed coverage per subgroup, but no total for the entire group. A coverage widget now aggregates scanner coverage across every project in the group and its subgroups, and shows the percentage and number of projects where each scanner is enabled, not enabled, failing, or stale. To focus on one scanner, such as SAST or Dependency Scanning, use the scanner dropdown list. Then select a status to filter the project list, and turn on scanners for the projects that aren’t covered.
The Security Inventory also now lets you control which columns are shown. To show or hide the Vulnerabilities, Tool coverage, and Security attributes columns, select Display.
When secret detection finds a leaked GitLab personal access token in a public project, automatic response revokes it. In GitLab versions earlier than 19.4, revocation used only one detection rule and revoked only the legacy token format. Tokens created on GitLab 18.3 and later use the routable or versioned routable format. GitLab detected and reported these tokens without revoking them.
In GitLab 19.4 and later, revocation recognizes all three GitLab personal access token detection rules:
gitlab_personal_access_tokengitlab_personal_access_token_routablegitlab_personal_access_token_routable_versionedRevocation also covers findings from GitLab Secret Scanning for Source Code and the Gitleaks-based analyzer. You do not need to change any configuration. Instances with automatic response enabled get this wider coverage immediately.
Previously, pasting a copied table into a table cell always merged the copied cells into the existing table, which made it difficult to create a nested table.
Now you can choose how a pasted table behaves:
You can also use a keyboard shortcut to paste a table into a cell as a nested table: Command+Option+V on macOS, or Control+Alt+V on Windows and Linux.
Standard paste with Control+V or Command+V works as it did before.
In previous versions of GitLab, Dependency Scanning only surfaced packages with known CVEs. Malicious packages, those crafted to harm through typosquatting, compromised maintainer accounts, or embedded malware, produced no findings.
GitLab 19.4 introduces malicious package detection in beta. Dependency Scanning now checks
your dependencies against GitLab malware advisories,
so threats can surface before they are widely known. Findings appear in your Dependency List
and Vulnerability Report with a red Malware badge, always Critical severity, identified
by a GLAM- ID, not a CVE.
You can also block malicious packages before they merge, using the malware rule in merge request approval policies.
You don’t need any additional setup. Coverage applies to the supported package types: npm, PyPI, Maven, Go, NuGet, Cargo, and RubyGems. The same advisories power continuous vulnerability scanning, and offline instances download them manually.
Share feedback on issue 606036.
Security dashboards are now available at the organization level.
Users can see the following data across all top-level groups in an organization:
In GitLab 19.4, the GitLab MCP server provides the following new tools for vulnerability management:
list_vulnerabilities, which lists security vulnerabilities in a GitLab project with optional filtering
by severity and report type, with cursor pagination.get_vulnerability, which fetches full details for a single vulnerability by numeric ID, converting
it to the gid://gitlab/Vulnerability/<id> global ID format.save_vulnerability, which covers five write operations on GitLab vulnerabilities in a single
consolidated tool:These new vulnerability management tools allow AI agents to run vulnerability triage and remediation actions through the GitLab MCP server.
GitLab 19.3 introduced email notifications for reservation thresholds and for the moment a capped capability is cut off. The spend cap itself had no early warning, so the first email about a cap arrived when usage had already stopped.
GitLab now emails billing account managers when a capability’s on-demand usage reaches 50% or 80% of its monthly spend cap, naming the capability and the cap in credits. Only the highest threshold crossed is sent, at most once per capability per billing period. Caps of less than $10 are skipped, so a small cap does not generate noise.
Geo SSH proxying enabled by default
The following feature flags are enabled by default in GitLab 19.4:
geo_proxy_fetch_ssh_to_primarygeo_proxy_push_ssh_to_primaryGeo SSH proxying provides a more reliable path for SSH fetches and pushes to a Geo secondary site when the operation is proxied to the primary site. It also resolves long-standing bugs where proxied operations failed, such as pushes with push options and fetches from large repositories.
Action required for Cloud Native GitLab deployments
Cloud Native GitLab deployments using the bundled NGINX Ingress must either:
Otherwise, SSH fetches and pushes through Geo secondaries may hang or time out.
See the Geo troubleshooting documentation for SSH proxying for more information.
When a subscription had temporary evaluation credits, all usage drew from that shared pool first. Every user’s included monthly credits sat idle until the evaluation pool ran out, and then reset at the end of the month.
GitLab now consumes each user’s included credits first, and draws from the shared pool of temporary evaluation credits only after a user has used their included amount. The Monthly Commitment Pool, One-Time Charge credits, and On-Demand credits are consumed in the same order as before, so your bill is unaffected.
The credit usage export gave you one row per day, which told you how much a subscription spent but not what it spent on. Attributing credits to a team, a project, or a single automation meant guesswork.
The export now returns a ZIP file with two CSV files: the daily summary you already had, and a per-event file with one row for each billable event. Each row includes the product, flow type, session, user, namespace, project, credits used, and token counts. Exports run in the background, and GitLab emails you a download link when the file is ready.
Credit caps limit how many GitLab Credits each user can consume, but until now you could only configure them through the GraphQL API. Setting a different cap for a handful of users meant writing mutations by hand.
The new Credit caps page lets you set the flat cap that applies to every user by default, and add per-user overrides for individual users through a searchable picker. This page is available in GitLab Credits for group Owners on GitLab.com and administrators on GitLab Self-Managed. The GraphQL mutations still work if you prefer to script cap changes.
pnpm 12.5 expands Python support with editable project packages, shared workspace
environments, automatic interpreter downloads, and lockfiles for multiple
platforms and Python versions. It also accepts Package URLs in pnpm add, adds
machine-wide task concurrency groups, and cleans up obsolete registry metadata
with pnpm cache prune.
pnpm install chooses an interpreter that satisfies
each project's requires-python, preferring .python-version when present.
Different projects can use different interpreters. Set python.executable to
choose one interpreter for every project.
When no installed interpreter fits, pnpm downloads a shared
python-build-standalone
interpreter and reuses it on later installs. runtimeOnFail
controls this behavior: download permits downloads, error fails, and warn
or ignore use an available interpreter despite the version mismatch.
Environments now live under python-envs in the pnpm store. Each project keeps
its .venv link, which the next install migrates from the old project-local
layout. Old .pnpm/python-envs directories remain until you delete them after
running programs stop using them. With frozenStore, environments remain local.
Wheel imports use packageImportMethod.
Choose clone-or-copy or copy when installed files may be modified. Isolated
build environments use clones or copies to keep backend writes private.
pnpm now installs a Python project's own package
editable when it declares [build-system], so its imports and [project.scripts]
commands work immediately. [tool.uv].package can override whether it is packaged.
Dynamic metadata comes from the build backend, and projects with only
requirements.txt can receive an environment and lockfile too.
Declare local dependencies through [tool.uv.sources]:
[tool.uv.sources]
shared = { workspace = true }
# Alternatively: shared = { path = "../shared", editable = true }
Build backends need approval through allowBuilds, using keys such as
'pkg:pypi/hatchling': true. Git dependencies and source distributions are also
supported and require distribution approval. Direct wheel URLs are supported.
A workspace dependency without a source declaration is refused instead of
silently fetched from an index.
Members can share one environment:
[tool.uv.workspace]
members = ["packages/*"]
[tool.pnpm.python]
shared-environment = true
pnpm resolves every member into one pylock.toml and one .venv at the root.
Conflicting dependency requirements produce an error naming the members.
Independent environments remain the default.
Each project can also select its own extras and dependency groups
under [tool.pnpm.python]. Workspace defaults skip names a project does not
define; explicit project selections must exist.
Combine platform targets with Python versions:
supportedArchitectures:
- linux-x64-manylinux_2_28
- darwin-arm64
- win32-x64
python:
enabled: true
versions: ['3.12', '3.13']
Every platform is paired with every version. One pylock.toml
pins wheels and conditional dependencies for all of them. Installs select their
matching environment and reject interpreters outside the declared environments.
python.overrides and python.constraints control
versions throughout the graph. uv overrides and constraints are read too.
Python filtering now selects projects by name, path, and local-source dependency
relationships; pnpm add --filter <selector> pypi:<package> updates every selected
project.
pnpm add accepts Package URLs:
pnpm add pkg:npm/express@4.18.2
pnpm add pkg:cargo/serde@1.0.188
pnpm add pkg:pypi/requests@2.31.0
Each writes to its ecosystem's manifest. pkg is now a reserved registry alias,
regardless of case.
registries entries can name ecosystem: npm, cargo,
or pypi. Each Python index declares the names it serves with
packages, and a package resolves only from the index
that claims it — declaration order carries no meaning, and a missing package or
a registry error never falls back to another index. Cargo accepts one sparse
index. Credentials come from .npmrc, matched by origin; registry URL keys
cannot contain credentials.
registries:
https://packages.example.org/simple/:
ecosystem: pypi
packages: ["company-*"]
https://pypi.org/simple/:
ecosystem: pypi
packages: ["*"]
Name-based routing arrived in 12.5.1; 12.5.0 searched the indexes in declaration order.
This replaces python.indexUrl, python.extraIndexUrls, and cargo.indexUrl.
Without ecosystem declarations, PyPI and crates.io remain the defaults.
supportedArchitectures
accepts a list of exact platforms, such as linux-x64, linux-x64-musl, and
darwin-arm64, or Rust target triples. current names the install's platform.
The existing os, cpu, and libc mapping still works.
concurrencyGroups limits
tasks across pnpm processes on the same machine, including pipelines:
tasks:
test:rust:
concurrencyGroup: cargo
concurrencyGroups:
cargo: 2
A nested pnpm run in the same group reuses its parent's slot.
tools configures mirrors for Node.js, Bun, and Python
in global config.yaml or PNPM_CONFIG_TOOLS. Node.js also supports per-channel
mirrors. Workspace tool mirrors are ignored, and pnpm pack-app uses tools.node
for its embedded runtime.
pnpm cache prune removes obsolete metadata directories
left by the registry cache naming change. Use --dry-run to preview deletions.
pnpm cache list-registries now prints full URLs instead of encoded names.pnpm-lock.yaml, so a later frozen install accepts it
(#14912).pnx and pnpm dlx prompt for dependency build
approval in interactive terminals, including cached installs with pending builds.pnpm audit also avoids hangs on graphs with many shared
dependencies.See the v12.5.0 release notes for the complete list of changes.
You and your agents can now deploy static artifacts to Vercel in under one second through Vercel CLI.
Run vercel deploy to share a prototype, publish an HTML report, or preview a page created by your coding agent.
Vercel automatically detects eligible deployments, and valid artifacts will skip the build step, immediately returning a live URL.
Instant deployments support directories with:
Up to 10 HTML or Markdown files
A total size of 5 MB or less
Supported file extension include .html, .htm, and .md.
Update to Vercel CLI version 59.16.0 or later to get started, and run vercel deploy --help for all available options.
You can now opt into Turbo build machines on any individual deployment. This is useful when you need to increase resources temporarily without changing project settings. You can do this in three ways:
Include #VERCEL_BUILD_MACHINE=TURBO in your Git commit message before pushing to GitHub
Use vc deploy --turbo (Vercel CLI 59.20.0 or later)
Set buildMachine to turbo when creating a deployment with the REST API
Learn more in the managing builds documentation.
You can now run Harbor evals on Vercel Sandbox.
Harbor is the open-source harness behind Terminal-Bench, whose registry includes many other benchmarks such as SWE-bench, tau3-bench and OSWorld. Pass --env vercel to harbor run and each trial executes in its own isolated Firecracker microVM, so you can parallelize far beyond what your local machine is capable of.
A task's network policy is enforced at the sandbox firewall, outside the VM. Optional credential injection attaches secrets to matching outbound requests at that firewall, so they never enter the sandbox.
Paired with AI Gateway, one AI_GATEWAY_API_KEY reaches hundreds of models from multiple providers, and benchmarking another model is the same command with a different --model:
Swap --model to vercel_ai_gateway/openai/gpt-5.6-luna to run the same benchmark against an OpenAI model.
Requires Harbor 0.22.0 or later. Follow the step-by-step guide for setup, configuration, and troubleshooting. Learn more in the Sandbox documentation.
At Android, we are constantly working to provide developers and enterprise partners with the data they need to keep devices protected. Today, we're thrilled to announce the stable release of the AndroidX Security State version 1.1.0 and Security State Provider version 1.0.0 libraries which provides a centralized mechanism designed to bring further transparency to the comprehensive security posture and pending updates across the Android ecosystem.
Whether you develop security-critical, consumer-facing apps (such as banking, fintech, or healthcare) or Mobile Device Management (MDM) solutions, these libraries enable you to programmatically verify the security state of the device per component. Rather than relying on a coarse, monolithic Security Patch Level (SPL), you can evaluate true component-level protection and whether remediations are actively pending via the androidx.security.state library. For OEMs and Over-The-Air (OTA) client developers, the companion androidx.security.state.provider library allows you to expose update availability via standardized mechanisms.
Rather than taking an all-or-nothing approach to device access, developers and enterprises can combine DSPL, PSPL, and ASPL to make smart, contextual security decisions. For example, a banking or enterprise app can compare a device's current security patch (DSPL) against pending updates (ASPL) before initiating sensitive workflows like high-value payments or credential enrollment. If an update is waiting to be installed, developers and enterprises can require the user to update their device first. For even finer control, developers and enterprises can query whether specific high-risk vulnerabilities (CVEs) have been patched on the device, such as verifying that critical NFC or Bluetooth fixes are in place before authorizing tap-to-pay or proximity data sharing.
androidx.security.state library to make informed, context-aware decisions:
androidx.security.state.provider library establishes a standardized, Android IPC mechanism for update clients to report update availability directly on the device. Historically, even if proprietary OTA clients surfaced update availability, this information was siloed and not queryable by third-party applications. Going forward, apps can access ASPL details through a single, unified API, regardless of whether the update is delivered via an OEM’s dedicated OTA client or Google Play, as long as it is provided by the update client.
Here are two ways this approach benefits enterprises and Android OEMs:
We value your feedback! Please try out the libraries and let us know your thoughts or report any issues on the public Android Issue Tracker.
skills@1.7.0 adds Notion skills databases as an install source for agent skills .
Notion skills are reusable agent skills written as Notion pages. Teams author, review, and update them in the workspace they already use, then install them into any agent the skills CLI supports. No Git repository required.
To browse your Notion workspace's skills, run:
The CLI lists the skill packs shared with you and installs every skill in the packs you select.
To install a single skill, pass its Notion page URL:
Both commands use the Notion CLI (ntn) to authenticate. To set it up:
ntn login requires a Notion personal access token, so your workspace must allow them.
Access follows Notion's page permissions. You only see skills shared with you, so controlling who can install a skill is the same as controlling who can view the page.
This integration is built on Notion's new Agent Skills API, which exposes skills stored in Notion as standard Agent Skills folders. Because the format is standard, the same skills work in any agent that reads them.
Get started by creating a Notion skill.
Eroom’s law (hint: read Eroom backwards) is Moore’s law’s evil twin. The exponential drop in the price of computing power over the past 70 years has given us personal computers, the internet, cell phones, and now the AI revolution. Pharmaceutical research, unfortunately, has gone in the opposite direction, with the cost of developing each new drug doubling every nine years.
AI agents have the potential to reverse this trend, but general purpose solutions aren't built with the domain specificity that life science organizations need. That's why we developed Deep Life Sci: an open source agentic assistant created specifically for clinical and lab scientists.
The runaway cost growth in pharma comes from both stages of the drug development process: preclinical research and clinical trials. Identifying promising drug targets involves sifting through millions of scientific papers and massive biological datasets for insights. Once a candidate molecule appears likely to be safe and effective, it graduates to human clinical trials, where tens of thousands of pages of paperwork must be done to ensure compliance with a growing body of FDA regulations.
Many AI companies have promised that their tools will help restore research productivity, but general-purpose AI assistants like Claude and ChatGPT lack the necessary domain knowledge and integrations with scientific data sources. More specialized AI products for biotech often charge large markups.
In both cases, the agent harnesses are proprietary, preventing users from customizing them and locking them into expensive closed-source models. This issue is particularly critical in life sciences, where GxP validations require thorough documentation and audit logs that can articulate why the system behaves the way it did, requiring companies to have complete control of whatever system is being used to drive clinical decision making.
At LangChain, we believe that organizations that own their own intelligence will hold the advantage. We developed Deep Life Sci, an open source agentic assistant for clinical and lab scientists built on our Deep Agents harness, as a template for companies to adopt and modify for their use-cases.
Deep Life Sci can access clinical trial records from over 600,000 registered studies on ClinicalTrials.gov, 29 million paper abstracts through PubMed, and 12 million full-text articles on PubMed Central, reviewing hundreds of documents at once by assigning them to sub-agents. Each agent comes with a LangSmith sandbox, allowing it to safely run code to perform arbitrary data analyses. Users can upload PDFs, images, tabular data files, bibliographic files such as RIS, sequence ones such as SMILES, FASTA, and more, for the agent to include in its work.
In a typical workflow, a lab scientist finishes an RNA-seq or proteomics screen and uploads the results table. The agent runs enrichment in the sandbox to identify differentially expressed genes, then searches the literature for prior evidence linking each hit to the phenotype, separates well-described genes from novel ones, and returns a ranked table with the supporting papers.
A clinical development or HEOR team, on the other hand, might need to find every published trial of the standard of care in an indication, with the endpoint value, N, population characteristics, and follow-up duration extracted consistently. The agent runs the search, screens against the criteria, extracts each trial into a common schema, and produces both the table and a forest-plot-style comparison.
During the clinical trial phase, thousands of pages of different types of documents are created, ranging from informed consent, clinical protocol documents and amendments, case report forms, and more – all of which must be thoroughly audited, reviewed, and edited numerous times before being finalized. Using Deep Life Sci, users can upload reference protocol documents, research and gather additional statistical information, and quickly curate necessary feedback and edits that could ultimately cut clinical documentation time significantly.
The value of Deep Life Sci further compounds when the agent is optimized and integrated into a company’s ecosystem. Deep Life Sci knows what the primary endpoint is, but it doesn’t know company-specific nuances such as results from internal assays, which endpoints regulators pushed back on, or which trial sites actually enrolled rather than just promising to.
Integrating this context into the harness is what owning your intelligence looks like in practice, and because Deep Life Sci’s code is open source, organizations can approach this however they wish. This customization can include adding integrations with internal data and documentation, leveraging different frontier and open source models, enforcing guardrails and approval gates, and more.
Modifying the harness puts you inside the agent development lifecycle (ADLC): build, test, deploy, monitor, then feed what you learned back into the next version. Tracing and evaluations help power this development loop.
Every Deep Life Sci run is logged end-to-end in your own LangSmith account, including the literature searches the agent issued, the code it ran in the sandbox, the documents each sub-agent read, and how it moved from those to its answer. These trajectories allow for debugging and improvement of the agent, and serve as an audit record.
Evaluations tell you whether a change to the agent helped its performance. Deep Life Sci ships with a default eval set that can be modified and added to as you add integrations and identify new use cases. Run the set before and after you swap a model or rewrite a prompt, and you'll see whether the new version actually improved or quietly regressed.
Agentic AI is already revolutionizing fields like coding and mathematics. Biomedicine, where cost-effectiveness and iteration speed directly translate into human lives saved, should not be left behind. Biotech and pharma companies that combine open source tools like Deep Life Sci and the ADLC capabilities of LangSmith can reverse Eroom’s law by delivering cost savings and faster iteration across the drug development cycle.
Get started with Deep Life Sci here.
An engineer at a software company is building an agent to keep the company's view of the market up to date. It monitors a few hundred thousand prospects and customer accounts for signals that an account is open to engagement: a new funding round, a leadership change, a product launch, or a hiring surge that indicates budget.
The account records already live in Databricks, in Delta tables governed by Unity Catalog and joined to the company's own usage and pipeline data. But the signals that move an account live outside the company, on the web. The agent's job is to combine the two, continuously, into one coherent and up-to-the-moment picture of every account, so it can tell a salesperson which handful to call this week.
The first version isn't one system. It's the same enrichment logic, rebuilt from scratch three separate times, once in each tool the engineer reached for. The first pass runs in Claude Code, where the agentic parts (deciding which accounts need a fresh look, chaining searches, writing the summary) are most of the work. When a colleague mentions that Codex handles a certain kind of batch scripting faster, the engineer ports the enrichment loop over to check. A third copy skips the harness entirely and calls a model directly over the API, for a lightweight nightly job that just needs a single prompt and a response, no tool orchestration required. Same job, three builds, each shaped by whichever tool fit that moment.
Each harness bundles its own tools and its own web search and wires them up its own way, so the engineer builds the same enrichment logic three times, once in each harness's config format. That is where the day goes. Instead of improving how accounts get enriched, the engineer is learning how Claude Code wants its tools declared, why the same MCP server connects differently in Codex, and what the raw API path is missing that the other two had for free.
The tools are not equivalent, and so neither are the results. The web search bundled into one harness returns different data than the next. A source reachable in one is missed in another. Built-in web search tools for LLMs can find high-level information like funding rounds and leadership changes, but miss granular details like tech stack changes. Access to online information is the thing this agent exists to produce, but its quality now depends on web search that can’t reliably surface key details on the web.
And nothing sits above the three of them. No shared meter, so no one can see or cap what a cycle costs across a few hundred thousand accounts. No shared rulebook, so which sources an agent may read and when a human signs off are set three ways or not at all. No shared record, so when a result is wrong, there is nowhere to reconstruct what the agent read, spent, or decided.
It sort of works in that it produces a result. And that is exactly why it never gets fixed. It works well enough to keep, but not enough to fully trust.
Omnigent is the layer that reins in the sprawl. It sits above the individual harnesses, so the engineer defines the agent once, the model it runs on, the tools it can reach, the policies and limits it operates within. The three rebuilds collapse into one definition, and the engineer's attention goes back to account enrichment. Tools stop being whatever each harness came bundled with and become declarations on the agent, set once and swapped freely. Running on a Databricks-hosted model, the model calls route through the Foundation Model APIs, where every call is captured for cost, audit, and governance in one place instead of scattered across three runtimes. And when the model or the economics change, the engineer changes one line, picking a new model or downshifting to a cheaper one without disruption.
That closes most of the sprawl, but it leaves one critical thing decided by default rather than by design. Web search is one of the core capabilities every harness bundles, and no two bundle the same one. The same query gives one result through Claude Code and another through Codex. Omnigent provides you the ability to define a consistent choice across each task, but it does not make the decision for you. You have to assign a partner search capability. With a partner like Nimble, you can put something in the slot that adapts to the task instead, and give every harness underneath the same expert read.
Nimble’s Search API can ground answers in fresh, real-time web data through live search. For deep research tasks, Nimble’s Web Search Agents automate web search and extraction orchestration to fulfill your task, working many sources, cross-checking them, and returning an answer with the citations to back each claim, an audit trail that the general path could never produce.
While general web search tools treat every use case the same, Nimble specializes in the agent’s specific use case, self-learns the best retrieval methods, and adapts web search and crawling to go deep into the domain to capture data that generic search tools miss. It gets to the data behind JavaScript, filters, and pagination that an ordinary crawler gives up on. And because it remembers the best way to retrieve the relevant data, it reuses data retrieval paths rather than rediscovering everything from scratch to reduce token costs. Named as the provider in the config, this is the fast path to a more complete web context for your agents.
In Nimble's testing, adding Nimble’s web search raised LLM benchmark accuracy from 46 percent to 71 percent, while cutting web search costs in half (Claude vs Nimble web search costs). Web Search Agents can be pointed at a domain and kept there, so it remembers which sources and which retrieval paths produced the right data and reuse them the next time. It gets sharper the longer it works a domain, and the cost of rediscovering where a signal lives drops on the accounts it runs against most.
Returning to the engineer, the agent is now on a path to becoming a coherent, manageable, trustworthy whole. The agent is defined once in Omnigent, on a Databricks-hosted model, with its tools, policies, and limits in a single spec. The three rebuilds are gone. So is the plumbing tax; the engineer is back on enrichment, not on how each harness wants its tools declared.
Web search is now one decision instead of three. Naming Nimble on the web_search builtin points every harness underneath at the same Nimble Search API for fast and efficient web search:
For the accounts that need a defensible answer rather than raw web data, Omnigent can reach for Nimble's Web Search Agents, which automate web search and extraction for research, enrichment, or dataset building.
The key comes from a Nimble account, which you can start free.
Control now has one home. Model calls route through the Foundation Model APIs under governance, cost is visible and capped in one place, and what the agent reads, spends, and decides is captured consistently across one governance surface.
And the two halves of the picture finally sit together. The internal record in Databricks and the external signal from Nimble, in one place, governed and read by one agent. Version 1.0 was three harnesses and no vantage point. This is one agent, grounded in what the company knows and what the web can tell it, running where the data already is. Consistent where it used to drift, deep where it used to be shallow, and full governance over external web context retrieval.
Standing this up takes two steps.
Connect Omnigent to Databricks. Databricks runs the Omnigent server for you. On your own machine, install the CLI with the Databricks integration and register the machine as a host:
Then sign in with your workspace identity and run your first agent on a Databricks-hosted model. Omnigent on Databricks is the place to start; it covers the managed setup end-to-end and links the CLI steps. Two prerequisites to check first: the Omnigent Beta has to be enabled for your workspace, and the workspace has to be in a region that supports Unity AI Gateway. For other install methods and requirements, the full install reference has them.
Your agents run on the managed server, so the same sessions follow you across every surface:
Point search at Nimble. Name Nimble on the web_search builtin, the one-line change from earlier, and every Databricks-hosted agent grounds its answers through it. For defensible, auditable work, reach for the research pass. The Nimble connector docs cover both. You will need a Nimble key, start a free trial to get one.
The internal record is already yours. This is what it takes to let your agents reason over the rest of the web, with the same platform holding both halves.
As organizations face growing security and regulatory requirements, maintaining compliant infrastructure becomes increasingly complex. Many organizations use policy as code to define and enforce guardrails consistently across their infrastructure estates. But operationalizing policy as code can still require significant time and specialized expertise.
We recently introduced the public beta of Terraform policy (tfpolicy), a declarative, HCL-based policy-as-code framework deeply integrated with Terraform. Terraform policy gives teams a familiar way to author and enforce policies while bringing governance closer to their Terraform workflows.
Today, we are expanding that experience with the public beta release of native pre-written policy experience in HCP Terraform. While creating a policy set, teams can now discover HashiCorp-managed pre-written policies, review relevant policy details, select the policies they need, and configure enforcement.
In this post, we’ll look at the challenges of operationalizing policy as code and how this release provides a faster, more integrated way to apply compliance guardrails at scale.
HashiCorp already provides pre-written Sentinel policies for common security and compliance requirements, including AWS CIS Foundations Benchmark, AWS Foundational Security Best Practices (FSBP), NIST SP 800-53, and other frameworks.
Previously, teams had to find the appropriate policies outside HCP Terraform and bring them into their policy workflows. Teams creating their own policies also had to interpret compliance controls, translate those controls into policy logic, and test and maintain the resulting policies over time.
This work grows as organizations adopt more cloud providers, services, and compliance frameworks. The challenge is not simply making pre-written policies available. Teams also need a straightforward way to discover, review, choose enforcement for, and apply them through their existing Terraform workflows.
The native pre-written policy experience brings HashiCorp-managed policies into the HCP Terraform policy set creation workflow. With this new approach, users can:
Select the new pre-written policy set type
Search and filter available policies by cloud provider, service, and compliance framework
Review policy details before selecting
Select one or more policies for the policy set
Configure the supported enforcement mode for each policy
Attach the completed policy set to an organization, project, or workspace
Native pre-written policies are managed by HashiCorp and remain read-only in HCP Terraform. This helps protect the integrity of each policy while allowing organizations to decide where and how they should be enforced. The initial public beta focuses on policies aligned with AWS Foundational Security Best Practices (FSBP) and AWS CIS Foundations Benchmark, with support for additional compliance standards including a limited set of CIS Foundations Benchmark Policies for Microsoft Azure and Google Cloud coming soon.
The experience supports both existing pre-written Sentinel policies and new pre-written policies authored using Terraform policy through the same policy set workflow. Sentinel pre-written policies are available for organizations using agent execution mode. Pre-written policies default to Advisory enforcement, allowing teams to identify violations without blocking Terraform runs. When teams are ready, supported policies can be configured as Mandatory to block non-compliant runs.
Together, these capabilities make it easier for teams to adopt policy as code, apply consistent guardrails, and scale governance across their Terraform environments.
Pre-written policies reduce the work required to apply common guardrails in HCP Terraform while preserving the flexibility to create custom policies for organization-specific requirements.
To try it today, select the Pre-written policies option when creating a new policy set in HCP Terraform. Refer to our manage policy sets documentation for step-by-step instructions.
Looking to author custom policies alongside these pre-written controls? Check out our introduction to Terraform policy to get started.
HashiCorp is deprecating HCP Vagrant through a phased process. The Vagrant CLI and source repository will remain available, but customers must move their Vagrant boxes to another hosting provider and assume the associated hosting costs.
HCP Vagrant will stop supporting new box and registry creation on October 1, 2026, and customers using HCP Vagrant have until December 31, 2026 to find a new provider and rehost their boxes. To help with the transition, we will release additional capabilities, including:
The option to export boxes to a local drive
Guidance on how to host Vagrant boxes in Amazon S3
Details about the folder structure required to support multiple providers and architectures
Instructions for taking a snapshot of all existing Vagrant boxes and placing them into a static archive that uses URL redirects during the transition
End of new creation: October 1, 2026. After this date, users cannot create new Vagrant boxes or registries
End of support and maintenance: November 2, 2026. HashiCorp will end support and maintenance for existing Vagrant deployments
End of operations: December 31, 2026. HashiCorp will decommission all remaining Vagrant deployments
This deprecation applies only to HCP Vagrant. The Vagrant CLI and source repository on GitHub will remain available, enabling teams to continue building boxes locally and sharing them through a new customer-hosted box repository. However, users will no longer be able to create new boxes or share box environments through the HCP Vagrant and Vagrant Public Registry interfaces after the applicable shutdown dates.
Start your migration by taking inventory of where HCP Vagrant Registry is used across your organization.
Consider reviewing:
Vagrantfiles that reference registry-hosted boxes
CI/CD pipelines that download or publish boxes
Internal developer documentation
Onboarding guides
Automation scripts
Public or private boxes your team maintains
Any downstream users or teams that depend on those boxes
Once you understand how your organization uses HCP Vagrant, evaluate where you will host those boxes after the service ends. Your replacement repository must make the .box files and catalog metadata available to the Vagrant CLI. Catalog metadata preserves information about box versions, providers, architectures, download URLs, and checksums.
To support a smooth transition, we will publish migration guides that explain how to migrate your HCP Vagrant Registry data to customer-managed hosting solutions. If you encounter issues during this phased deprecation, please open an issue on our Vagrant GitHub repository or reach out to vagrant@ibm.com
Before a vector database can search vectors, it has to store them. But storing high-dimensional vectors at full precision is quite expensive. Vector quantization (VQ) reduces the number of bits needed to store a vector, making it a critical part of maintaining a vector database.
Because VQ is so important (to both vector databases and LLMs), many research papers are published on the topic every year. Pinecone has been using quantization since its first prototypes. But we can always do better, so we set out to survey and benchmark newer results. We were pretty overwhelmed by just how many quantizers are out there. To make matters worse, every paper seemed to evaluate performance differently, measuring different metrics on different datasets and optimizing for different hardware. We were unable to find any systematic attempt to evaluate the leading methods against one another.
Of course, faithfully implementing dozens of quantizers from scratch comes with its own challenges. Luckily, as we dug deeper into the literature, we began to notice a pattern. Many published quantizers are actually just slight variations of existing ones. In fact, most of them are built from a relatively small set of primitive operations. That gave us an idea: what if we published an open-source library of these core primitives, where building a quantizer was as easy as writing a recipe of which primitives to use and in what order? Then, we would be able to evaluate all of these quantizers in a fair and reproducible way. It would also make it easier to experiment with new variations of existing quantizers or invent new ones altogether.
This was the start of the VQ-bench project. With this post, we're excited to share VQ-bench with the public, including:
Note that this is just the first iteration of VQ-bench; we encourage feedback, corrections, and contributions, and we will add more quantizers over time.
A quantizer is anything that can take a set of vectors, compress them, and recover desired information later on. In VQ-bench, a quantizer must implement four methods:
| Method | Function |
|---|---|
| fit | given a sample of vectors (and optionally queries), learn a model |
| encode | given the model and a set of vectors, return per-vector codes |
| reconstruct | given the model and the code for vector x, reconstruct it |
| score | given the model, a query vector q, and the code for x, estimate the dot-product score ⟨q, x⟩ |
Quantizers are rarely built from scratch. In the literature, they are assembled from a small set of basic operations, which VQ-bench formalizes as primitives. A primitive implements the same four methods as any other quantizer, plus two more that specify exactly how it hands data to the next stage:
| Method | Function |
|---|---|
| apply | given the model, transform the vectors into what the next stage should see |
| apply_queries | given the model, transform the queries into what the next stage should see |
A primitive's reconstruct and score methods also take as input the next stage's reconstruction and score estimate, respectively.
That makes six methods in total. The extra two are the chaining contract: they are what let primitives be composed, which is the subject of the next section.
VQ-bench implements three groups of primitives.
Center, Normalize, PCA, RandomRotate, ...).CastUint, CastAngular, CastNormal, KMeans, ...).Segment).A pipeline is a special type of quantizer given by composing two or more primitives in a chain. Compressing a vector walks it forward through the chain, and recovering a vector (or its score) walks it backward.
fit and encode follow the same path. At each stage, they perform that stage's job (learning the model / computing the codes). Then, they call apply to transform the vectors to the next stage and recurse. At the end, fit concatenates each stage's model and encode concatenates each stage's codes.reconstruct starts at the last stage. Each stage above it folds its own contribution back in (e.g., adding back the mean, undoing a rotation, etc.) until the first stage has an approximation of the original vector.score works the same way, except every stage needs the query as it saw the data. So, it begins by walking just the query forward with apply_queries. Then, it performs the backward pass on the score.A quantizer does not have to be a pipeline. Anything that implements the four methods qualifies, and the interface leaves room for methods that are built some other way. But most published quantizers can be expressed as pipelines of primitives, which is what makes the decomposition worth building on.
For example, E-RaBitQ is a popular quantizer (which we found to be quite performant in our experiments). The E-RaBitQ pipeline consists of four primitives:
A diagram of this pipeline and table for the primitive functions are given below.
The E-RaBitQ pipeline.
| Center | Normalize | Random Rotation | Angular Cast | |
|---|---|---|---|---|
| fit | mean dataset vector μ | none | rotation seed | none |
| encode | none | the norm ‖x‖ | none | grid(x) and cos(x, grid(x)) — b bits per dimension and one scalar |
| apply | x → x − μ | x → x / ‖x‖ | x → Rx | x → x − ĝ, where ĝ = grid(x) / ‖grid(x)‖ |
| apply_queries | identity | identity | q → Rq | identity |
| reconstruct | y → y + μ | y → ‖x‖ · y | y → Rᵀy | y → y + ĝ |
| score | s → s + ⟨q, μ⟩ | s → ‖x‖ · s | s → s, since the query was rotated too | s → s + ⟨q, ĝ⟩ / cos(x, grid(x)) |
We evaluated a suite of 14 quantizers on 5 datasets from VIBE. Each dataset consists of vectors to encode and queries to score. Below, we present some results for two of the datasets: ArXiv (1,344,643 vectors in 768 dimensions) and Yahoo (677,305 vectors in 384 dimensions). You can view the full results on the website.
Reconstruction MSE is the traditional metric for VQ, and it's important for applications like LLM weight compression. To measure it, we sample 1000 random dataset vectors . A quantizer reconstructs and we measure the average value of .
ArXiv
Yahoo
For vector databases, a more relevant metric is recall, specifically for reranking. To measure it, we take each query and compute the 1000 dataset vectors of maximum dot-product. A quantizer estimates these 1000 scores, and we measure what fraction of the estimated top-10 were contained in the true top-10 (averaging this fraction over all queries).
ArXiv
Yahoo
We also measure how long it takes to encode the entire dataset. Note that encoding is done in chunks and accelerated via multithreading. These results were obtained on an Apple M2 Pro with 16GB RAM using 6 threads.
ArXiv
Yahoo
Overall, we can see some clear trends. PQ and OPQ consistently have the lowest reconstruction MSE. EDEN and E-RaBitQ are comparable in terms of recall, especially at higher bit budgets. EDEN is also much faster to encode than PQ, OPQ, and E-RaBitQ, making it a good candidate for most quantization applications.
We built VQ-bench to be extended, and the repo takes two kinds of contributions.
Got a new quantizer? Usually just a few lines of code. The E-RaBitQ pipeline above is four primitives in a list, and many published quantizers are a similar reordering of primitives the library already ships.
Got a new primitive? Implement the six methods above and it composes with every other primitive in the catalog. Every pipeline can use it, including the ones nobody has written yet.
Either way, you get the evaluation harness. A short config runs your method over the whole suite, measured exactly the way every other method is measured: recall@k, reconstruction and score error, bias, softmax KL and total variation, size in bits per dimension, and encode, score, and reconstruction cost. Both lists keep growing as we add datasets and metrics. We refresh the published benchmark on a regular cadence, and new methods are folded in then.
We also want corrections. If we implemented your quantizer wrong, or we missed a method worth including, open an issue and tell us.
When we first launched Android Bench, we built a rigorous foundation for evaluating how large language models (LLMs) assist developers with real-world Android tasks. As AI models and agents rapidly evolve, we’ve been updating our methodology, such as aligning our benchmark framework with the Harbor framework. Today we’re releasing the first set of long-horizon tasks (LHT), which are tasks of great complexity that take an engineer multiple days or even a week to complete. We are also introducing agentic evaluation, starting with agents from corresponding model providers. This addition brings us to Android Bench 2.0—a major upgrade designed to evaluate AI models and agents against the scale, ambiguity, and complex multi-step problem solving that you tackle every day.
The first iteration of Android Bench, along with similar early AI coding benchmarks, focused on incremental changes to existing repositories, in many cases limited to bug fixes or smaller feature requests. This was a reflection of the capabilities of AI assistance at the time, as well as how you were using it.
To continue helping you find the models and coding agents best suited to your development workflow, we have raised the bar of our evaluations to match the work you delegate to AI. Android Bench 2.0 mirrors these ambitious challenges with LHTs that include upgrading dependencies, adding new features, building apps from scratch, or converting a cross-platform app to Android.
On multi-day engineering tasks, binary pass or fail grading doesn’t capture the full picture.
For example, an agent might refactor 40 screens to Jetpack Compose, set up database tables, and pass 90% of requirements, but fail a single edge-case assertion. Binary scoring rates this run as 0%, obscuring the model's architectural capabilities. We are moving to continuous scoring to provide a more meaningful signal, both for model development and for your understanding of how AI can help you.
We calculate this completion rate through a combination of factors like functionality, visual fidelity, and avoiding regressions. We also apply objective scoring penalties for deviations from evaluation instructions or structural constraints. Check out the updated leaderboard and click into each model’s card view to see additional elements such as the pass rate, completion rate, and average costs per model and per task.
The highest pass rate for LHTs is around 28%, much lower than the ~91% for the original tasks in the benchmark.
Beyond measuring how well AI handles long-running tasks, the LHT dataset helps us learn more about the strengths and weaknesses of tested models, and we offer you more practical guidance.
Across model tiers, AI does a better job at writing new code rather than refactoring existing code. Refactors and migrations get trickier because success depends on architectural complexity rather than code volume.
Models show strong capabilities on well-established, deterministic transformations, such as converting Java to Kotlin, swapping Retrofit for Ktor, or introducing a ViewModel layer. They apply these patterns consistently, even across 125+ files and 8,000+ lines of code.
However, models struggle when tasks require runtime validation (like missing dependency injection graphs), involve breaking framework changes, or run into knowledge gaps with unreleased libraries. Porting cross-platform apps to Android remains an open challenge—no model hits a 100% pass rate, and frontier models reach at most a 80% completion rate.
To help you get a better sense of how models perform when integrated into your agentic workflows, we are adding commonly used agents into our evaluation. We're starting by running new models against LHTs with agents from the corresponding model provider. For example, we ran GPT 5.6 Sol on Codex, and Gemini 3.8 Flash on Google Antigravity. This pairing shows how harness design positively impacts developer outcomes, as we’ve seen prompt caching and compact tool windowing can result in token reductions.
We’ll be expanding this in the future by also highlighting results across various model and agent combinations, to help you discover which combinations work best for you and your team.
We invest in this measurement because it’s important for you to be able to use your agent and model of choice for Android development, and we'll have more to share with you in the coming weeks.In addition, we are continuing to expand our leaderboard to ensure you have the most up-to-date data for your development decisions. We added Gemini 3.8 Flash, Gemini 3.7 Flash, OpenAI’s GPT-6, Anthropic’s Fable 5.1, Kimi K3, and Qwen 3.8 Max, with OpenAI’s GPT-6 Astra at the top with a 28% pass rate.
Android Bench 2.0 delivers a robust environment for measuring AI for Android development. By combining long-horizon tasks, multimodal evaluation, agents, and continuous scoring, we hope to empower AI research teams to build more capable, dependable AI coding partners, and we hope to provide you with more transparency about your options for AI development.
Check out the updated leaderboard along with the updated methodology. Your feedback directly influences how we evolve Android Bench, so please continue to share your feedback with us on GitHub, as well as our social channels like X and LinkedIn.
Antigravity Agent 09-2026: Released antigravity-preview-09-2026,
which replaces and deprecates antigravity-preview-05-2026.
If you run on a remote sandbox (environment: "remote") and read only
output_text or model_output steps, update the agent string and nothing
else changes.
If you run tools locally (local_environment) or parse function_call
steps, the built-in tools changed. Parameters use PascalCase instead of
snake_case, and file edits use line-range replacements instead of full
rewrites.
| Capability | 05-2026 | 09-2026 |
|---|---|---|
| File creation | write_file(path, content) |
write_to_file(TargetFile, CodeContent, Overwrite, Description) |
| File editing | write_file(path, content), full rewrite |
replace_file_content(TargetFile, StartLine, EndLine, TargetContent, ReplacementContent) |
| File reading | read_file(path, offset, limit), byte offsets |
view_file(AbsolutePath, StartLine, EndLine, ContentOffset) |
| Directory listing | list_files(path) |
list_dir(DirectoryPath) |
| File and code search | None, agents used shell commands | find_by_name(SearchDirectory, Pattern, MaxDepth) and grep_search(SearchPath, Query, IsRegex) |
| Shell execution | code_execution(command, timeout_seconds) |
Unchanged |
| Web search | google_search(queries) |
Unchanged |
See the Antigravity Agent guide.
antigravity-preview-05-2026 shuts down on October 5, 2026, tracked on the
deprecations page.
When Neon first launched in 2022, there was a gap between how fast teams were moving and what Postgres let them do. Compute and storage were welded together into a monolith, and every copy of a database was expensive to create, slow to spin up, and painful to throw away. It was already the era of GitHub, Vercel, automated CI/CD. Teams wanted their database to move as smoothly as the rest of their stack but were stuck with an outdated design.
To close that gap, we rebuilt the architecture underneath Postgres, pioneering what would later become the lakebase architecture. We kept 100% of Postgres but we separated compute from a distributed, versioned object storage engine. From this foundation, we were able to build features that gave the database a modern DX experience, like instant provisioning, real-time autoscaling, scale to zero, and branching.
Postgres was finally catching up with how developers worked. And then agents came along.
The other side of the Neon API are now agents acting on behalf of developers. Giving Postgres the right DX turned out to be the perfect starting point to provide a great AX, but when agents build apps they don't build on databases alone - they deploy backends.
When a coding agent ships an app it deploys Postgres and a set of tooling around it. Apps need to store uploads, run jobs that touch that data, authenticate users, call AI models. If those are wired up as separate services on top of the Neon database, the Neon experience breaks - the bucket points at production from every branch, the function doesn't know the branch exists, auth users live in a different system, and so on. This is not the right AX, so we're building these tools ourselves from the same semantics as Lakebase Postgres, our database.
When we say "we're building backends", we think of "backend" as a set of solid primitives an agent can call, not a bundle of managed services behind one bill. The distinction is deliberate. A backend-as-a-service bundles features and asks you to adopt its way of doing things. That is not what we're building.
The reason comes down to how agents write software. An agent is good at composing primitives it already understands: Postgres, an S3 API, a standard model SDK. Give it well-established pieces with predictable interfaces and it might get the app right on the first try. Auth and ORMs already showed the pattern: Better Auth gave agents a primitive they reach for by default, Drizzle did the same for the ORM, and the code comes out right because the primitive is solid. Your entire backend should work the same way.
We're building our backend as a set of primitives, each with a standard interface and an understanding of the Neon design principles: infra that adapts to the workload, instant deploys and restores, and branching-first, agents-first workflows. Nothing here asks you to learn a proprietary framework or trades your data for convenience, and you can point standard tools at any of it and leave whenever you want. But the primitives compose, and an agent can wire them together through one interface to build solid foundations for software.
A full walkthrough:
Postgres at the center, setting up the stage for all workflows.
> Create a Neon database for my app. Make the main branch autoscale up to 8 CU, make sure it suspends after 5 minutes of inactivity.import { defineConfig } from "@neon/config/v1";
export default defineConfig({
branch: (branch) => {
if (branch.isDefault) {
return {
postgres: {
computeSettings: {
autoscalingLimitMinCu: 0.25,
autoscalingLimitMaxCu: 8,
suspendTimeout: "5m",
},
},
};
}
return {};
},
});The Neon database you already know:
S3-compatible object storage that branches with your data.
> Add a private bucket called `uploads` to this Neon backend. Keep it on the same branch as the database so preview uploads cannot change production files.import { defineConfig } from "@neon/config/v1";
export default defineConfig({
buckets: {
uploads: { access: "public_read" },
},
});Your files now follow your database as you branch:
Long-running Node.js compute for those jobs touching Postgres.
> Add a function that reads an uploaded file and records its status in Postgres.import { defineConfig } from "@neon/config/v1";
export default defineConfig({
functions: {
processupload: {
name: "Process upload",
source: "./functions/process-upload.ts",
},
},
});Serverless functions you can deploy right next to Postgres:
DATABASE_URL and credentials for other Neon primitives injected automaticallyAuth that branches, with the Better Auth code you know.
> Add Managed Better Auth to this app and use it as the identity provider.import { defineConfig } from "@neon/config/v1";
export default defineConfig({
auth: true,
});Your auth lives next to your database and branches with it:
neon_auth schema, where they can be queried with SQL and used with RLSFrontier and open-weight models powered by Databricks Foundation APIs.
> Use Neon AI Gateway for model calls. Keep the model configurable so I can test another model in a preview branch without changing production.import { defineConfig } from "@neon/config/v1";
export default defineConfig({
aiGateway: true,
});You can call AI models directly from Neon:
Our backend will keep growing on the same foundation. The Electric team, who built PGlite and the Electric sync engine, is now part of Neon, and realtime is exactly the kind of primitive agents should be able to find on Neon. We'll share more soon.
In the meantime, we want to see what you build with these tools. Tag us on X, send us feedback, and tell us what to improve. We're in Discord too.
Our Safari release notes have never been as long as they are for this version. The number of features alone rose from 58 to 83 since the first beta in June.
Safari MCP makes working with coding agents dramatically easier. Customizable select turns the real <select> element into something you can fully restyle — now with new UA default styles that provide an even-better starting place. Scroll anchoring stops content from jumping when something loads in above. The <model> element comes to iOS, iPadOS, and macOS, giving 3D a powerful HTML element. Websites can now provide immersive environments on visionOS. And much more.
Are you developing websites using coding agents? The Safari MCP server, now available in Safari 27.0 will make your workflow faster and more powerful. Give Claude Code, Codex, or the agent of your choice control over the browser window so it can see how your code renders. Safari MCP provides access to the DOM, network requests, screenshots, and console output. Your agent can do more on its own while you do less hopping between windows, less dropping screenshots in your terminal, and less typing prompts to describe what’s not working.
The Safari MCP server enables your agent to:
And much more. The MCP server runs entirely on your local machine. It makes no network calls of its own. It does not have access to your personal information in Safari. And any captured data goes directly to the agent you’re running, not to Apple.
To give it a try, go to Safari > Settings > Developer > check “Allow remote automation and external agents.” (If the Developer pane is not available, first go to Advanced, and check “Show features for web developers”.)
If you’re using Claude:
claude mcp add safari-mcp -- "/usr/bin/safaridriver" --mcp
If you’re using Codex:
codex mcp add safari-mcp -- "/usr/bin/safaridriver" --mcp
For other agents, you put the following in your mcp.json or config.json file.
{
"mcpServers": {
"safari-mcp": {
"command": "/usr/bin/safaridriver",
"args": ["--mcp"]
}
}
}
Let us know what you think, especially if you have any feature requests. Learn more by reading Introducing the Safari MCP server for web developers, or by referencing documentation.
The biggest feature of Safari 27.0 isn’t a feature at all. It’s the tremendous effort that went into improving the quality of existing features. At WWDC, we were proud to announce 525 fixes. Then we added 60% more, reaching a total of 844. Plus the majority of feature work improves existing features.
When we look at the efforts we made to improve quality, the story can be seen in several themes.
Compatibility. Our team made many changes to help make specific websites work correctly for their users. For example, Hindi InScript typing in an online document editor, images vanishing from search results on a restaurant reservation site, and Pahawh Hmong text misrendering in an online encyclopedia.
Foundations. Sometimes the best way to improve quality is to start over. Safari 27.0 has an all-new ES module loader. We rebuilt CSS Zoom. And now inline layout places elements with subpixel precision.
Depth. We got deep into specific technologies. There are 66 fixes to SVG in this release alone, including an end-to-end review of the SMIL animation engine. HTML tables got a systematic pass, with absolutely positioned tables now handling percentage-sized children, min-height, and max-height correctly. Plus deep work on Media Source Extensions (MSE) and Encrypted Media Extensions (EME). And much more.
Alignment. Much of the work is to better match exactly what web standards prescribe. For example, we corrected the MathML Core operator dictionary and its spacing values across several fixes. Fixes to innerText bring Safari’s rendered-text output better in line with standards for display, visibility, white-space, and form controls. And we improved how HTTP cache obeys Cache-Control.
Integration. Sometimes two features each work perfectly alone, but combined, something starts to go wrong. We fixed a lot of these this year. For example, -webkit-line-clamp shipped in WebKit in 2010, while text-wrap: balance arrived in 2024. Before Safari 27.0, if you applied both to the same element, the balancing simply didn’t happen. Now that’s fixed.
We truly hope all of these efforts throughout the last year make your work as a web developer a little easier. Read through the resolved issues at the end of this article to see specifics. And learn more about what we are doing to raise the quality of WebKit by watching What’s new in WebKit for Safari 27.
The <select> element has been part of the web since the very beginning of HTML. But until recently, there wasn’t a lot you could to do style it or fill it with custom content. Customizable Select changes that. Now in Safari 27.0, it lets you build a fully custom drop-down menu to match the look and feel of your website or web app, without reaching for JavaScript or a pile of <div>. You can even push far beyond a typical drop-down menu to a very different UI. Because it’s a real form control, you get automatic, reliable support for keyboard navigation, screen readers, form submission, validation, change events and more.
Start by applying appearance: base-select in your CSS. This immediately switches to the look and feel provided by new UA styles, and enables the new powers in HTML.
select,
select::picker(select) {
appearance: base-select
}
You might notice that the default UA styles in Safari 27.0 are different than they were for the first beta back in June. The summer gave us the opportunity to reflect on what it will be like for web developers to write custom styles on top of the new defaults. We realized after 30 years of web developers struggling with form control styling, we wanted to provide something even better.
These defaults set you up with all the basics. You won’t be left with homework to do to get the select into a usable state. You can simply switch to the new control with appearance: base-select, and apply as little or as much additional code as you’d like. Don’t like the new defaults? You are in luck, it’s very easy to override them. Feel fine keeping any of these pieces like the new drop shadow, 4px rounded corners, touch-friendly line height, cleaner hover states, user-ready chevron & checkmark, subtle opt group styling, etc? Great! It’s already done for you, with support for all the variations like light & dark modes, forced color mode, disabled states and more.
We brought this new design to the CSS Working Group, where it’s being further discussed and refined. Once other browsers update their implementations, we will together reach our shared commitment for all browsers to support an identically interoperable starting place.
New pseudo-elements like ::picker-icon and ::checkmark let you easily target parts of the control that were previously unstylable. Plus, you can now insert HTML elements inside each <option> to add more detail. The new <selectedcontent> element can be used to adjust what gets displayed as the currently-selected option’s content. Learn more watching Rediscover the HTML Select Element from WWDC26.
Originally shipped a year ago in visionOS, the HTML <model> element is now also available in Safari on iOS, iPadOS, and macOS. This new element is a lot like video, audio, and img — this time embedding a 3D model in the page.
<model src="mallet.usdz"></model>
Just like the other HTML elements for media, you can link to multiple source files, including a fallback.
<model>
<source src="boot.usdz" type="model/vnd.usdz+zip">
<source src="boot.glb" type="model/gltf-binary">
<img src="boot.png" alt="workboot in light tan leather">
</model>
You can optionally include attributes like environmentmap to provide custom lighting for your model. Or stagemode, which sets the default interaction behavior. Target your model with JavaScript and open up a wide range of possibilities.
Learn all about it, including where to get a 3D model, how to optimize it for the web, and what can be done with JavaScript by watching Get started with the HTML Model Element from WWDC26. And check out these demos in Safari.
Safari 27.0 also adds support so the CSS dynamic-range-limit property can be applied to the <model> element, giving you control over HDR tone mapping and rendering range for 3D content on iOS and macOS.
Responsive image techniques get easier with the auto keyword for sizes.
<img src="photo.jpg"
srcset="photo-400.jpg 400w, photo-800.jpg 800w, photo-1200.jpg 1200w"
sizes="auto"
loading="lazy"
width="1200"
height="800"
alt="A mountain landscape">
Using sizes="auto" on an image with loading="lazy" tells the browser to automatically calculate the size based on the actual layout width once it’s known. This means you don’t have to predict the rendered layout width ahead of time.
Safari 27.0 adds support for the shadowrootslotassignment attribute on declarative shadow roots. This lets you configure the slot assignment mode (named or manual) directly in HTML when defining a shadow root declaratively, matching the JavaScript attachShadow({ slotAssignment: "manual" }) option.
Environments in visionOS are an incredible part of the experience of Vision Pro. They let you transform your physical surroundings into a different place—like Yosemite, Mount Hood, or the Moon. It’s been possible for Apple developers creating immersive apps for visionOS to provide custom environments with their app. Now in Safari 27.0, environments can be provided as part of a website.
You can provide an immersive environment with a simple <model> element and one JavaScript API call. The Immersive API on the model element works similarly to how the Fullscreen API does on video elements. Learn all about it in Explore immersive website environments in visionOS.
By the way, this new Immersive API replaces the developer preview originally called Spatial Backdrop. If you built anything using Spatial Backdrop, migrate it to the Immersive API on the <model> element.
Now the <img> element has a controls attribute in HTML. It works just like the controls attribute on the video and audio elements. When present, the browser offers controls to allow the user to adjust or more fully experience the media.
<img controls src="panorama.jpg" alt="A panorama of the Dolomites" >
In Safari 27.0 in visionOS when the controls attribute is present, Safari provides a user interface for interacting with spatial and panorama photos. This gives users an easy and consistent mechanism to view photos spatially or immersively, and eliminates the need for web developers to build their own UI.
Safari 27.0 adds support for texture array projection layers in WebXR Layers. When creating a projection layer with XRWebGLBinding.createProjectionLayer(), you can now request textureType: "texture-array" so each eye’s view renders into its own layer of a single texture array.
Many websites inject content into the page as the user is reading or viewing that content. The new content often appears above where the user is currently looking — like images, ads, or comments being injected into the page. In the past, this caused the content the user was reading to be suddenly pushed down, causing a disorienting jump to a random place on the page.
Now with support for Scroll Anchoring, Safari 27.0 instead adjusts the scroll position and keeps the content exactly where it was before the content insertion. As a web developer, you don’t have to do anything to enable this on your site. It just works.
Scroll anchoring is controlled by the overflow-anchor CSS property, which defaults to auto. If you have a specific need where you need to opt out of scroll anchoring, you can use overflow-anchor: none.
Safari 27.0 adds support for using stretch with the properties width, height, min-width, max-width, min-height, max-height, and flex-basis. The stretch keyword tells an element to fill the available space in the relevant axis.
.card {
width: stretch;
}
It’s just like using width: 100% — but this time accounting for margins, which prevents overflow. If you’ve been using -webkit-fill-available to solve this need, now is a good time to switch.
Safari 27.0 makes three updates to anchor positioning, as the web standard evolves and the tool becomes more powerful.
First, we added support for transform-aware anchor positioning. Now, when an anchor element has a CSS transform applied — scale, rotate, translate, or any combination — elements positioned relative to that anchor follow its transformed position instead of its pre-transform layout position. This works for transforms applied via the transform property as well as through the individual translate, rotate, and scale properties. If you use anchor positioning to attach a tooltip, popover, or annotation to a transformed element, it now tracks correctly, even with animated transforms.
Second, the default value for position-anchor changes from auto to normal, fixing a potential side effect where the positioning behavior of elements that don’t even use Anchor Positioning could be impacted. The new value none opts out entirely. The new default, normal, behaves the same as none unless position-area is also set, in which case it behaves like auto did, as originally intended.
And third, Safari 27.0 also adds support for anchor-valid and anchor-visible . Originally, position-visibility: anchors-valid hid an element if any of its required anchor references couldn’t be resolved. However, it wasn’t clear what constituted “required anchor references”. So the CSS Working Group changed the behavior to only look at the default anchor box. To match, some keywords were renamed to drop the plurality. The anchors-valid value is now anchor-valid , while anchors-visible is now anchor-visible. Safari 27.0 aligns with the new behavior, and temporarily supports the old keywords for compatibility.
The new alpha() relative color function is a shorthand for adjusting just the alpha channel of an existing color, without repeating the rest of its channels: alpha(from var(--mycolor) / 80%). It keeps the origin color in its own color space and only changes the alpha value — useful when you want a more transparent or more opaque version of a color you already have, without writing out the full relative color syntax.
The color-mix() function now accepts more than two colors, so you can blend several colors together at once, like color-mix(in oklab, teal 20%, olive 30%, blue 50%). If you leave out the percentages, each color contributes equally.
The image(<color>) function lets you use a solid color anywhere an <image> value is expected. Unlike background-color, which sits underneath all background layers, image(<color>) behaves like a real image layer — it can stack above other background images, get sized with background-size, and be positioned and clipped like any image.
Safari 27.0 also adds support for forwarding missing color components when interpolating between analogous color spaces. Previously, a color with an intentionally missing component (none), like an achromatic gray with no meaningful hue, could get incorrectly assigned a hard 0 when converted into an analogous space for interpolation, producing a subtly wrong blended color. Now the missing component is carried forward as missing instead, so interpolation behaves the way you’d expect.
The light-dark() function now accepts <image> values, not just colors, so you can specify different images for light and dark color schemes in a single declaration: background-image: light-dark(url(day.png), url(night.png)). Gradients work here too.
Safari 27.0 adds support for the :heading pseudo-class, which matches any heading element — <h1> through <h6>. Instead of writing h1, h2, h3, h4, h5, h6 in your selector list, you can just write :heading. Plus, :heading also has a functional form for targeting specific levels, for example, :heading(1, 2) matches only <h1> and <h2>.
The revert-rule keyword is now supported in Safari 27.0. Like revert and revert-layer, revert-rule rolls back the cascade — but specifically to the state as if the current style rule had not been present. It gives you a more precise tool for working with overrides, especially in component libraries and design systems where you want to selectively undo declarations within a rule without losing the rest.
The CSS progress() function now supports a no-clamp option in Safari 27.0. By default, progress() returns how far a value sits between two bounds as a ratio from 0 to 1, clamped to that range. Adding no-clamp removes the clamp, so the result can fall below 0 or above 1, which is useful when you want an effect to keep scaling past its defined bounds instead of flattening out at the edges.
Safari 27.0 adds support for contain: style applying to CSS quotes. This allows you to scope effects of quotes to a certain subtree.
Safari 18.4 added support for text-autospace to control spacing between Chinese/Japanese/Korean (CJK) and non-CJK characters. Safari 27.0 now adds the insert keyword, making text-autospace: ideograph-alpha ideograph-numeric and text-autospace: ideograph-alpha ideograph-numeric insert equivalent.
The Dutch IJ digraph is now supported in Safari 27.0. When the content language is Dutch (lang="nl"), text-transform: capitalize and ::first-letter now correctly titlecase “ij” to “IJ” at the start of words.
Safari 27.0 adds support for the case-sensitive s modifier in CSS attribute selectors. Adding s after the value forces a case-sensitive match — for example, a[href$=".PDF" s] matches only a literal uppercase .PDF. This is the counterpart to the i modifier you may already be using to force case-insensitive matching (a[href$=".pdf" i] matches .pdf, .PDF, .Pdf, and so on); s lets you go the other way when you need an exact-case match on an attribute HTML would otherwise treat as case-insensitive.
Safari 27.0 also adds support for the :host:has() compound selector, letting a shadow host style itself based on what’s inside its own shadow tree. Because :has() can compound onto any selector, :host:has(:checked) or :host:has(::slotted(img)) let a custom element’s host change its own appearance depending on the state of its shadow content — useful for web component authors who want the host to react to what’s inside it without reaching for JavaScript.
Safari 27.0 adds the animation property to the AnimationEvent and TransitionEvent interfaces, letting event handlers directly access the Animation object associated with the event.
Safari 27.0 adds quite a few improvements to SVG.
Now the lang and xml:lang attributes are supported inside SVG. Use it to specify the language of text content to ensure correctness of both text rendering and accessibility announcements.
Safari 27.0 adds support for <use> referencing an external SVG file without a # fragment identifier. Previously, in order to point <use href="…"> at another SVG document, you had to name a specific element inside it with a fragment but now <use> can reference the external file on its own. There’s also a fix so <a> elements in SVG are treated consistently with HTML <a> elements for origin/security checks.
Several non-standard and legacy SVG interfaces have been removed to better align with the SVG 2 specification:
SVGLocatable and SVGTransformable interfacesnearestViewportElement and farthestViewportElement properties on SVGGraphicsElementviewTarget property on SVGViewSpecglyph-orientation-horizontal propertyPlus, there are a huge number of SVG fixes shipping this year. See the list below for what’s improved in Safari 27.0.
Safari 27.0 adds support for WebAssembly JavaScript Promise Integration (JSPI). JSPI lets synchronous-looking WebAssembly code suspend and wait for JavaScript Promises, making it much easier to port existing C, C++, Rust, and other language code to the web where that code expects synchronous I/O.
Before JSPI, porting code that called synchronous APIs to Wasm required rewriting everything on top of a callback or async state machine. With JSPI, the Wasm module can suspend at a call site and resume when the Promise resolves — the rest of the module sees straight-line synchronous code. This is a significant capability for the Wasm ecosystem.
Safari 27.0 includes a complete standards-compliant rewrite of the ECMAScript module (ESM) loader. The new loader is implemented in native C++ and conforms directly to the ECMAScript specification’s module loading algorithms, replacing an earlier implementation based on an abandoned 2016 WHATWG Loader proposal that predated top-level await entirely.
The rewrite fixes module execution ordering and initialization issues that could cause imports to access exports before they were fully evaluated. It was validated against test262, the Web Platform Tests, and additional test cases.
Top-level await is a foundational feature of modern JavaScript module authoring, and it’s been a real pain point in Safari for a while — a known source of cross-browser bugs that developers building module-based apps had to work around. This fix closes that gap. To learn more, read Fixing Top-Level Await in Safari.
Safari 27.0 adds support for the Service Worker static routing API. This lets a service worker declare routing rules that the browser can use to bypass the service worker entirely for certain requests, reducing overhead for high-performance PWAs.
Safari 27.0 adds three improvements to ReadableStream. First, the async iteration with for await...of:
const response = await fetch("/data");
for await (const chunk of response.body) {
process(chunk);
}
Second, the ReadableStream.from() static method for creating a stream from any async iterable or iterable:
Continued at the source.
Swift 6.4 is now available. Swift aims to be a great choice across the stack, from apps and servers to systems code, embedded devices, and the browser. This release deepens that support, and makes everyday code easier to write. Highlights include:
Span now bridging directly with C++20’s std::span, and Swift/Java interop extending its async and callback support.Iterable protocol for iterating without copies.There’s so much more. Read on for a detailed guide to the new changes, or see the Swift Evolution dashboard for the full list of proposals in Swift 6.4.
Swift 6.4 streamlines your day-to-day programming to make your code simpler and clearer.
some and any types. When writing an optional some or any type, you no longer have to wrap the type in parentheses. Instead of (some Rocket)?, you can simply write some Rocket? (SE-0521).@diagnose attribute (SE-0522).CommonThing, using the :: selector lets you clearly specify which of those you intend (SE-0491).withTaskCancellationShield API (SE-0504).You can combine asynchronous calls in defer blocks and cancellation shields to make sure that cleanup work always happens, no matter how the function returns:
func processFile(at url: URL) async throws {
let handle = try FileHandle(forReadingFrom: url)
defer {
// flushMetrics is a network call, so it can suspend after cancellation
// is requested; the shield ensures it runs to completion and isn't
// included in cancellation.
await withTaskCancellationShield {
await flushMetrics(for: url)
try? handle.close()
}
}
try await processContents(of: handle)
}
Improvements to Foundation and the standard library make it easier to use modern APIs with existing types.
For example, ProgressManager added API to provide async/await support (SF-0023), and @Observable types now have fine-grained and continuous change notifications (SE-0506).
The Subprocess library — originally introduced as SF-0007 and released as an initial 0.1 version in 2025 — has reached 1.0. It provides a cross-platform package to run and interact with subprocesses, built from the ground up using Swift concurrency. The following example, from Getting Started with Subprocess, illustrates running a process and capturing its output.
let result = try await Subprocess.run(
.name("ls"),
arguments: ["-la"],
output: .string(limit: 4096)
)
print(result.standardOutput)
Swift 6.4 makes it easier to migrate existing projects to use Swift Testing. You can now safely use XCTAssert in Swift Testing tests or #expect within XCTests (ST-0021), and customize the values shown in failed expectations using the CustomTestReflectable protocol (ST-0022). swift test lets you repeat test cases to focus and save time (ST-0024) and record attachments that conform to the Transferable protocol on Apple platforms (ST-0023).
Swift now has a documentation site, and the documentation content for the standard library is now open source.
Swift 6.4 brings a range of tooling improvements that make everyday development smoother, from debugging and building to editor support:
Swift’s interoperability expands its reach across more of the stack: from systems-level C++ to Android’s Java runtime, and from WebAssembly (Wasm) in the browser to Embedded Swift on microcontrollers.
Language interoperability goes deeper this release.
@c with @implementation to use a Swift function to provide the implementation for a C header with no separate C declaration. Without @implementation, the compiler emits the declaration into the generated header. Either way, @c functions can get safe wrappers, such as a function that uses Span in place of a raw pointer-and-count pair.std::span with Swift’s Span, so you can pass a Span to a C++ API that expects a std::span, and receive a std::span back as a Span, without writing manual conversion code at the boundary.Runnable mapping for closures, variadic parameter import, and support for Java record types.Swift’s platform support deepens as well.
JavaScriptKit has better performance when bridging to Wasm in Swift 6.4, with safe bridging up to 40 times faster than earlier dynamic bridging. The Wasm SDK is available from the Install Swift page of Swift.org, so compiling Swift for the browser requires no extra setup beyond adding the SDK.
Foundation updates for Swift 6.4 improve FileManager support on WASI (the WebAssembly System Interface).
Swift on Android continues to advance. This release of the Swift SDK for Android is built with the new LTS NDK 30, which provides Android availability attributes both in the Swift runtime libraries and for your Swift packages using the default NDK. Swift Build now supports Android in SwiftPM as well, removing the need for a post-install script.
The earlier post Embedded Swift Improvements Coming in Swift 6.4 covers Embedded Swift’s other improvements in this release in more depth, including generalized support for existential types (such as any Protocol), which lets you naturally express heterogeneous collections and throw and catch any Error.
Embedded Swift also gains a new EmbeddedRestrictions warning that you can enable across a whole target:
// Package.swift — enable EmbeddedRestrictions warnings for the target
.target(
name: "FirmwareCore",
swiftSettings: [
.treatWarning("EmbeddedRestrictions", as: .warning)
]
)
Swift 6.4 makes it easier to avoid unnecessary copies of your data while staying memory-safe, extending earlier work on Span, non-copyable types, and InlineArray.
Span or InlineArray through a property (SE-0507), non-copyable types can now conform to Equatable, Comparable, and Hashable, and new Ref and MutableRef types give you a first-class, storable container that lets you borrow or mutate one value at a time (SE-0519). Optionals of non-copyable types now work the same way, so you can inspect or update what’s inside an Optional without consuming it (SE-0532).UniqueBox gives you a smart pointer that uniquely owns a heap value, including non-copyable values, without reference counting (SE-0517). UniqueArray stores non-copyable elements without the copy-on-write allocations you would see when using Array and provides a buffer that grows dynamically (SE-0527). You can loop over elements and borrow them with the Iterable protocol, instead of copying each value, which extends beyond what the Sequence protocol supports (SE-0516).withTemporaryAllocation provides a scratch buffer that is automatically initialized and cleaned up (SE-0524). A new safe loading API lets RawSpan and its variants load and store bytes safely, replacing the unsafe-flagged functions (SE-0525).Swift 6.4 reflects the contributions of many people across the Swift community, through code, proposals, forum discussions, and feedback. The community’s thoughts and real-world experience provide invaluable insights and motivation!
If you’d like to get involved in what comes next, the Swift Forums are a great place to start.
Try out Swift 6.4 today by following the instructions on the Install Swift page, or download the new 6.4 toolchain with Swiftly.
When your Swift program hits a breakpoint and stops so you can inspect it, the debugger’s expression evaluator has to find the exact Swift module your code was built from. Until now, that lookup wasn’t always precise. The upcoming Swift 6.4 release will include changes, begun in Swift 6.3, that address this by updating how the Swift compiler references explicitly-built Swift modules in debug info.
The majority of developers will automatically benefit from faster, more reliable debugging and smaller build products, without any modifications to their SwiftPM or Xcode projects.
For developers who maintain their own build systems using, for example, Bazel, Buck, or CMake, some adjustments may be necessary to take advantage of these changes.
This article explains how the debugger uses Swift modules. Next, it describes how Swift 6.3 changes the way modules are tracked in debug info to solve several problems with the previous representation. Finally, it shows how to adjust build systems to take advantage of the new representation and eliminate some build steps that are no longer necessary.
LLDB’s standout feature is its powerful expression evaluator. Because LLDB embeds the Clang and Swift compilers, it can JIT-compile any valid source code and run it in the context of your application while stopped at a breakpoint. This includes not just calling code in your application, but also defining new data types, functions, and closures. Debugging features that are usually reserved for interpreted or JIT-compiled languages like JavaScript become available to ahead-of-time-compiled languages like C++ and, of course, Swift!
In order to JIT-compile user expressions that make use of data types defined in the debugged program, LLDB’s embedded Swift compiler needs to import the Swift modules defining those types. In a world before explicitly-built modules, LLDB would find the base name of the main module at the current breakpoint in the debug info and then kick off an implicit import of a module with that name. With a cold module cache this would launch an expensive compilation of that module and all its dependencies.
To illustrate this, let’s walk through a simple example:
(lldb) p myObj
Here myObj is just a local variable: LLDB can find its location in the debug info and resolve its type via reflection metadata. No need to bother the Swift compiler.
Let’s make it more complex:
(lldb) p myObj.myComputedProperty
In this case, myComputedProperty is really a function call; in order to evaluate this, LLDB needs the expression evaluator to run code in the target. In order to initialize a Swift compiler instance with the state of the current module, LLDB finds the name of the current function’s Swift module in debug info.
We can visualize what LLDB does using the dwarfdump utility:
$ dwarfdump Foo.o
...
DW_TAG_module
DW_AT_name ("Foo")
Conceptually, LLDB then wraps the expression in a function that can be compiled:
(lldb) log enable lldb expr
(lldb) p myObj.myComputedProperty
...
import Foo
func lldb_expr(_ $__lldb_arg : UnsafeMutablePointer<Any>) {
let myObj: MyObject = /* some LLDB magic */
// Expression begins here:
myObj.myComputedProperty
...
One problem with this is that import Foo is quite imprecise: Even though the Swift language doesn’t allow multiple modules to have the same name, even the most stringently engineered application may have more than one copy of the same module. For example, there might be a private version of a module containing all of its private declarations (which would be great for LLDB) and also a Swift interface file that only contains the public interface for the module. Or there might be macOS and Mac Catalyst variants of the same module in the same process.
Let’s look at where those modules are found next. In order to communicate the location of Foo.swiftmodule to LLDB, Swift build systems rely on some cooperation from the linker. On Darwin the system linker accepts an option called -add_ast_path and build systems are expected to specify this option to list every binary Swift module when linking.
# Linker invocation on macOS
ld -add_ast_path /path/to/Foo.swiftmodule Foo.o -o MyApplication
The linker translates these options into symbol table entries. The debug info linker dsymutil then collects all Swift modules and stores them in a special __swift_ast section in the dSYM bundle, where LLDB can find them by name. Alternatively, when debugging without dSYM bundles, LLDB reads the symbol table entries in the binary to collect a list of all binary Swift modules.
Such an approach would not work on platforms where the linker isn’t aware of Swift. For these platforms, which include Windows, Linux, and FreeBSD, the Swift compiler provides a -modulewrap action that takes a binary Swift module and outputs an object file with a .swift_ast section holding the contents of the module. This object file can then be passed to any linker to get added to the binary, where LLDB can find it.
# Modulewrap and linker invocation on Linux
swift-frontend -modulewrap Foo.swiftmodule -o Foo.swiftmodule.o
lld Foo.o Foo.swiftmodule.o -o MyApplication
This can create scalability issues, especially for large applications:
To evaluate expressions, the debugger needs to be able to find and import Swift modules. Until now, this relied either on special linker support or additional compilation steps, with a high cost for binary size. On top of that the debugger was imprecisely locating Swift modules by name.
Starting in Swift 6.3 and continuing since, we have been making changes to the Swift compiler, the Swift driver, and LLDB that improve performance, reliability, and scalability. These changes are built on top of explicitly-built modules.
Explicitly-built modules track their explicit Swift dependencies: Explicitly-built binary Swift modules have always kept track of their explicitly-built Clang module dependencies. This is why LLDB can import explicit modules so much faster than implicit modules, which may need to recompile their dependencies from source. In Swift 6.3, explicitly-built binary Swift modules also keep track of their Swift module dependencies. This makes importing an explicitly-built module fast and unambiguous because no module needs to be looked up by name. This happens automatically. Users don’t need to make any changes. Users with distributed build systems will already be familiar with the Swift frontend’s path remapping options, which now also affect Swift module paths.
Debug info stores path of object file’s own Swift module: Once LLDB finds the top-level module it can precisely import it and all of its dependencies. But how can LLDB find precisely the module that belongs to the Swift file at the current breakpoint? In Swift 6.3, the Swift compiler can store the path to it in the debug info. Because a Swift file’s own Swift module is not an input to an object file compilation, there is a new -debug-module-path compiler option to communicate the path to each object file compilation action. This path is also subject to the standard path remapping options used by users with distributed build systems.
Swift driver passes module path to compile jobs: Users of swiftpm or Xcode do not need to think about this, because the Swift driver also knows about the new -debug-module-path option and automatically passes the path to the object file’s own Swift module to the compiler. However, users maintaining their own third-party build system to orchestrate Swift compilations with explicitly-built modules that are calling the Swift frontend directly and bypassing the Swift driver need to make sure to communicate the path to the top-level module to each object file compilation job.
Beginning in Swift 6.4, you can safely make the following changes.
swiftc -modulewrap and ld -add_ast_path: Because the module paths are now communicated via debug info and the module headers themselves, third-party build systems doing explicit module builds can now remove all -modulewrap actions on Linux and Windows; and remove the use of the -add_ast_path linker option on Darwin (macOS, iOS, etc…).
Binary Swift modules in dSYM bundles: As a consequence, dsymutil will no longer process binary Swift modules. This is a good thing, because binary Swift modules—which can only be parsed by the exact toolchain that produced them—were always at odds with dSYM bundles being a long-term archival format. Moreover, Swift modules often depend on Clang modules, and these Clang modules also were never included in dSYM bundles. By removing the binary Swift modules, dSYM bundles will get smaller.
po. Binary Swift modules are still needed to evaluate complex expressions like function calls or computed getters. Expression evaluation continues to work as long as LLDB finds all binary modules in their original (or remapped) location. This is always the case when debugging a just-built binary on the same machine. If the absence of binary Swift modules in dSYM bundles creates an unforeseen problem with your workflow, please let us know, either on the Swift LLDB forum or by creating an issue on the bug tracker.When compiling with caching enabled, all paths pointing to Swift modules and module debug info are content-addressable storage references, identified by content rather than file location, so everything described here also works transparently with compilation caching.
Beyond more reliable path tracking, Swift 6.4 will also speed up importing bridging headers, a step common enough across Swift projects that most developers will feel the difference.
Up to and including Swift 6.3, LLDB always compiles a bridging header from source, a step that can add noticeable time to debugging sessions that use one. In recent nightly development toolchains, LLDB can use the new precise explicit module information to import precompiled bridging headers and their explicit module dependencies directly. This makes debugging explicitly-built projects with bridging headers as fast and reliable as debugging fully modularized projects.
With these changes for explicitly-built modules:
-modulewrap actions and remove -add_ast_path from the linker flags, but may need to pass -debug-module-path to the compiler if they are not letting the Swift driver handle the frontend options-add_ast_path when linking them often had confusing debugging issues inside those archives as a result. This entire class of issues has been designed away.tl;dr: -modulewrap and -add_ast_path are replaced by -debug-module-path. Debug info gets smaller and more precise.
We decided to go all-in on React Native back in 2020, and that bet has been extremely successful. We saved a ton of time building features just once, enabled developers with no mobile background to contribute to our apps, and freed ourselves from constantly chasing feature parity.
In January 2025, I wrote that the future of React Native was bright and that Shopify planned to keep investing in it. That was true based on what we knew then. React Native was working well for us, and it remains an excellent framework. But since then, coding models have gotten dramatically better, and for our apps and our team, building the same feature in Swift and Kotlin no longer carries the cost it used to.
We don’t hold on to a decision just because it was successful at the time. When a core assumption changes, we’re willing to go back and ask whether it’s still the right call. LLMs changed one of the core assumptions behind our 2020 decision, so we reevaluated our mobile stack from first principles.
What we found led us back to native.
We decided to switch from native to React Native in 2020 for three reasons:
React Native consistently delivered these benefits. We found ourselves spending a significant amount of time and resources on optimizing performance, improving key foundational areas in React Native, and keeping up with framework updates and external dependencies, but these were acceptable tradeoffs. The benefits of using React Native far outweighed the investments we had to make in these areas.
Shopify has been using LLMs to build software since 2021 (one year before ChatGPT!). Initially, we used them to implement features, investigate and fix bugs, and review code. As the models improved, so did the complexity of the work we trusted them to take on. By late 2025, they were no longer just helping us write code faster. They were capable of making us question whether building software twice still meant doing twice the work.
We decided to reevaluate our mobile tech stack and started prototyping to see whether our technology choices still held up. We rebuilt several core parts of our biggest apps in Swift and Kotlin using LLMs and were surprised by how well it worked. Agents:
Native still means building and maintaining software on two platforms, that cost has not disappeared. What changed is that agents can now do enough of the implementation, translation, testing, and review work that it’s no longer the deciding factor it was in 2020.
React Native apps can be fast. Ours are. We are making this change because agents have reduced the advantages of sharing implementation, while the advantages of building for each platform remain. Native keeps us closer to platform capabilities and first-party tooling, with fewer framework and dependency layers between our code and the platform.
Before we get into how we’re migrating, we want to make sure we do this transition cleanly. From the beginning, we wanted to contribute back to React Native to make it better. We’ve published open-source libraries that have become the top choice in their respective categories. We’re grateful for the incredible reception from the community and are committed to making sure this is a smooth transition with no surprises.
Shopify will continue sponsoring this through the end of 2026, and William Candillon will continue working on it beyond that. He will fork the repo in the coming months and start publishing the library under a new name. The original repo will be archived when this transition is complete. We’ll post updates along the way so that everyone has ample time to migrate. If your app relies on this library, please consider sponsoring it.
This library gets ~2M downloads/week and has become the default way to render high-performance lists in React Native. Given how important it is for the ecosystem, Shopify will continue to fix critical issues that break compatibility. We’re currently in discussions with several companies about taking on long-term stewardship of FlashList. If you’re interested, reach out to me here.
Restyle has a smaller user base than our other libraries, so we're archiving this repo. We'll keep it working through the end of 2026, then stop maintaining it. Anyone is welcome to fork it and take it forward, and we'll help with the handover if a team wants to pick it up.
Shopify has several large apps (Shopify, Shop, Point of Sale, Inbox). Millions of merchants and buyers around the world rely on them every single day to earn their livelihood and buy products they want from the brands they love.
We debated between gradually migrating to native (brownfield) versus rebuilding them from scratch (greenfield). In the past when we migrated to React Native, we picked the brownfield approach for some of our biggest apps, as it’d take years to rewrite them and we’d have to stop shipping new features while the rewrite was in progress.
However, this time greenfield emerged as a clear winner for the following reasons:
The Shop app, which is regularly at the top of the list in the shopping category in the app stores, is the first to be migrated. Assisted by AI, the team was able to go from a proof of concept to a fully rebuilt native app published in the app stores in just 12 weeks. We’ve written about this migration in depth here.
The migration of the Shopify app (our biggest with 300+ screens, home & lockscreen widgets, Apple Watch app, complications, Siri Shortcuts, etc.), is also underway and will ship later this year. The rest of our apps will be migrated soon.
It’s tempting to just point an LLM to the React Native codebase and try to one-shot the same features in native, but it doesn’t work. Even if you ask it to gather as much information as it can up front, freeze that into specs, task files, and then implement it, you end up with a huge amount of unmaintainable code that can’t be shipped.
To solve this problem, we built a system called Helix that takes a more gradual approach. It doesn't expect the first output to be correct, and builds a loop where an imperfect attempt simply cannot move forward until it becomes a good result.
The developer points Helix at a screen. Helix reads the React Native code and proposes a sequence of checkpoints (small, ordered slices of the work) that can be reviewed in minutes. Then, checkpoint by checkpoint, it builds: each one must prove its behavior with tests, match the running app in a visual review, survive two adversarial code reviewers, and get a human's nod before it's committed and the next one starts. Feedback from every review is remembered, so the loop gets more autonomous as the migration progresses.
Helix rebuilding a screen in the Shopify mobile app using Swift and Kotlin
This approach has been working extremely well and is allowing us to rebuild our apps in a fraction of the time.
Agentic control of simulators has been a bottleneck. We found ourselves constantly babysitting them as they couldn’t reliably build, test, and iterate. We built tooling to allow agents to reproduce bugs, fix them, and verify the fix autonomously but it was slow and brittle. React Native’s hot module reload helps the situation but it doesn’t solve it, due to simulator control being slow. This is primarily due to reliance on the accessibility tree, or screenshots to get the state of the app, take actions, and verify results. Agents can make code changes in seconds, but it takes them several minutes to test the output. This makes iterating extremely slow and manual. It doesn’t matter how good the model is if it can’t test its work quickly, which is especially difficult on mobile.
We’re fixing this by designing our app architecture to work for both humans and agents. The core principle here is that business logic should be completely decoupled from the UI and be able to run headlessly on desktop. We then make it available to agents via a CLI that allows them to iterate on it in milliseconds instead of minutes without involving simulators.
Navigating the app and performing actions using the CLI
The CLI allows agents to inspect the state of the app, navigate between different sections, and perform actions all without needing to touch the UI. This enables extremely fast feedback loops and allows agents to work autonomously for hours at a time.
When simulator interaction is needed, the CLI can connect to them via a remote mode and drive the UI via commands without having to inspect the layout or the accessibility tree. This enables blazing-fast performance and E2E tests.
This is real-time (not sped up)
We are going to migrate all our mobile apps to Swift and Kotlin using AI throughout the process. Shop has already shipped as a fully native app, the Shopify app is underway, and the rest will follow soon. We’re moving quickly, but not by lowering the bar. Every rebuild must meet or exceed the performance, stability, accessibility, and product quality people expect today. This isn’t just the same apps rewritten in different languages. We’re rebuilding them so both humans and agents can understand, test, and change them quickly.
The migration isn’t the finish line. Success means our teams can deliver better experiences for merchants and buyers faster than before. We’ll measure that through product velocity, app quality, and how much work agents can complete autonomously.
We’ll share what we learn along the way, including deeper dives into Helix, our agent-addressable architecture, and how we’re building mobile apps with agents. We were open about what we learned from React Native, and we intend to be just as open about this transition.
This is one of the most ambitious mobile engineering projects we’ve taken on. If you want to help build the next generation of Shopify’s mobile apps, we’re hiring mobile engineers, infrastructure engineers, and developers working at the intersection of AI and software engineering.
Native is the right choice for Shopify now, but React Native was the right choice for Shopify in 2020. That success was only possible because of the people who made it work.
Thank you to the React Native team at Meta for being excellent stewards of the framework, listening to our feedback, and working closely with us over the years. React Native is substantially better today because of your investments in its architecture, performance, tooling, and community.
Thank you for creating React Native Skia and taking it much further than any of us imagined. You redefined what was possible for graphics and animation in React Native, and we’re excited to see where you take it next.
Thank you for all your work on Reanimated, for listening to our feedback, and for helping us solve some of the hardest animation and performance problems in our apps.
Hundreds of engineers contributed to adopting React Native, migrating our apps, building shared foundations, improving performance, maintaining integrations, and contributing back to the ecosystem. Many of you became beginners again, challenged long-held assumptions, and made the transition successful while continuing to ship for merchants and buyers. Thank you.
Thank you to everyone who used our open-source libraries, contributed code, reported issues, challenged our decisions, and shared what you learned. Your contributions and feedback, including the spicy kind, made our work better.
The tools, lessons, and relationships built over the past six years will continue to shape how we build mobile apps at Shopify. We’re deeply grateful to everyone who was part of it.
Big one today — Tailwind is joining Shopify.
When I started working on Tailwind over nine years ago, my only goal was to create something that would make it easier to build beautiful interfaces for my own projects. Fast-forward to today and the framework is installed over 110 million times per week and is trusted by many of the world's biggest companies to style products like ChatGPT, X, Cloudflare, Reddit, and Shopify.
We're joining Shopify to give Tailwind a stable long-term home where it will be actively maintained for the millions of people who depend on it.
We built a great little website template business around Tailwind over the years, but deep down I've always wanted the framework to be developed in service of a real product. A complex application solving important problems for real people, where we'd face the same challenges as our users, and could invent solutions that make the framework better for everyone.
Shopify provides an incredible surface area for us to do this work. Merchants need to be able to design and host beautiful custom storefronts, and manage sales and inventory in a powerful admin area. Their customers need delightful shopping and checkout experiences, and an intuitive way to keep track of their orders and discover new products through the Shop app. Shopify is also on the frontier of where user interfaces need to go next with their explorations into agentic commerce.
Shopify was also one of the very first companies operating at scale to see the potential in Tailwind CSS and start building with it, not only for themselves but betting on it for their customers too. Tailwind is a load-bearing very important part of the stack at Shopify, and they're invested in making sure it's actively maintained and continues to improve and adapt for how the ways we build are changing.
On a less technical note, I'm personally excited because entrepreneurship has completely changed my life. We are not doing enough as a society to produce and empower more entrepreneurs, and I believe deeply in Shopify's mission to help more people start, run, and grow their own business.
Nothing changes with Tailwind CSS or any of our other open-source projects. Everything will always be MIT-licensed, and our team will continue to lead and maintain these projects for the community with the support of Shopify.
On the commercial side, we'll no longer be trying to grow the business around Tailwind. All existing customers will of course maintain their access to products like Tailwind Plus and ui.sh, but we're closing sign ups for new customers to focus on Tailwind CSS at Shopify.
Thank you so much to everyone who has built something with Tailwind and supported us over these last nine years. I never could've imagined the project would become what it has today, and I truly believe there's no better place for us to continue to do this work than Shopify.
The Kotlin 2.4.20 release is out! Here are the main highlights:
kotlin.test assertion functions.Package.swift files for SwiftPM dependencies.require() calls in @JsFun declarations, improved companion object initialization order, and support for Wasmtime in the Kotlin Gradle plugin.For the complete list of changes, see What’s new in Kotlin 2.4.20 or the release notes on GitHub.
The latest version of Kotlin is included in the latest versions of IntelliJ IDEA and Android Studio.
To update to the new Kotlin version, make sure your IDE is updated to the latest version and change the Kotlin version to 2.4.20 in your build scripts.
If you need the command-line compiler, download it from the GitHub release page.
If you run into any problems:
Stay up to date with the latest Kotlin features! Subscribe to receive Kotlin updates by filling out the form at the bottom of this post. ⬇️
Welcome to “What’s new in Swift,” a curated digest of releases, videos, and discussions in the Swift project and community.
Here’s an update from guest contributor Simon Leeb on Swift’s progress as a language for web scenarios:
Hi, Simon here! I am the creator of the elementary-swift project, a collection of packages born from a simple wish: I want to build web UIs in Swift and ultimately help Swift become a first-class choice for the web.
This journey began after I started using Swift for backend services. The web frontend, however, still lived in a separate ecosystem, and I really wanted it to feel as ergonomic, safe, and efficient as the Swift I was writing everywhere else.
That led to the creation of Elementary: a modern and efficient HTML rendering library with a familiar declarative API, built for the web. It integrates easily with frameworks like Vapor and Hummingbird, and has become a practical option for server-rendered web UIs.
Around that same time, years of community work in the swift-wasm project made compiling Swift to WebAssembly increasingly viable, while Embedded Swift was taking its first experimental steps. This made me wonder: “How hard can it be to use Embedded Swift and build a state-driven web UI framework that produces tiny WebAssembly binaries?” Turns out: quite hard, actually!
But it was too late. Despite my better judgment, I was in the middle of creating what is now known as ElementaryUI. Where Elementary renders HTML on the server, ElementaryUI runs in the browser itself. You can watch my talk at Swift@FOSDEM 2026 if you want to know more about the why, what, and how.
To showcase where the project is heading, I recently posted a small Full-Stack Swift on Cloudflare demo. It features Swift in the browser communicating with a Swift backend on an edge worker through shared message types. I hope it gives people a concrete sense of how much the core technologies and the surrounding tooling have advanced.
ElementaryUI is still young, with plenty left to build. Visit elementary.codes to try it, share feedback, contribute, or sponsor its development. Let’s work together and make Swift a first-class choice for the web!
Now on to other news about Swift:
Span and non-copyable types, plus how to audit unsafe code with strict memory safety and incrementally migrate existing C modules.RawRepresentable and LosslessStringConvertible conformances? lexic generates them for you via macros, and runs the same on Linux as on Apple platforms.json_each table function, so nested data can be queried and updated by key path without leaving Swift’s type system.The Swift project adds new language features through the Swift Evolution process. These are some of the proposals currently under review or recently accepted for a future Swift release.
Under active review:
Issue.record call. This proposal adds structured fields, including error, confirmationMiscount, exceededTimeLimit, and expression, so tools like Xcode and VS Code can show richer, more specific failure information.Recently accepted:
deinits - Non-copyable types that manage a resource, like a file handle or buffer, often need to run the same cleanup logic in their deinit that they use elsewhere, but until now self inside a deinit could only be borrowed, not mutated or consumed. This proposal lets a deinit mutate or consume its own stored properties directly, so existing cleanup methods can be reused instead of duplicated.Attachment/Encodable interop - Swift Testing lets you attach extra data, like a screenshot or JSON snapshot, to a test for inspecting after a failure, but attaching custom types previously required extra setup code and offered no way to choose the encoding format. This proposal adds new Attachment initializers that let you attach Encodable or NSSecureCoding values directly, picking the format or supplying your own encoder.Recently accepted with modifications:
/search endpoint to the registry specification and a swift package-registry search subcommand, letting you find packages by name, scope, author, and other criteria, with support for qualifiers like author:"Mona Lisa Octocat" and searches that span every configured registry at once.Iterable - Looping over a collection in Swift traditionally means copying out one element at a time, which doesn’t work for newer types that can’t be copied, like Span and InlineArray. This proposal introduces Iterable, a new way to loop over data without copying, and was renamed from BorrowingSequence and given support for typed throws before acceptance..taskLocal(_:_:) trait that does it in one line, like @Suite(.taskLocal(FeatureFlags.$isEnabled, true)).Kotlin Toolchain 0.12.0 is out. This release brings some long-awaited features: multiplatform libraries publication, a preview of Wasm application support, Compose Hot Reload from the command line, and more.
Read on for the details, and check the release notes for the full list of changes and bug fixes.
Additionally, klibs.io now uses the Kotlin Toolchain in production. A real backend and not a sample, it’s built on JDK 21, Spring Boot 4 (with Spring AI), PostgreSQL, and OpenSearch. We’ve converted nine convention plugins to Kotlin Toolchain templates, and two Gradle plugins with no built-in equivalent: Jib and Git Properties, which we’ve implemented as local Kotlin Toolchain plugins. Check out the sources yourself.
To get support for Kotlin Toolchain’s latest features, use IntelliJ IDEA 2026.2.1 (or newer). Make sure the latest version of the Kotlin Toolchain plugin is installed.
Library publishing arrived in preview in 0.11, but only for JVM libraries. Starting with 0.12, multiplatform libraries work too, with exactly the same configuration:
product:
type: lib
platforms: [jvm, android, iosArm64, iosSimulatorArm64, wasmJs]
settings:
publishing:
enabled: true
group: org.example
version: 1.0.0
The Kotlin Toolchain publishes everything your users need to depend on your library from any of its targets: the common API, one artifact per platform, the sources, and the module publication metadata that lets build tools pick the right pieces automatically.
Cinterop bindings are supported as well. They are published both commonized and per platform, so your users get the same C API you compiled against without setting up interop themselves. The result is consumable from Gradle projects like any other multiplatform library.
For more details, see the documentation.
Note: Resources of Compose Multiplatform libraries are not part of the publication yet. Follow KTC-5698 for progress.
Because of the new quotas on Maven Central publications that Sonatype will soon enforce, we made a few notable changes to reduce the number of files published by default:
.asc.sha1) are not necessary and are no longer published..md5 and .sha1 checksums are published by default now. If you need to continue publishing the .sha256 and .sha512 checksums, use settings.publishing.checksums: [md5, sha1, sha256, sha512].wasm-js/app modules can now be built into a ready-to-use web application.
Among the supported features are:
kotlin run command.index.html and other resources. More information on working with Wasm web applications is available in the documentation.
We are actively working to make the output of the kotlin command less verbose and more user-friendly.
For example, here are some of the recent diagnostics improvements:
Running tests are now visible in the status widget under the respective tasks and their suites. There are also short test execution statistics visible during the run.
There are more things to iron out, but we’ll get there.
Android modules and kmp/lib modules that have Android as one of their targets now support the Compose preview feature, powered by the androidx.compose.ui.tooling.preview.Preview annotation and the Android plugin.
The IDE now correctly recognizes Compose resources, updates Res classes on the fly, provides navigation, completion, and refactorings that update both XMLs and your code.
Adding to the Compose preview support mentioned above, we have also brought support for more of the Android features you are accustomed to, such as:
R class) navigation and completionStarting with IntelliJ IDEA 2026.2.1, the experience of working with iOS applications should be closer to what you’re used to in Gradle projects.
The run configuration now lets you pick a device, configure Xcode options, and choose a debug/release configuration mode.
We’ve also fixed a few issues with Kotlin/Swift interoperability, which should be more stable now.
Catalog dependencies in module files and templates now have an inlay hint next to them displaying coordinates that each entry points to.
We now properly support Compose Hot Reload from the command line using the kotlin run --compose-hot-reload-mode command.
We now support an MCP server for agents to interact with applications running with Compose Hot Reload.
To get started, add the following snippet in your mcp.json:
{
"mcpServers": {
"Compose Hot Reload": {
"command": "./kotlin",
"args": [
"compose-hot-reload-mcp-server"
]
}
}
}
With this, agents can interact with, reload, restart, and view window snapshots, and dump the tree of composables. Read more about these capabilities here.
// prefixPreviously, the only way to define local module dependencies was to use relative paths starting with the . (dot) symbol. This approach had several problems. For example, moving a module from one directory level to another required changing all the dependency paths, such as from../../foo to ../foo. And having a multitude of ../ in deeply nested directory structures generally made paths hard to read.
The new recommended way to define local module dependencies is to use project-root-relative paths starting with the // prefix. You might be familiar with this syntax from tools like Bazel. The // prefix represents the project root directory and can be used not only in the dependencies block but in any place that expects a path as well, for example, apply.
The old relative-paths approach still works for now.
This is a step toward allowing multiple modules with the same directory name.
Until now, the minimum JDK version supported by the Kotlin Toolchain was not clearly documented anywhere, and the build would just fail in different places if you used a JDK that was too old. There is now a clear diagnostic and a clear minimum: only JDK 17 and higher are supported to compile your code. You can still use settings.jvm.release to set a lower target if your code should be runnable on lower JREs.
The minimum Kotlin compiler version was raised from 2.1.10 to 2.2.20. This allows simplifying our code, and is in line with the new security support policy for the Kotlin standard library.
We’ve also updated some of the default versions for built-in toolchains and frameworks:
To get started with the Kotlin Toolchain, check out our Getting started guide. Take a look at some examples, follow the tutorial, or read the comprehensive user guide, depending on your learning style.
To update an existing project, use the kotlin update command.
The Kotlin Toolchain is still in Alpha and under active development. You can provide feedback about your experience by joining the discussion in the #kotlin-toolchain Slack channel (get invite: https://kotl.in/slack) or by sharing your suggestions and ideas in a YouTrack issue. Your input and use cases help shape the future of the Kotlin Toolchain!
This month, Svelte 5.57 shipped with new SvelteMap methods and a few quality-of-life additions while SvelteKit 3 got closer to the finish line with its Release Candidate.
The sv CLI also got a new ai-tools add-on that replaces the old mcp one, and sv@next now ships a task-based sveltekit-3 migration for existing apps.
Let’s dive a bit deeper!
SvelteMap now has getOrInsert and getOrInsertComputed methods for the common “read or initialize” pattern (5.57.0, Docs, #18728)createContext now returns a third has function so you can check whether a context has been set without triggering the get error (5.57.0, Docs, #18472)<select> now supports the defaultValue attribute, so the select reverts to that value on form reset (5.57.0, #18591)svelte/server now exports the RenderOutput, SyncRenderOutput, Csp and Sha256Source types for typing server render output and CSP sources (5.57.0, #18648)For the full list of patches and bug fixes, see the Svelte CHANGELOG.
Prereleases have kept rolling on the @next line this month, adding a few more features and refining the surface:
pre and post groups so adapters can run transforms at the right point in the pipeline (3.0.0-next.18, breaking, #16711)+ prefixes are now ignored during routing if their names contain test, spec or stories, so colocated tests don’t accidentally become routes (3.0.0-next.19, #16715)logLevel and customLogger (3.0.0-next.20/25, #16744, #16858)defineParams and the associated types have moved to @sveltejs/kit/params (3.0.0-next.19, breaking, #16716)+server.js files can now export a QUERY HTTP method handler (3.0.0-next.24, #16782)preload filter for fonts now receives the project-relative source filename so you can filter by directory or component (3.0.0-next.24, #16443)applyReroute helper for split serverless function deployments (3.0.0-next.25, #16665)For the meantime, all of the SvelteKit 3 docs live on next.svelte.dev, and the full changelog is on the version-3 branch. The stable 2.x line kept moving too with three patch releases (2.70.1, 2.70.2, 2.70.3) - see the SvelteKit CHANGELOGs for the details.
mcp add-on has been replaced with a broader ai-tools add-on that can set up the Svelte plugin (Claude Code, opencode) or pick individual tools (MCP server, skills, sub-agents) per client (sv@0.17.0, Docs, #1050)@sveltejs/sv-utils picks up isKit3, resolveLibPrefix and libSubpathImports helpers so add-ons can transparently handle SvelteKit 2 and 3 (sv-utils@0.3.3, #1199)sv migrate has been reworked to prepare for the SvelteKit 3 migration - it now lists tasks, ships a sveltekit-3 task that bumps dependencies and rewrites $lib to #lib, adds an $app/state task, delegates the older migrations to svelte-migrate@1, and creates a list of changes a developer or agent should resolve if they cannot be migrated automatically (sv@1.0.0-next.0, Docs, #1138, #1241, #1249)#lib (Node subpath imports) instead of $lib, matching SvelteKit 3 (sv@1.0.0-next.0, #1185)experimental add-on is now scoped to enabling experimental features, and the versions option value was renamed to kit-3 (sv@1.0.0-next.0, breaking, #1241, #1185)dynamicCompileOptions argument for the current Vite environment, useful when you want to compile differently for the client, server or SSR environment (vite-plugin-svelte@7.3.0, #1386)svelte2tsx and svelte-check learn about SvelteKit 3’s flattened config structure so the type generation and diagnostics keep working through the migration (svelte-check@4.7.6/svelte2tsx@0.7.61/svelte-language-server@0.18.4, #3104, #3106)Want to dive deeper? Check out the Svelte CLI, language-tools and ai-tools releases.
.odt and .docx documents that runs entirely in the browser or as a PWAThis Week in Svelte
To Read
UI Components and Design Systems
Animations and Icons
Frameworks and Dev Tools
That’s it for this month! Let us know if we missed anything on Reddit or Discord.
Until next time 👋🏼!
Today we published the first release candidate for Remix 3.
The last time we shared updates about Remix 3 was when we announced our beta preview 4 months ago. We're incredibly proud of what we've built and all the improvements we've made since our first beta, and we're excited to start sharing with you why exactly we think Remix is so awesome.
This post barely scratches the surface of everything we've packed into Remix 3: database management, schema validation, a fast, type-safe router, an unbundled asset server, and a brand-new UI runtime complete with composable event handling, styles, animations, and built-in components. Remix is more capable than it has ever been, and it's all in a single remix package.
We're going to dig into all the details in the coming weeks and months, and we'll release Remix 3 on October 2 at Remix Jam (tickets are still available).
If you've ever planted a tree, you might have heard an adage that goes something like this:
The first year your tree won't grow a whole lot. That's because it's rooting, adjusting to the new soil, and building a strong foundation for growth. In year 2, it's going to suddenly take off, doubling, even tripling in size.
At least that's what I was told.
The journey of Remix 3's development has been a lot like planting a tree. Last year at Remix Jam, we metaphorically invited you into our "garage" to hear our demo tape. We talked about the ideas we love from React and React Router (previously Remix v2). We also talked about the things we don't love so much about those projects, and where we want to take Remix. With Remix we want to provide a fully stacked web framework, unapologetically built on web primitives and productive out the gate in an agentic programming landscape.
From that point up until we released our first beta, we were focused on establishing and rooting all of those ideas and APIs. We were giving every piece its proper place, making sure the packages would work well together and that the majority of them could be used independently.
Since that beta, Remix has grown a lot:
router.mount(), and improved TypeScript inference.remix.json for configuring databases, assets, tests, and remix doctor, plus CLI tooling for inspecting browser-reachable assets.And that just scratches the surface. We've also made numerous bug fixes and stability improvements across 350+ commits this summer (or winter for Mark).
We have been a lot quieter about Remix than we like or intended. It's not for lack of excitement about Remix. We have genuinely struggled to wrap exactly what is exciting about Remix into simple, straightforward explanations that we can deliver in quick videos, blog posts, demos, etc.
This is largely for 2 reasons:
This has meant that we've defaulted a lot more to building than to showing. But no more.
We've been putting Remix through its paces, particularly by migrating this very website as well as the Remix Store (works on my machine, I really should deploy it). We have been gathering feedback from early adopters, both externally and internally here at Shopify, some of whom you may or may not hear from at Remix Jam.
Overall, our best pitch for Remix is that we like it. Everyone is trying to pitch that they're building for agents. No one really knows what that means. We built this for ourselves, and we use agents. We use agents a lot. We're the most biased of the bunch, but we're finding again and again that:
You shouldn't (and really can't) take our word for it, though. Try it out. Build something. And if you don't find Remix to be as grokable as we say, we still have 1 trick up our sleeve. The default remix template package.json looks like this:
"dependencies": {
"remix": "3.0.0-rc.1"
},
With Remix, you can build truly full-stack applications, not merely a backend-for-frontend (though you can do that) or a server-rendered app that punts on the database (no offense, React Router; we still do love you and are making you better all the time). No more middle-stack. No more leaning on other libraries for anything interesting you want to do on the server, or punting in the browser and relying on React (which we do still love and have a ton of respect for, by the way).
Remix is a truly full-stack framework in a single dependency. That means less surface area in your package.json for the next npm supply-chain attack. It also means less churn from you or your agent cobbling together various packages just to build your blog with 4 posts (guilty). And if you don't like one of our choices, you can swap out just that piece. For example, use Zod instead of @remix-run/data-schema (it's compatible), or pull in Drizzle instead of @remix-run/data-table. Heck, you can even swap our render middleware with a React-based one if you really want to. It's composable by design so that you always stay in charge.
We built Remix to be something we want to use, and we hope to see agents continue to excel with it and other humans come to love it.
You're probably wondering at this point how long it will be before we call Remix 3 stable. We know many are waiting to give Remix a real try until we mark the software as stable, or at least until we have some better docs (that one's mostly on me, but the first few entries in our guides are pretty decent if you haven't checked them out).
At this point, we are done adding features to Remix 3 before the official release. We have plenty of ideas for continuing to expand and improve the framework (I've had multiple people ask me if we're gonna include background jobs), and we probably could spend years in pre-3 release mode if we wanted. Agents only enable our addiction to enhancing and improving and expanding Remix. But endlessly perfecting the perfect framework is no good for you, and we really want Remix to help you build your ideas. So it's time to ship and start binding ourselves to SemVer so you can trust Remix with your ideas.
The Release Candidate marks the end of feature development (if we can help it) and allows us to clean up known bugs, perform thorough security audits, gather feedback from early adopters, write more docs, and, oh yeah, prepare for Remix Jam (where we will release Remix 3).
Want to take this RC for a spin?
npx remix@next new my-remix-app
You can also learn more and stay up-to-date by checking out:
Alright, back to work for us, we've gotta ship this thing! See you in person or online at Remix Jam!
Compose Multiplatform 1.12.0 is out! This version brings new tooling for AI assistants, improvements to web resource management, and finer control over desktop window states.
Here are the highlights of this release:
For a complete overview of the changes, check out What’s new in Compose Multiplatform 1.12.0 or the release notes on GitHub.
Get Started with Compose Multiplatform
Compose Hot Reload now ships with an experimental Model Context Protocol (MCP) server that connects AI coding agents to your running application.
Using the MCP server, an agent can trigger reloads, take screenshots, inspect the semantic tree, simulate clicks and text input, and read application logs. In practice, this means the agent can verify the results of its own edits. It can confirm that the reload succeeded, inspect the rendered UI, catch a runtime exception, and iterate – all without you describing what’s on screen.
For the full list of available tools and instructions on connecting your agent, see the Compose Hot Reload documentation.
Compose Multiplatform for web now handles characters that your application’s fonts don’t cover. When it encounters an unresolved character during rendering, it downloads the matching Noto font subset on demand and recomposes the affected text. As a result, Japanese, Arabic, Devanagari, and emoji render correctly without you having to bundle fonts for them.
This release introduces an experimental v2 of the API for WindowState and DialogState in the androidx.compose.ui.window.v2 package. It gives you finer control over how windows and dialogs are positioned and sized. You can:
The enhanced API also makes the asynchronous nature of window state changes explicit: It distinguishes the state you request from the state the window currently has.
For example, to center a window and give it a fixed size, use WindowPositionProvider and WindowSizeProvider:
val windowState = rememberWindowState(
initialBoundsProvider = WindowBoundsProvider(
positionProvider = WindowPositionProvider.CenteredOnScreen,
sizeProvider = WindowSizeProvider.Fixed(DpSize(400.dp, 200.dp))
)
)
With the API v2, you can also use WindowSizeProvider.Unconstrained to size the window to its content initially, while still letting that content expand with fillMaxSize() when the user enlarges the window:
WindowBoundsProvider(
positionProvider = WindowPositionProvider.CenteredOnScreen,
sizeProvider = WindowSizeProvider.Unconstrained
)
See the Window and dialog API v2 documentation for the full details.
Update your dependencies, try out the new APIs, and let us know what you think about Compose Multiplatform 1.12.0.
For everything that didn’t make it into this post, check out the full release notes or What’s new.
Full Changelog: v0.34.3...v0.34.4-rc0
tests: add -b/--backend option to test-llama-archs for testing a specific backend (#27372)
tests: add backend option to test-llama-archs
Update tests/test-llama-archs.cpp
Co-authored-by: Johannes Gäßler johannesg@5d6.de
Co-authored-by: Johannes Gäßler johannesg@5d6.de
Website:
Attestations:
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP binding, image outputs from function calls, and several chat parser/UI fixes.
conv2d with implicit GEMM (#29135)llama_adapter_lora_init_from_file_ptr() for loading LoRA from an open FILE (#28993)llama_model_load_from_file_ptr() as reading from the current position and requiring aligned mmap (#28993)LLAMA_VOCAB_TYPE_TEST dummy tokenizer (#29084)input_image support to server function-call outputs (#22575)--host to accept comma-separated TCP addresses and UNIX sockets (#28690)--fuse-qkv support for Muse Glimmer (#29203)llama-bench --version (#29033, #28971)/tools endpoint on every message (#28646)ggml_dsv4_hc_pre_gated(), optional ggml_dsv4_hc_post() comb, and RPC protocol major v7.Nightly build: b11146
Continued at the source.
See kubernetes-announce@. Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.
See kubernetes-announce@. Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.
See kubernetes-announce@. Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.
Classic MuJoCo provides fast CPU-based robot simulation for developing, testing, and controlling robots and it can parallelize sampling across CPU cores. But as learning workloads grow, the question shifts from how quickly one world can run to how many worlds can run at once. GPU acceleration makes it possible to advance those worlds in large batches while keeping simulation and learning data close to the device.
MuJoCo Warp (MJWarp), built on NVIDIA Warp, takes compatible MuJoCo models into that GPU-scale regime. In this article, we will move an SO-101 follower arm from a familiar MuJoCo workflow to as many as 2,048 parallel MJWarp environments and examine the technology and validation steps that make the transition possible.
Figure 1. How MJWarp connects Python to GPU simulation. MuJoCo loads and compiles the MJCF model; MJWarp implements the physics in NVIDIA Warp, which compiles CUDA kernels to advance simulation states on NVIDIA GPUs.
This is the second article in our State of Simulation for Physical AI series. The first article mapped the robot-simulation landscape. Here, we prepare and scale the simulation environment; we do not train a policy. The later Newton and Isaac Lab installments cover the next integration layers.
| Layer | Role in the stack |
|---|---|
| NVIDIA Warp | Python kernel language: single instruction, multiple threads (SIMT), autodiff, PyTorch/JAX interop |
| MJWarp | MuJoCo physics on Warp: same MJCF, batched GPU throughput |
| Your scene (SO-101) | Familiar Menagerie / Robot Studio assets + task geometry |
| Next (Newton / Isaac Lab) | Multi-solver API, USD, sensors, managers, training loops |
Decision shortcut:
| If you need… | Reach for… |
|---|---|
| Single-robot MPC / teleop | MuJoCo CPU |
| Max throughput on raw MuJoCo physics | MJWarp (or mjlab) |
| JAX training recipes | MuJoCo Playground / MJX (impl='warp') |
| Multi-solver + Isaac Lab integration | Newton — next post in this series |
NVIDIA Warp is a Python framework for writing high-performance, GPU-accelerated kernels. Warp lets developers author statically typed kernels in Python and compiles them for CPU or CUDA execution. The first launch builds and caches a native module; later launches reuse it. The kernel language is a performance-oriented subset of Python, while ordinary Python remains responsible for configuration, allocation, and launch orchestration.
This small robotics-oriented kernel advances point positions under gravity. One logical thread handles one point, so the same code scales from two points to millions without introducing GPU terminology into the control flow.
The three value propositions of Warp are:
| Pillar | What you get |
|---|---|
| Performance | Native-CUDA speed via JIT compilation, kernel fusion, and CUDA Graphs |
| Ease of use | Pure Python authoring with built-in vectors, matrices, quaternions, BVHs, hash grids, sparse matrices, and tile primitives |
| Capability | Differentiable kernels and DLPack-style interop so simulation can sit inside an ML training loop |
import numpy as np
import warp as wp
@wp.kernel
def integrate(
positions: wp.array[wp.vec3],
velocities: wp.array[wp.vec3],
dt: float,
):
i = wp.tid()
velocities[i] += wp.vec3(0.0, 0.0, -9.81) * dt
positions[i] += velocities[i] * dt
wp.init()
device = "cuda:0" if wp.is_cuda_available() else "cpu"
start = np.array([[0.0, 0.0, 0.5], [0.2, 0.0, 0.5]], dtype=np.float32)
positions = wp.array(start, dtype=wp.vec3, device=device)
velocities = wp.zeros_like(positions)
wp.launch(
integrate,
dim=len(start),
inputs=[positions, velocities, 0.01],
device=device,
)
wp.synchronize_device(device)
print(positions.numpy())
Two further Warp capabilities are worth knowing, even though neither is used in the SO-101 workflow in this article. Warp kernels are differentiable: a wp.Tape records the forward kernel launches made inside its context and replays their adjoints in reverse when backward() is called, which is why teams build differentiable geometry, CFD, and custom physics in Warp, including CAE workflows for simulation and design optimization. Warp also supports deterministic execution, introduced in Warp 1.15: GPU atomics are scheduler-dependent by default, so repeated launches of the same kernel can differ slightly, and the opt-in deterministic modes trade some performance for reproducible ordering in simulation, validation, and regression tests. These are Warp capabilities, not guarantees of differentiability or determinism for an entire MJWarp rollout. See the Warp documentation on differentiability and deterministic execution for the details.
Try Warp: pip install warp-lang (≥ 1.15 for GPU determinism), then python -m warp.examples.browse, or the tutorial notebooks.
A robot simulator repeatedly computes what happens next: given the current joint positions, velocities, controls, and contacts, it advances the scene by one small timestep. In this article, a world means one independent copy of that scene and its state. One world might contain the SO-101 arm reaching for a cube; another can contain the same arm starting from a slightly different pose.
MuJoCo and MJWarp can run the same compatible robot and task, but they organize the work differently. MuJoCo naturally suits developing and inspecting one or a few CPU worlds. MJWarp is a NVIDIA Warp implementation of MuJoCo’s physics pipeline that places the model and a batch of independent states on NVIDIA GPUs; one call to mjw.step advances the entire batch.
MJWarp’s value is not necessarily a faster step for one world. It is the ability to advance hundreds or thousands together, giving the GPU enough parallel work to improve aggregate throughput, the total world-steps completed per second. That favors reinforcement learning and large-scale sampling, where collecting experience matters more than minimizing one environment’s latency.
This blog covers the following:
Solver tuning, Jacobian representation, and specialized multi-GPU or determinism topics are not required for this migration and can be covered separately.
Then, the distinction is precise:
| MuJoCo host workflow | MJWarp workflow |
|---|---|
| mujoco.MjModel | mjw.put_model(mjm) creates a device model |
| mujoco.MjData | mjw.put_data(mjm, mjd, ...) preserves and batches an existing state |
| mujoco.mj_step(mjm, mjd) | mjw.step(m, d) advances every world in d |
| Host arrays such as mjd.ctrl | Batched device arrays such as d.ctrl with shape (nworld, nu) |
Use mjw.make_data() when default/fresh state is intended. Use mjw.put_data() when the exact initialized MuJoCo state must cross the migration boundary.
Allocating batched resources requires defining the following parameters (refer to Batch sizes):
| Parameter | Meaning |
|---|---|
| nworld | Total number of parallel environments |
| nconmax | Expected contacts per individual world (overall capacity ≈ nconmax * nworld) |
| naconmax | Alternative setting: global maximum contacts across all environments combined (takes precedence if both are defined) |
| njmax | Hard upper limit on constraints per world |
1. CUDA graph capture:mjw.step is many kernel launches; capture once, replay often:
with wp.ScopedCapture() as capture:
mjw.step(m, d)
wp.capture_launch(capture.graph)
2. Size nconmax / naconmax / njmax tightly: memory and work scale with them. Tune with mjwarp-testspeed: --measure_alloc and watch overflows in mjwarp-viewer.
Additional tuning considerations. After sizing contact and constraint buffers, test solver iteration limits without changing task behavior. Meshes and CCD settings can increase memory use; nccdmax / naccdmax can reduce CCD buffer allocation when the measured contact counts allow it. MJWarp’s compact solver uses MuJoCo’s Newton constraint solver and sleeping, not the separate Newton physics-engine framework. Compact-solver and multi-GPU configuration are beyond this walkthrough; consult the MJWarp performance-tuning documentation.
To train policies on MJWarp physics:
Install / try: pip install mujoco-warp · mjwarp-viewer path/to/scene.xml · Colab tutorial
The scene. Nothing here is MJWarp-specific yet: an SO-101 arm, a table, and two cubes to stack, written as ordinary MJCF.
Figure 2. SO-101 pick-and-place scene, rendered from the MuJoCo CPU simulation. The task is to grasp the red 44 mm cube and stack it on the blue cube; the same robot and scene are used for MJWarp validation.
<mujoco model="so101_pick_place">
<include file="so101.xml"/>
<worldbody>
<light pos="0.3 0 1.5" dir="0 0 -1" directional="true"/>
<geom name="floor" type="plane" size="0 0 0.05"/>
<geom name="table" type="box" pos="0.35 -0.04 0.012"
size="0.16 0.26 0.012" rgba="0.32 0.32 0.32 1"
friction="1 0.005 0.0005" condim="3"/>
<body name="red_cube" pos="0.33 -0.13 0.046">
<freejoint name="red_cube_joint"/>
<geom type="box" size="0.022 0.022 0.022" mass="0.08"
rgba="0.85 0.05 0.04 1" friction="1.2 0.005 0.0005" condim="3"/>
</body>
<body name="blue_cube" pos="0.33 0.06 0.046">
<freejoint name="blue_cube_joint"/>
<geom type="box" size="0.022 0.022 0.022" mass="0.08"
rgba="0.05 0.20 0.90 1" friction="1.2 0.005 0.0005" condim="3"/>
</body>
</worldbody>
</mujoco>
For an MJCF box, the size values are half-extents: size=”0.022 …” defines a cube with 44 mm edges. The task uses this size for its success thresholds. The arm base is at the origin, its reach is along +X, and the cubes are arranged along Y.
In the companion repository this file is generated rather than hand-written: resolve_pick_place_scene() copies the Menagerie arm into .generated/, fills the table and cube coordinates from a robot profile, and writes scene_pick_place.xml. The walkthrough uses the SO-101 profile; the optional reBot variant is described below.
Loading it. Compilation and stepping are ordinary MuJoCo:
import mujoco
mjm = mujoco.MjModel.from_xml_path("scene_pick_place.xml")
mjd = mujoco.MjData(mjm)
fps = 50 # controller rate
sim_substeps = 10 # physics steps per control frame
frame_dt = 1.0 / fps
mjm.opt.timestep = frame_dt / sim_substeps
controller = PickPlaceController(spec=spec) # waypoints + damped-least-squares IK
for _ in range(600): # 600 control frames
ctrl = controller.step(mjm, mjd, frame_dt)
for _ in range(sim_substeps):
mjd.ctrl[: mjm.nu] = ctrl
mujoco.mj_step(mjm, mjd)
Keep that shape in mind: compute controls once per frame, step physics sim_substeps times. Gate 2 changes only the inner loop, which is what makes the migration easy to review.
Match the simulation and control rates. At 50 control frames per second and 10 physics substeps per frame, use a physics timestep of 0.002 seconds. Set it before the CPU rollout and before uploading the model with mjw.put_model so both backends advance the same simulated time:
mjm.opt.timestep = frame_dt / sim_substeps # 50 Hz × 10 substeps -> 0.002 s
Without that line, every later measurement inherits the mismatch: parity comparisons, throughput numbers quoted as “simulated seconds,” and any learned policy whose action rate no longer matches deployment.
Check whether the cubes are stacked successfully. With 44 mm cubes, success becomes two measurable conditions: a horizontal center error of xy_err ≤ 0.015 m (measured between the cube centers) and a vertical separation of 0.035 m ≤ dz ≤ 0.055 m between cube centers (one cube edge, with slack for settling). Evaluate both conditions after the cubes have settled; a successful process exit alone does not establish task success.
Run the CPU task from the companion checkout. Publication blocker: confirm the accessible repository URL and pinned dependency and asset versions before publishing these instructions; the repository placeholder below is not an executable URL.
git clone https://github.com/NVIDIA/accelerated-computing-hub.git blogs
cd blogs/tutorials/sim2real-blogs/notebooks/mujoco
uv venv --python 3.12 && source .venv/bin/activate
uv pip install -r requirements.txt
cd /tutorials/sim2real-blogs/notebooks/mujoco
python solutions/so101_pick_place_solution.py --headless-steps 600 --debug
The run ends by printing the two numbers above (stack check: xy_err=… dz=…), which is the assertion the rest of the article compares against. so101_pick_place.py next to it is the same program with the physics steps left as exercises.
The arm comes straight from MuJoCo Menagerie pinned to a known-good commit, since Menagerie assets change, so treat the scene as a template. Optional reBot variant. The companion code also exposes --robot rebot with a separate profile for the scene layout, gripper, and capacity limits (nconmax=256, njmax=500). This walkthrough uses SO-101. Validate the reBot asset and task separately before reporting its results.
Run one world on the GPU first, with the host still in the loop, so you can watch the same task in the same viewer and compare the same two numbers. Upload the model, allocate batched state, seed it from the initialized host state, and run one forward pass before stepping:
wp.init()
import mujoco_warp as mjw
device = wp.get_device()
m = mjw.put_model(mjm)
d = mjw.make_data(mjm, nworld=1, nconmax=spec.nconmax, njmax=spec.njmax)
wp.copy(d.qpos, wp.array(mjd.qpos[None, :], dtype=wp.float32, device=device))
wp.copy(d.qvel, wp.array(mjd.qvel[None, :], dtype=wp.float32, device=device))
wp.copy(d.ctrl, wp.array(mjd.ctrl[None, :], dtype=wp.float32, device=device))
mjw.forward(m, d)
Every device array carries a leading world dimension, which is why the host state is indexed as mjd.qpos[None, :], shape (1, nq) instead of (nq,). Scaling to thousands of worlds later changes only that leading dimension, not the calls. mjw.put_model() also doubles as a compatibility check: it raises if the model uses unsupported features rather than silently dropping them.
Seeding the three fields explicitly is the transparent option, and it makes clear exactly what crosses to the device; mjw.put_data(mjm, mjd, nworld=…) carries the whole initialized struct over in one call instead.
The frame loop is then the Gate 1 loop with its inner step redirected to the GPU and mirrored back:
def simulate_frame() -> None:
ctrl = controller.step(mjm, mjd, frame_dt)
for _ in range(sim_substeps):
mjd.ctrl[: mjm.nu] = ctrl
wp.copy(d.ctrl, wp.array(mjd.ctrl[None, :], dtype=wp.float32, device=device))
mjw.step(m, d)
mjd.qpos[:] = d.qpos.numpy()[0]
mjd.qvel[:] = d.qvel.numpy()[0]
mujoco.mj_forward(mjm, mjd)
The .numpy() reads synchronize and copy data to the host on every substep, so this is a task-validation path, not a throughput benchmark. It keeps inverse kinematics, viewing, and task checks on the host. After copying qpos and qvel, call mujoco.mj_forward(mjm, mjd) to refresh derived host quantities such as mjd.xpos before using them for control, viewing, or the stack check. Reading those fields after the loop does not refresh them automatically. Gate 4 removes these per-step host copies from the throughput path.
MJWarp allocates contact and constraint buffers before stepping. Exceeding those capacities invalidates the affected rollout for verification or benchmarking, even when execution continues with an overflow warning rather than an exception. Increase the relevant limit and rerun the task. Larger buffers use more GPU memory, so verify capacity over the full task before tightening the allocation.
Set contact and constraint limits for the robot and task being simulated. The SO-101 profile uses nconmax=128 and njmax=300 as starting capacities. Check that these limits are sufficient during the most contact-heavy part of the task:
d = mjw.make_data(mjm, nworld=nworld, nconmax=spec.nconmax, njmax=spec.njmax)
Size them against the most contact-heavy moment of the task, for pick-and-place, the instant both jaws and the table touch a cube, not the arm hovering in free space. An overflow is reported rather than raised: with Option.warn_overflow at its default, MJWarp prints the budget to increase (“narrowphase overflow - please increase nconmax to …”) to the terminal running your script or the viewer, and flags the affected worlds in Data.overflow for you to read back after a step. Only mjw.put_data raises an error outright, because it can compare the budgets against a MuJoCo state it already holds. mjwarp-testspeed --measure_alloc reports the contacts and constraints a scene actually consumed, and it aborts the rollout with the offending world IDs as soon as any world overflows. Treat those reports as failures: raise the limit and re-run before trusting either the trajectory or the benchmark, then tighten again whenever the model, collision geometry, or task changes.
Once one-world parity passes, reallocate at the target size and replicate the initialized state across the batch. Two things change relative to Gate 2: nworld, and the fact that nothing crosses the PCIe bus per step.
nworld = 2_048
d = mjw.make_data(mjm, nworld=nworld, nconmax=spec.nconmax, njmax=spec.njmax)
wp.copy(d.qpos, wp.array(np.tile(mjd.qpos, (nworld, 1)), dtype=wp.float32, device=device))
wp.copy(d.qvel, wp.array(np.tile(mjd.qvel, (nworld, 1)), dtype=wp.float32, device=device))
wp.copy(d.ctrl, wp.array(np.tile(mjd.ctrl, (nworld, 1)), dtype=wp.float32, device=device))
mjw.forward(m, d)
with wp.ScopedCapture() as capture:
mjw.step(m, d)
step_graph = capture.graph
np.tile gives every world the same starting state, which is the right baseline for a throughput measurement; per-world randomization would instead write different rows of d.qpos on the device.
CUDA Graphs reuse the model and data buffers captured here. Update d.ctrl in place between replays, and capture a new graph after replacing buffers, changing nworld, or rebuilding the model. Graph capture requires CUDA.
Figure 3. Scaling the SO-101 task from one CPU world to 2,048 independent GPU states using the same compatible model. A single MJWarp step advances the full batch. This conceptual illustration highlights aggregate throughput, measured as world-steps per wall-clock second.
GPU launches are asynchronous, so a naive timer measures how fast Python queued work, not how fast the GPU finished it. Warm up first — the first launches pay kernel compilation and allocation — then synchronize immediately before and after the timed region:
import time
for _ in range(10): # warm-up: compilation, allocation, caches
wp.capture_launch(step_graph)
wp.synchronize()
t0 = time.perf_counter()
for _ in range(200):
wp.capture_launch(step_graph)
wp.synchronize() # without this you time the queue, not the work
elapsed = time.perf_counter() - t0
total = 200 * nworld
print(f"{total / elapsed:,.0f} world-steps/second")
Report both aggregate world-steps per second and milliseconds per batched step, together with the batch size. Use the measured curve to identify where additional worlds improve throughput and where memory or compute limits reduce the benefit. Results depend on the scene, simulation settings, and hardware; a one-world latency comparison does not establish batched throughput.
To see that curve on your own hardware, scaling_study.py sweeps the batch size and prints ms/step alongside throughput and speedup:
cd /tutorials/sim2real-blogs/notebooks/mujoco/part2
python solutions/so101_mjwarp_solution.py --headless-steps 600 # parity, needs CUDA
python scaling_study.py --worlds 1 64 1024 2048 8192 --steps 100
Warp (kernel layer)
pip install warp-lang → python -m warp.examples.browse → docs · GitHub
MJWarp (GPU MuJoCo)
pip install mujoco-warp → mjwarp-viewer benchmarks/humanoid/humanoid.xml → docs · GitHub · Colab tutorial
SO-101 context
SO-101 sim-to-real course · Physical AI learning paths
Train on top of MJWarp
mjlab · MuJoCo Playground · Isaac Lab + Newton (upcoming posts)
This post covered raw Warp → MJWarp: GPU kernels, batched stepping, and an SO-101 scene using mjw.step.
Next, we will port the same MJCF environment into Newton, using MuJoCo Warp as its rigid-body solver (newton.solvers.SolverMuJoCo). Newton will manage the model, state, controls, and contacts, while MJWarp runs underneath.
You will also see what Newton adds: multi-format assets, swappable solvers, sensors/IK helpers, and an Isaac Lab path.
The migration guide continues with the same SO-101 task and its optional reBot profile, explaining the changes required by Newton and the separate Isaac Lab integration.
If you build something with Warp or MJWarp, open an issue on the linked repositories or find us on Discord NVIDIA Omniverse.
This post is co-written with Mauro Rallo and Patrick van der Plas from HEMA.
When engineers at HEMA needed an answer, they went portal-hopping, navigating disconnected wikis, service catalogs, and IT portals to find it. To turn that friction into instant answers, the 100-year-old Dutch retailer built a knowledge layer on Amazon Bedrock AgentCore. HEMA has over 750 stores across multiple countries, served by a technology organization of engineers, product owners, and business analysts driving digital transformation. It needed a solution that worked across roles and tools.
Over the years, HEMA had quietly built something valuable: a large, structured picture of its own technology landscape. A service catalog mapped people to teams, teams to services, and services to the APIs we expose, and the business capabilities we support. The problem was never that the knowledge didn’t exist. It was that the knowledge was hard to reach. As the engineering organization grew, the informal “just ask the person next to you” model broke down, and teams ended up scattering answers across portals, wikis, and documentation that few people knew how to navigate.
In this post, we describe the challenge HEMA faced with fragmented internal knowledge, why we chose to build HAL, HEMA’s internal AI assistant, using Model Context Protocol (MCP) and Amazon Bedrock AgentCore, and how it changed the way our teams work.
The idea rests on two complementary goals. HAL puts knowledge in one place, and MCP delivers that knowledge inside the tools people already use (the HAL chat, Kiro, Claude, and other agents). Security is anchored in Microsoft Entra ID, with no AWS credentials on the client. What began as a developer tool is already a cross-role assistant. The same architecture will be the foundation for a next step: turning HAL from a read-only knowledge layer into an action layer.
HEMA’s knowledge problem had two distinct layers.
The first layer, structured infrastructure knowledge, was actually in good shape. For years, HEMA has maintained a service catalog that captured how the technology estate fits together: which teams own which services, what APIs those services expose, and how they map to business capabilities. Structured data from systems such as the product information management (PIM) engine and the data-mesh tables had been imported and organized. For anything about what exists and who owns it, the answer was usually available, if you knew where to look.
The second layer was the gap. Knowing what exists is not the same as knowing how to do something. “How do I request access to an API? How do I get a new group provisioned? What’s our rule for X?”. These procedural questions had no single home. When teams were small and everyone knew each other, that was fine. People asked directly. As HEMA grew and onboarded new engineers, that model stopped scaling, and there was little written documentation to fall back on.
That translated into slow onboarding for new joiners, inconsistent answers depending on where someone looked, constant context-switching, and friction that pulled people out of their actual work. Finding an answer that once meant navigating three or four portals, sometimes across an entire afternoon, now happens in seconds, from inside the Integrated Development Environment (IDE) or chat window.
Two goals shaped the solution, and they map cleanly onto the two technologies we chose.
The first goal belongs to HAL: consolidate HEMA’s fragmented knowledge into one governed source of truth. The second goal belongs to MCP: deliver that knowledge to people where they already work, rather than forcing them to visit yet another portal.
Why MCP: Model Context Protocol gives us a standardized interface between AI clients and backend capabilities. Instead of building a bespoke integration for every knowledge source and re-building it for every client application, we expose each source once as an MCP tool.
MCP-compatible clients such as the HAL web chat, Kiro, Claude, and other agents can then consume the same tools without custom work. This is what makes “access from your daily tool” practical rather than a per-tool engineering project.
Why Amazon Bedrock AgentCore: Amazon Bedrock AgentCore is a platform to build, connect, and optimize agents at scale, with any framework or model. For HEMA, it meant building HAL without standing up and operating custom MCP server infrastructure. The capabilities that mattered most:
Together, these gave us enterprise-appropriate footing: Entra ID OAuth, read-only access today, and access control driven by existing Active Directory groups, safe enough to expose real internal knowledge.
HAL didn’t arrive fully formed. It grew in two deliberate steps. First, the team built a standalone assistant with its own chat UI. Then, once that foundation proved itself, we opened it up to the tools people already work in through MCP.
The first version of HAL was a self-contained assistant: a web chat UI (built with Next.js) backed by an agent that could answer questions from HEMA’s knowledge. There were no MCP and no external clients yet, only the HAL UI talking to the HAL agent.
The HAL agent is a Strands agent packaged as a Linux/ARM64 container and hosted on AgentCore runtime, together with AgentCore memory (short-term conversation context) and Amazon Bedrock Guardrails (Standard tier, EU Cross-Region inference for Dutch-language support). AgentCore runtime and AgentCore memory are capabilities of Amazon Bedrock AgentCore.
The agent reaches knowledge along two distinct paths:
We started from the structured data we already had, the service catalog, and added the highest-value documentation, prioritizing by pain and by how often something was asked. Behind HAL sit several knowledge bases built on Amazon Bedrock Knowledge Bases, the fully managed Retrieval Augmented Generation (RAG) capability: IT and how-to documentation, API/OpenAPI specifications, Kafka event-streaming topics and their Avro schemas, Data Consolidation Layer (DCL) data-exchange channels, and the service catalog (people, teams, services, and APIs).
There’s no custom MCP server code. AgentCore Gateway generates the MCP tools directly from OpenAPI specifications for the API passthrough targets, and from a Lambda function for semantic search over the Knowledge Bases. Pointing the Gateway straight at our existing API specifications isn’t the ideal end state. An API designed for system-to-system use does not always map cleanly onto a tool an agent can reason about, so we plan to refactor those definitions into more agent-friendly tools.
For now, though, exposing the APIs as-is delivered high value for little effort. The kb-search Lambda wraps the Amazon Bedrock Retrieve API over the Knowledge Bases. It’s scoped by AWS Identity and Access Management (IAM) to the specific Knowledge Base Amazon Resource Names (ARNs), plus read access to the source-document Amazon Simple Storage Service (Amazon S3) bucket.
Retrieval follows a two-step pattern: an initial Knowledge Base search answers most questions, and fetch_full_document pulls the complete document when a single chunk isn’t enough. Retrieval quality is improved with Amazon Bedrock reranking on semantic queries and team_id metadata filtering for team-scoped lookups.
HAL worked well in its own chat UI, but people live in other tools: their IDE, their AI assistant. The second step was to let external MCP clients such as Kiro and Claude reach the same knowledge and tools, without handing out AWS credentials. That meant adding a second AgentCore Gateway, authenticated with Microsoft Entra ID instead of IAM.
Because an AgentCore Gateway supports only a single inbound authentication type, we could not reuse the agent’s IAM-authenticated Gateway from Step 1 for these external clients. So, we added a second Gateway, an Entra MCP Gateway authenticated with a custom JWT through Microsoft Entra ID, dedicated to external MCP clients such as Kiro and Claude. It shares only the read-only Knowledge Bases with the agent Gateway. There is no shared code, so the external-facing surface can evolve, or fail, without impact on the internal agent.
AgentCore Gateway exposes tools from OpenAPI specifications and Lambda functions. To surface the knowledge bases as a Gateway target, we built a small intermediate Lambda function that the Gateway calls as a tool, and which performs the semantic search over the knowledge bases on behalf of the Gateway. The live internal APIs, by contrast, are exposed directly as OpenAPI targets.
One interesting piece is how external clients authenticate without AWS credentials. In front of the Entra Gateway sits an MCP auth proxy, an Amazon API Gateway v2 HTTP API backed by a single Lambda, that reconciles the MCP OAuth specification with the specifics of Entra ID. It serves the OAuth discovery documents and rewrites the requested scope to the resource app’s invoke scope. It also strips the legacy resource parameter that Entra v2.0 rejects, adds response_mode=query so desktop clients can capture the authorization code, and proxies /mcp with the bearer token.
One detail is worth calling out because it is the only place DCR appears in the whole system. MCP clients expect DCR, a POST /register call that hands back a client ID. Rather than implementing true dynamic registration, the proxy uses a stubbed /register that returns a fixed, pre-provisioned client ID. DCR is emulated, not real.
The full handshake looks like this:
For the end user, the payoff is that configuration is only the proxy URL and an empty oauthScopes list, no AWS credentials, a browser login on first connect, and automatic token refresh thereafter.
The infrastructure is defined in AWS Cloud Development Kit (AWS CDK), a TypeScript monorepo using npm workspaces. The internal agent runs as a Docker container on AgentCore runtime. Environment-specific configuration, such as tenant, client, and resource identifiers, is supplied through AWS Systems Manager (SSM) parameters.
Before going live, HEMA deployed HAL to a staging environment and opened it to both engineers and business users for hands-on testing over a one-month period. This validated answer quality, coverage gaps, and day-to-day usability before the solution was promoted to production for wider adoption across the organization.
HAL began as a developer tool, but it is already a cross-role assistant, and that breadth is the point.
The pull from non-developer roles is real and growing: HEMA’s end-to-end team, mapping and optimizing product-manager processes, is already engaging with HAL as part of that work. Internally, HAL is distributed through an “Everyone Skill” and accompanying steering files, shared and maintained through the monthly HEMA AI Development Forum.
Today, HAL is read-only and delivers instant answers. The next step is instant action, performed by the same assistant.
This is feasible now precisely because the underlying portals are already API-enabled and already integrated with existing Microsoft Entra ID single sign-on. That means HAL can expose those operations as MCP action tools using the very same Entra ID authentication and Active Directory group authorization model that already secures the read tools. No new security model is required, only new, carefully scoped tools.
The flagship example is provisioning a new AWS account. Today a developer goes to a dedicated portal to request one. Next, they will make the same request directly from chat, Kiro, Claude, or other agents without visiting the portal at all. And because HAL already serves product owners and business analysts, the same action pattern extends naturally beyond developer operations to the wider set of operational requests those roles make every day.
The lesson is that the read architecture earns the write step: by getting identity, multi-client access, and governance right for answers, we have laid the groundwork for actions.
HAL puts HEMA’s organizational knowledge in one governed place. MCP and Amazon Bedrock AgentCore make that knowledge reachable, multi-client, and secure without forcing every consuming application to rebuild authentication, authorization, or routing from scratch. The outcome isn’t a developer chatbot, but a cross-role assistant for developers, product owners, and business analysts alike. The knowledge layer is live. The logical next step is extending it into an action layer. There, the same governed, authenticated infrastructure that today answers questions could tomorrow execute requests: provisioning access, triggering workflows, and acting on behalf of users directly from chat, Kiro, Claude, or other MCP-compatible agents.
If you want to explore the building blocks used in this post, the following resources are a good starting point. To learn about MCP server hosting, authentication, and gateway routing, see the Amazon Bedrock AgentCore documentation. For the Model Context Protocol specification and client compatibility guidance, visit the MCP specification site. To get hands-on with Strands Agents, the open source agent framework used to build HAL’s internal agent, see the Strands Agents GitHub repository. If you’re building a similar knowledge layer for your organization, the Amazon Bedrock workshop walks through RAG patterns, Knowledge Bases, and guardrails in a guided environment.
For related reading, see Building and connecting a production-ready ecommerce MCP server using Amazon Bedrock AgentCore and Mistral AI Studio and Introducing Amazon Bedrock AgentCore Identity: Securing agentic AI at scale.
Mauro is an Enterprise Architect and DevOps Product Owner for HEMA’s central platform (CIP), where he focuses on architecture, developer experience, and making engineering knowledge easy to reach across the organization.
Patrick is a Software Engineer on HEMA’s AI team, where he built the MCP gateway and tooling behind HAL, HEMA’s internal AI assistant. He also builds the wider platform for HEMA’s AI agents to run on and drives adoption of AI-assisted development across HEMA’s engineering teams. Outside the office, Patrick spends his time in the gym, running, competitive gaming, and enjoying life with his fiancée.
Amit is a Senior Solutions Architect at AWS, working with enterprise retail customers in the Benelux region. He helps customers design cloud-native architectures, navigate complex modernization journeys, and adopt AI/ML capabilities at scale. Outside of work, he enjoys exploring new places and chasing the perfect shot, whether through a camera lens or on a running trail.
How we rebuilt the diff surface in the GitHub Copilot app to open a million-line pull request with hundreds of inline review comments.
Broad refactors and migrations often have to land as one change.
Stacked pull requests are a great way to split work into smaller changes, which makes reviews easier and helps teams ship with less risk. But some changes, like this one, can’t be split cleanly. That leaves you with a single pull request that can get very large, and the review conversation causes it to grow.
The review experience needs to remain fast and smooth even when the diff and its conversation are enormous. In the GitHub Copilot app, we rebuilt the pull request view with that requirement in mind.
To see how far that goes, we opened the biggest pull request we could find: an open source one with 2,200 files, over a million changed lines, and more than 400 inline review comments. Here’s how we made even this extreme pull request performant.
Rendering a large diff at speed is well-understood: virtualize the rows, keep the mounted DOM small, and lean on the fact that every row is a line of code at a known height.
Comments are the hard part. A comment’s height depends on how its markdown wraps, the expandable sections, whether there’s a reply box in it, and whether its images have loaded yet. You find all of that out at render time. This forces a different architecture.
Three problems:
The first step is to understand the geometry that makes a code-only diff fast. Once comments enter the picture, that geometry is no longer enough.
You cannot put a million DOM nodes on a page. The standard answer is virtualization: mount only the rows that are on screen, plus a small margin, and recycle those same DOM elements as the user scrolls. The list behaves as if all million rows exist. The scrollbar is the right size, scroll-to-row works. But only about 100 rows are ever real at once.
For this illusion to hold, something has to supply the geometry. The scrollbar height is the sum of all row heights. The position of row N is the sum of the heights of the rows above it. Jumping to a row, drawing the scrollbar, deciding what’s on screen, it’s all arithmetic over a table of heights. You can build that table from estimates and correct it as rows get measured, and general-purpose variable-height virtualizers do exactly that.
But if every row is a line of code at a known font size, you don’t have to. You can compute the whole table up front and it never changes, so there’s nothing to correct later.
Call this the “all heights known before paint” contract. Our diff surface is built around it:
None of it scales badly, because no per-frame work grows with the total row count. On pure code this design is the right one, and we kept all of it.
Now put a review thread in the middle of the diff. How tall is it?
You don’t know, and you can’t know without rendering it. Its height depends on things that only exist at render time, and they can keep changing after first paint:
<details> blocks the user can expand or collapse in placeThe obvious answer is to reserve a fixed-height slot for each comment, sized by an estimator. It falls apart on a big pull request. An estimator that’s right on average is still wrong at the extremes. It over-reserves most comments, leaving gaps of whitespace, and under-reserves the expensive ones, which clip or sprout a nested scrollbar. If you measure the real height after paint and write it back into the shared offset table, everything below moves, while the user is already scrolling. That’s a scroll jump, and on a big pull request it’s a large one.
So comments need a different contract. “All heights known before paint” is unachievable for this content. What we could promise instead: heights are bounded, measured lazily, and corrections are small and anchored to whatever the user is looking at.
The idea that made this tractable was to stop forcing one geometry to serve both kinds of content. We split the document’s height into two independent domains:
total height = deterministic code height (exact, known up front)
+ Σ dynamic block effective heights (estimated, then measured)
+ scroll paddingCode geometry keeps the original world. It’s deterministic, prefix-summed, exact, never rebuilt when a comment resizes.
Dynamic block geometry covers everything whose height we can’t predict, such as review threads, drafts, and reply composers. Each one is a block identified by what it is rather than where it currently sits. It has a stable key that survives its content loading, and it’s anchored to a file, line and side rather than to a pixel coordinate, so a reflow can’t lose track of it. We also keep a fingerprint of everything that could change the block’s height: its content, whether a <details> is open, whether a composer is active. And we record the width it was last measured at, rounded into buckets, so an ordinary window resize doesn’t invalidate every measurement in the document.
A block’s effective height is then simple: the measured height if we have a valid one, a cached height if the fingerprint and width still match, and the estimate otherwise. Those heights live in their own index, separate from the code rows, so a resizing comment never forces the code geometry to be rebuilt. And the number of blocks is bounded by comments, not by rows. A few thousand blocks is fine, as long as first paint never mounts or measures all of them at once.
This part took the longest to get right, because our first design was wrong in an instructive way.
The obvious way to measure dynamic content is one ResizeObserver per block, which watches the element and writes its measured height back into the layout whenever it changes. This is what we designed and then rejected during performance hardening. It is the feedback loop that big virtualized surfaces have to avoid. An observer that writes a height back into the layout of the element it’s watching can retrigger itself, and the cost grows with every mounted block.
What shipped instead is a single idle- and scroll-gated measurement pass, held to the same discipline as the deterministic side:
<details>. Each mounted block keeps a ResizeObserver, but by default all it does is flag the block so the idle pass re-reads it. It never writes a height itself, which is what would close the feedback loop we rejected. It disconnects on unmount, and an inactive pull request tab observes nothing.<details>, opening a reply composer, an image landing. The block grew immediately, but the code below it only moved on the next idle pass. For one frame the comment was taller while everything under it sat at its old position, and you could see the two steps. So when a block is mounted and on screen, the observer now measures it and applies the correction in the same frame, before paint. The block grows, the code repositions, and everything below shifts together. Two safeguards keep this from becoming the loop we were avoiding: at most one synchronous commit per frame, so a burst of resizes collapses into one, and never during an active scroll, where it falls back to the batched pass.When a measured height differs from its estimate, the scrollbar arithmetic changes, and the naive result is that the viewport jumps. The fix is to correct by identity rather than by pixel:
Plus a few rules that keep it from feeling wrong:
<details> or opened a reply in a visible block → suppress above-block correction for that block, so the interaction feels direct, and let the content below flow down naturally.That last rule has a sharp edge, and it bit us. “Don’t correct while the user is scrolling” was implemented as a guard on the last observed scroll, and programmatic scrolls refreshed that timestamp too. Toggling the file-tree sidebar changes the width of the diff pane. With line wrapping on, every wrapped line above you reflows to a different number of visual lines, the whole coordinate space shifts, and the surface emits a small scroll of its own as it settles. The guard read that as “the user just scrolled” and skipped the very correction that was supposed to keep your place, so the file you were reading drifted off screen. The fix was to tell user scrolls apart from ones the surface caused itself. Any “is the user interacting?” check has to be one your own side effects can’t satisfy.
So corrections stay small, they reuse measurements we already have, and they follow whatever you’re looking at.
A diff surface can only be as fast as the data feeding it, and three habits from that side of the work shaped what the UI could do. The first is stream structure before content. The diff is requested incrementally, so the file tree and metadata paint while the document is still loading, and the full set of review threads is resolved up front rather than trickling in. The second is defer per-item work until something needs it. Syntax highlighting runs off the main thread, so rows appear as plain text immediately and get colored when the results arrive. Highlighting improves the surface instead of blocking the scroll. Large markdown bodies and suggested-change context work the same way: nothing is built until it approaches the viewport.
The third habit is about which costs are worth keeping. Releasing a diff document when you navigate away is the right default. These documents are large, and holding on to every one you’ve visited is how a long session ends up eating memory. But pull request metadata persists, so the shell around the diff, the header and the file tree, repaints instantly when you go back, and then sits there for several seconds waiting for a diff it had complete moments ago. An instantly-drawn shell around an empty diff looks broken, even though you’re waiting less time overall. So the policy stayed and we added a cache: keep the last few diffs resident, evict anything beyond that, and let the background refresh notice when one has gone stale.
Almost every bug in this project was invisible until it wasn’t, and reproducing one by hand is miserable. A typical report reads: “a strip of whitespace appears below some comments, but only sometimes, only on big pull requests, and it heals if you scroll past and back.” You can’t debug that by staring at the screen, so we built tooling to debug it mechanically.
The naive workflow is to sprinkle console.log calls, exercise the flow by hand, copy the output, paste it to someone (or something) that can analyze it, delete the logs, and repeat. It’s slow, it needs a human in the loop, and worst of all you end up measuring your own hand-rolled instrumentation rather than the app’s real behavior.
So the surface carries permanent, structured probes for its own invariants. They’re plain questions it answers about itself on every render:
These are the objective pass/fail signals, and they’re asserted as budgets in an end-to-end test against a synthetic many-comment huge-pull-request fixture. CI can now tell us whether the surface is healthy.
The centerpiece was an autonomous change → measure → improve loop. Two lanes:
A headless probe lane ran a declarative flow (open a pull request, scroll to a fraction, toggle a details block, resize the window) against a mock server, reading the app’s own production instrumentation: React render counts, the performance timeline, and a requestAnimationFrame sampler for jank. It did the whole instrument, drive, collect, analyze, rank cycle by itself and printed the bottlenecks in order. Because the flow is just JSON handed to the probe at runtime, an agent could profile any flow by describing it in plain English, without editing a line of source.
An autopilot drove the actual desktop app through the huge-pull-request flow, unattended, on a loop: first cold, with comments still skeletons, then warm, with comments loaded, toggling <details> blocks, opening and cancelling reply composers, collapsing and expanding files, toggling the sidebar tree, sweeping deep into the file list, resizing the window. Every measurement was mirrored to the app’s on-disk log, so an agent could read runtime behavior with nobody at the keyboard. Each sample carried a health signal, and that was the objective check. A warm sample counted as healthy only if there were no unfilled gaps between comments, no comment blocks left blank, and real thread content actually mounted, across the entire scroll range, deep-file sweep included.
The loop we ran was:
Reviewing a pull request this large used to mean one of two things: waiting, or giving up and reading it somewhere else. A review isn’t a document with known dimensions. It’s a conversation that changes shape while you’re reading it, and the surface underneath has to be built for that from the start rather than patched into it afterwards.
The result is a pull request view where a million-line diff with hundreds of threaded comments opens, scrolls, and behaves like a normally sized pull request. Comments render in full instead of clipping into a scrollable box. Expanding a collapsed section moves the code below it and nothing else. Coming back to a pull request you just left puts you where you were.
If you review code for a living, it’s worth feeling the difference on a pull request you already know is painful. Open the worst one you’ve got.
Principal Design Engineer
Discover tips, technical guides, and best practices in our biweekly newsletter just for devs.
Your email address
With video intelligence powered by agentic AI, you can ask natural language questions about uploaded videos and get answers within seconds. Organizations across media, security, insurance, and professional services are generating more video than their teams can review. Meeting recordings accumulate in shared drives, and security cameras capture weeks of unreviewed footage. Field inspection videos sit in object storage long after the initial review. The information inside these videos is often valuable: a design decision discussed three weeks ago, the exact moment a person arrived at a door, or the sequence of events leading to a vehicle collision. But accessing it has traditionally required watching hours of content manually. The alternative, building custom machine learning (ML) pipelines for each specific question type, demands significant development effort. Each new use case meant new development work:
In this post, we walk through the architecture and key patterns for building a video intelligence solution that accepts natural language questions and returns answers from video content. The solution uses an agentic architecture that decides at runtime which AWS services to invoke. For previously analyzed content, responses return in under a second. Initial analysis of new videos takes 5–10 minutes depending on length and services required. The complete implementation is available in the companion GitHub repository.
Rather than pre-building a fixed pipeline for each question type, we use the Strands Agents SDK to create a single AI agent that orchestrates Amazon Bedrock, Amazon Rekognition, and Amazon Transcribe based on what the user asks. A major media and entertainment company adopted this approach during an AWS Professional Services engagement. With this solution, their consultants can query recorded discovery session content, extracting design decisions, action items, and stakeholder positions. The result: a reduction in manual review time of approximately 80 percent across a backlog of more than 200 multi-hour recordings, based on the customer’s internal before-and-after comparison of analyst hours per recording (not independently verified).
The solution is an AI agent that accepts video files and makes their content instantly queryable through natural conversation. A user can upload a 90-minute meeting recording and ask “What decisions were made in this meeting?” or “Did anyone mention the budget timeline?” The agent determines whether to invoke transcription, visual analysis, or both, then synthesizes the results into a coherent answer. The same system handles security footage queries (“Did this person appear?”), content analysis (“Summarize the first 30 minutes”), and investigative questions (“Which vehicle changed lanes before the collision?”). No separate processing pipelines are required for each use case.
The following screenshot shows the interface that provides a chat panel for natural language queries and a sidebar for file uploads and analysis mode selection.
The key insight is that the pipeline is determined at runtime. The agent calls Amazon Transcribe for spoken-content questions, turns to Amazon Rekognition for face matching, and reuses cached results for follow-up questions about previously processed content. The model handles the routing, not application code.
To follow along with the implementation in this post, you need:
pip install strands-agents strands-agents-tools).The system consists of an agent orchestrator connected to multiple AWS AI services, with Amazon S3 providing storage for uploaded videos and cached analysis outputs. The agent orchestrator is the reasoning engine. It’s built with the Strands Agents SDK and powered by Amazon Bedrock, using Claude Sonnet or another large language model (LLM) that supports tool use. It receives natural language queries from users and determines which tools to invoke based on the question, sequences multiple service calls when needed, and synthesizes the results into conversational responses. The agent maintains conversation history, so follow-up questions build on prior analysis without reprocessing.
Amazon Rekognition provides visual analysis, including detecting objects, scenes, activities, and faces in video frames. The agent invokes Amazon Rekognition when the user’s question concerns something visible in the video. Amazon Transcribe converts spoken audio to text with automatic language detection across more than 100 languages (see Amazon Transcribe supported languages) and speaker diarization. The agent uses Transcribe when the question relates to spoken content. Amazon Bedrock Data Automation (BDA) offers an alternative analysis path that combines video summary, chapter detection, and full transcription in a single API call. This is useful when the user wants comprehensive analysis in one step, or when Amazon Rekognition or Transcribe aren’t available. All uploaded videos and analysis outputs are stored in Amazon S3 with per-user prefixes for multi-tenant isolation.
These three services are the starting set, not a fixed one. Because the agent selects tools from their descriptions rather than from hard-coded workflow logic, the same architecture accepts additional services as tools. We return to this point in Extending beyond video. For production deployments, we recommend adding Amazon Bedrock Guardrails to enforce content filtering and grounding checks on agent responses, particularly for face-matching and surveillance use cases where responsible-AI controls are essential.
In a conventional video analysis application, the developer defines a fixed processing pipeline: upload the video, run transcription, perform visual analysis, present results. This approach processes every video through the same steps regardless of the specific query, and users wait for the full pipeline to complete before asking questions. The agentic approach inverts this model. With minimal pre-processing limited to uploading video files to an S3 bucket, the agent reasons about each question independently and calls only the services needed to answer it.
When a user submits a query, the agent first parses the intent: the user wants a transcript summary, a visual search, or a face match? Then it checks whether relevant analysis has already been performed and cached. If not, it selects the appropriate tools, executes them (potentially in sequence when one tool’s output feeds another), and combines the results into a natural language answer. In our testing with 60-minute videos, the first question about a video typically takes 5–10 minutes (while transcription or visual analysis runs). Subsequent questions about the same content return in under a second because the agent reuses cached results. Actual times vary based on video length, resolution, and the AWS services invoked.
The following code shows the complete agent setup. We define the model provider, a system prompt that guides the agent’s reasoning behavior, and the set of available tools. With Strands, the entire orchestration logic (deciding which tools to call, in what order, and how to combine their outputs) is handled by the LLM rather than application code. We show two representative tool implementations (search_faces_in_video and analyze_with_bda). The remaining tools, including transcribe_video and analyze_video_visuals, follow the same pattern and are available in the GitHub repository.
from strands import Agent
from strands.models.bedrock import BedrockModel
from tools import (
transcribe_video, analyze_video_visuals,
search_faces_in_video, analyze_reference_image,
analyze_with_bda, upload_video
)
model = BedrockModel(
model_id="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
max_tokens=4096
)
SYSTEM_PROMPT = """
You are a video intelligence assistant. For each user query:
1. Determine whether it requires spoken content analysis,
visual content analysis, or both
2. Check if prior analysis results are already cached
3. Invoke the appropriate tools
4. Synthesize results into a clear answer with timestamps
"""The production system prompt spans approximately 250 source lines. The following abbreviated example illustrates three representative policies (cache reuse, service fallback, and multi-modal orchestration) rather than reproducing the prompt verbatim:
# --- Cache management (excerpt) ---
CACHE_GUIDANCE = """
Before invoking any analysis tool, check the cache:
- Call get_cached_result(video_id, analysis_type) first
- If cached results exist and are < 24 hours old, use them
- If the user says "re-analyze" or "fresh analysis", bypass cache
- After any new analysis, store results with cache_result()
# --- Tool fallback behavior ---
If a tool call fails or returns low-confidence results:
- Transcribe failure: suggest BDA as fallback
- Rekognition low confidence (<60%): report uncertainty to user
- BDA timeout: fall back to individual Transcribe + Rekognition calls
# --- Multi-modal orchestration ---
When the query requires both audio and visual understanding:
1. Run Transcribe and Rekognition in parallel when possible
2. Correlate timestamps across modalities
3. Synthesize a unified answer referencing both sources
4. Cite specific timestamps for each claim
"""
agent = Agent(
model=model,
system_prompt=SYSTEM_PROMPT,
tools=[transcribe_video, analyze_video_visuals,
search_faces_in_video, analyze_reference_image,
analyze_with_bda, upload_video]
)The rest of the production prompt inventories the available tools and defines workflows for file selection, cache reuse and explicit re-analysis, BDA setup and access-denied fallback, reference-image search, transcription and captions, sports highlights, architecture diagrams, and choosing between BDA and service-specific analysis. It also standardizes unified multi-file responses, requires confirmation before cleanup, reuses prior results for follow-up questions, and applies scope and upload-progress guardrails.
With this configuration, the agent handles the routing, tool sequencing, and response synthesis autonomously. Adding a new capability (for example, detecting on-screen text) requires only defining a new tool function and adding it to the tools list. No workflow logic changes are needed.
Each AWS service is exposed to the agent as a Python function decorated with @tool. The function signature defines the parameters, and the docstring tells the agent when and how to use it. This docstring is critical: It serves as the agent’s instruction manual for the tool. The following example shows the face search tool that wraps Amazon Rekognition:
from strands.tools import tool
import boto3
@tool
def search_faces_in_video(
video_s3_key: str,
collection_id: str,
confidence_threshold: float = 80.0
) -> dict:
"""Search for a specific person in video footage.
Use this tool when the user provides a reference photo
and asks whether that person appears in a video.
Requires a face collection created first via
analyze_reference_image.
Args:
video_s3_key: S3 key of the uploaded video
collection_id: Rekognition collection with the
indexed reference face
confidence_threshold: Minimum confidence for a
match (default 80%)
Returns:
Dict with matched_faces containing timestamps
and confidence scores for each appearance
"""
rek = boto3.client("rekognition")
response = rek.start_face_search(
Video={"S3Object": {
"Bucket": BUCKET, "Name": video_s3_key}},
CollectionId=collection_id,
FaceMatchThreshold=confidence_threshold
)
job_id = response["JobId"]
# Poll for completion and collect results...
return {"matched_faces": matches}The following example shows the BDA tool, which provides comprehensive video analysis (summary, chapters, and transcript) in a single API call:
@tool
def analyze_with_bda(
s3_uri: str,
analysis_types: list[str] = ["SUMMARY", "CHAPTERS", "TRANSCRIPT"]
) -> dict:
"""Perform comprehensive video analysis using Amazon Bedrock Data Automation.
Use this tool when the user wants a full video summary, chapter
breakdown, or complete transcript, or when Rekognition/Transcribe
are unavailable. Returns all requested analysis types in one call.
Args:
s3_uri: S3 URI of the uploaded video (s3://amzn-s3-demo-bucket/key)
analysis_types: List of analysis types to perform.
Options: "SUMMARY", "CHAPTERS", "TRANSCRIPT", "IAB"
Returns:
Dict with keys for each requested analysis type,
including timestamps and confidence scores
"""
bda = boto3.client("bedrock-data-automation-runtime")
response = bda.invoke_data_automation_async(
inputConfiguration={"s3Uri": s3_uri},
dataAutomationConfiguration={
"dataAutomationArn": BDA_PROJECT_ARN
},
outputConfiguration={"s3Uri": f"s3://{BUCKET}/bda-output/"}
)
invocation_arn = response["invocationArn"]
# Poll for completion...
result = poll_bda_result(invocation_arn)
return {
"summary": result.get("summary"),
"chapters": result.get("chapters", []),
"transcript": result.get("transcript")
}To illustrate how the agent chains multiple tool calls, consider a user who uploads a reference photo and asks “Did this person appear in my security footage?” The agent must first index the reference face, then search for it in the video. These are two sequential operations that depend on each other. The following trace shows the agent’s internal reasoning:
[Think] This is a face-matching query. I need to index
the reference photo first, then search the video.
[Act] analyze_reference_image(image="reference.jpg")
[Observe] {"faces_detected": 1, "face_id": "face_001",
"confidence": 99.8}
[Think] Face indexed. Now searching the video.
[Act] search_faces_in_video(
video_s3_key="security_cam_july5.mp4",
collection_id="user_collection")
[Observe] {"matched_faces": [
{"timestamp": "00:14:32", "confidence": 97.2},
{"timestamp": "00:47:15", "confidence": 94.8}]}
[Response] Yes, the person appeared twice: at 14:32
(97% confidence) and 47:15 (95% confidence).The agent determined the correct sequence of operations and handled the dependency between them (the face search requires an indexed collection). It then presented the results conversationally. No application code defined this sequence. The model reasoned through it based on the tool descriptions and the user’s question.
For comprehensive analysis (when the user asks “analyze this video” or “summarize this recording”), the agent can invoke Amazon Bedrock Data Automation (BDA) instead of calling Amazon Rekognition and Transcribe separately. BDA produces a video summary, chapter-by-chapter breakdown with timestamps, and full transcript in a single asynchronous API call:
[Think] The user wants a full summary. BDA provides summary +
chapters + transcript in one call, more efficient than
running Rekognition and Transcribe separately.
[Act] analyze_with_bda(s3_uri="s3://amzn-s3-demo-bucket/meeting.mp4")
[Observe] {"summary": "Team discussed Q3 roadmap...",
"chapters": [{"title": "Introductions", "start": "00:00"},
{"title": "Roadmap Review", "start": "05:32"}, ...],
"transcript": "Welcome everyone. Let's start with..."}
[Response] Here's the meeting summary with chapters:
Summary
The team discussed the Q3 roadmap...
Chapters
- 00:00 - IntroductionsWhen results are ambiguous, the agent communicates uncertainty explicitly. A borderline confidence score (for example, 62 percent) produces a qualified answer: “I found a possible match at 14:32, but the confidence is low, so you may want to verify manually.” If transcription fails because of poor audio, the agent suggests alternatives: “The audio quality is too low for reliable transcription. Would you like me to try visual analysis of the presentation slides instead?”
The agentic pattern applies broadly to scenarios where users need to extract specific information from video content without knowing in advance which analysis type is required.
Meeting intelligence – A team member joining a project mid-stream uploads prior meeting recordings and asks targeted questions: “What architecture decisions were made in April?”, “When did the team agree to use GraphQL?”, or “Summarize discussions about the authentication approach.” The agent transcribes, searches, and summarizes, returning answers with timestamps that reference the specific moment in the recording.
Security and access monitoring – A building manager uploads lobby camera footage with a photo of an expected visitor and asks “Did this person enter the building this week? When?” The agent runs face matching against the video and returns specific timestamps with confidence scores.
Claims investigation – An insurance adjuster uploads dash-cam footage and asks “Describe the sequence of events before the collision” or “Which vehicle was in the wrong lane?” The agent combines visual scene analysis with audio (verbal reactions, horns) to reconstruct the event timeline.
The three examples above all analyze video, but nothing about the architecture is video-specific. The agent selects tools from their docstrings, so adding a new capability (or a new modality entirely) is a matter of wrapping another service as a @tool function and describing when to use it. No workflow logic changes. The same orchestrator, cache, and per-user isolation apply unchanged.
That makes the pattern a general template for multi-modal AI assistants, using either AWS services or third-party models:
In each case the extension point is the tool contract, not the pipeline. A team that has built the video assistant already has the scaffolding (orchestration, caching, authentication, and per-user isolation) to stand up an AI assistant for a different modality by adding tools.
The per-query cost depends on which AWS services the agent invokes. After the initial analysis (transcription or visual processing), follow-up questions about the same video only incur Amazon Bedrock reasoning costs because results are cached. The following table shows approximate costs for a 60-minute video:
| Service | Operation | Approximate cost |
| Amazon Transcribe | 60-minute audio transcription | $1.44 |
| Amazon Rekognition | Face search (60-min video) | $6.00* |
| Amazon Rekognition | Label detection (60-min video) | $6.00* |
| Amazon Bedrock | Agent reasoning (per turn) | $0.05–$0.15 |
| Amazon S3 | Storage (500 MB, 24 hours) | <$0.01 |
The $6.00 Amazon Rekognition cost is one-time per-video costs (subsequent queries only incur Bedrock reasoning costs).
Based on AWS service pricing as of July 2025 and the preceding cost table, a typical transcript-based query on a 60-minute video costs approximately $1.50 for the initial transcription plus Bedrock reasoning. Subsequent questions about the same transcribed content cost only $0.05–$0.15 per turn, covering only the Bedrock inference call. Actual costs depend on model selection, input length, and AWS Region. For current pricing, see Amazon Bedrock pricing, Amazon Transcribe pricing, and Amazon Rekognition pricing.
The solution deploys on Amazon Elastic Container Service (Amazon ECS) with AWS Fargate. The Streamlit application and the agent runtime run in Fargate tasks behind an internal Application Load Balancer, and an Amazon CloudFront distribution is the only public entry point. CloudFront reaches the load balancer through a virtual private cloud (VPC) origin, so the load balancer stays in private subnets with no route to an internet gateway and isn’t directly reachable from the internet. CloudFront also terminates viewer TLS using its default *.cloudfront.net certificate, which provides a publicly trusted HTTPS endpoint without a custom domain or an AWS Certificate Manager certificate. Amazon Cognito handles authentication (invitation-only, with mandatory multi-factor authentication (MFA) through a time-based one-time password (TOTP) by default), and uploads and cached output are stored in Amazon S3 under per-user prefixes with a 24-hour lifecycle policy.
A single script (./deploy/deploy-ecs.sh) builds the container image, pushes it to Amazon Elastic Container Registry (Amazon ECR), and deploys the AWS CloudFormation stacks. Deployment typically completes in 15–20 minutes, most of which is CloudFront propagation. Full deployment prerequisites, the AllowSelfSignup parameter and its trade-offs, and step-by-step instructions are in the repository README.
Kiro is an AI-powered development environment that supports spec-driven software development by turning high-level ideas into structured requirements, designs, and implementation tasks. We used its spec workflow, persistent project context, and agent hooks to move from concept to a deployable sample while building security into each capability as it took shape.
Specs defined each capability before implementation. The face-matching spec defined inputs (reference photo plus video), expected behavior (index the face, search, and return timestamps), and edge cases (no face detected, low-confidence matches). The transcription spec covered multi-language detection, speaker diarization, and cache behavior for repeated queries. Kiro generated implementation tasks from each spec and maintained context across the full feature lifecycle. Based on the team’s prior experience building similar integrations, this compressed what they estimated would typically be a multi-week effort into a focused sprint.
Threat modeling ran alongside the specs, not after them. As each capability was specified, we modeled how it could be abused and captured the result in a living threat model (see docs/threat-model.md in the companion repository). The model works through concrete kill chains (authentication bypass, network exposure, agent exploitation through prompt injection, over-privileged IAM, and audit evasion) and assigns each threat a disposition. Every Critical and High finding was remediated in the sample. The items that remain open are recorded there with an explicit decision (accepted residual, or a documented production change). The controls described in the next section are outputs of that process rather than an afterthought.
Security scanning was embedded in the development loop. Kiro Hooks ran automated static and infrastructure-as-code scans on changes as they landed, using tooling such as the Automated Security Helper (ASH), and the container image repository is created with scan-on-push enabled. Findings came back as tasks in the same workflow that produced the feature, so a misconfiguration surfaced while the code was being written instead of in a separate review at the end. The net effect is a shift-left posture: A single small team held feature velocity and security rigor in one workflow, and secure-by-design was the default path rather than an extra gate.
Video often contains sensitive business discussions and identifiable people, so a multi-user deployment must control access, isolate each user’s data, and limit the effect of any one user’s actions.
The reference deployment implements four primary controls:
Production deployments should still decide whether to use a custom domain with a stricter viewer TLS policy, attach AWS WAF, expand audit logging and monitoring, and define data-handling requirements for face collections, transcripts, and cross-Region Amazon Bedrock model inference. Work with your legal and privacy teams to define applicable consent, retention, deletion, and residency requirements. For the full control-by-control analysis, accepted residuals, kill chains, and hardening checklist, see docs/threat-model.md and the README’s Security Considerations for Production in the companion repository.
Continued at the source.
AI coding agents have become a core part of how developers write, debug, and refactor software. Open weight models on Amazon Bedrock now make these agents practical to run privately and cost-effectively. But most options require you to send your proprietary data to a third-party API, lock you into a single model provider, or charge per-seat subscriptions regardless of how much you use them. If you have data residency requirements, cost-sensitive workloads, or a need for model flexibility, these constraints create real friction.
What if you could run an AI coding agent that keeps your data in your own AWS account, switches between frontier open weight models on demand, and charges only for what you consume?
OpenCode is an open source, terminal-native AI coding agent built in Go. It reads and edits files, runs shell commands, and understands project structure through Language Server Protocol (LSP) diagnostics. It connects to over 75 large language model (LLM) providers including Amazon Bedrock. When you pair OpenCode with open weight models on Bedrock, you get a coding assistant that runs locally while inference happens securely within your AWS account. There’s no infrastructure to manage and no per-seat fees.
In this post, we show you how to set up OpenCode with open weight models on Amazon Bedrock, configure multi-model workflows that match the right model to each task, and walk through practical coding examples using Moonshot AI Kimi K3, OpenAI GPT-OSS 120B, and NVIDIA Nemotron 3 Super 120B. We also share how Ethara.AI deploys this architecture in production with multi-agent orchestration to power AI engineering and research workflows at scale.
The industry is shifting toward open weight models. According to McKinsey’s Open-source technology in the age of AI report (2025), 76 percent of organizations expect to increase open source AI usage, and leading AI adopters are 40 percent more likely to use open weight models. For coding workloads, five factors drive this shift:
Performance parity: Fine-tuned open weight models can outperform proprietary alternatives on domain-specific tasks. CrowdStrike’s fine-tuned NVIDIA Nemotron achieved 96% valid query accuracy, outperforming GPT-4o (61%) and Claude Sonnet 4.5 (94%).
Cost efficiency: According to Gartner’s 2026 analysis, agentic workflows multiply token consumption 5–30x, making cost-per-token critical. At scale, on the order of multimillion conversations per month, switching to open weight models on Bedrock can reduce annualized costs.
Customization and control: Open weights support fine-tuning, distillation, and domain adaptation. Smaller models can replace expensive general-purpose ones while maintaining quality.
Model flexibility: With open weights, you can adopt the right model for each task and evolve as new ones emerge. Switching models is a single API parameter change on Amazon Bedrock.
Transparency: Inspectable model architecture and behavior supports regulated industries with AI governance requirements.
Amazon Bedrock provides fully managed, serverless access to open weight models. There’s no GPU provisioning or inference infrastructure to manage. For enterprise coding workflows, Bedrock offers several advantages over self-hosting or direct model providers:
Data residency and compliance: Code, prompts, and responses stay in your AWS account. Models accessed through an in-Region or geographic profile run in that Region or geography. You can invoke Kimi K3 through a cross-Region inference profile. For workloads without regional restrictions, we recommend using the global profile, global.moonshotai.kimi-k3, which routes each request to any supported commercial AWS Region worldwide. Global cross-Region inference costs approximately 10% less than a geographic profile. The US geographic profile, us.moonshotai.kimi-k3, keeps processing within the US geography for data residency requirements. Amazon Bedrock is in scope for common compliance programs including HIPAA, SOC 2, ISO 27001, FedRAMP, and GDPR. For the full list, see AWS services in scope by compliance program.
Enterprise security controls: Open weight models inherit the same AWS Identity and Access Management (IAM) policies, AWS CloudTrail logging, AWS PrivateLink connectivity, and encryption controls as proprietary models. No separate security stack required.
Flexible pricing: Three tiers match cost to workload: Priority for latency-sensitive production, Standard for on-demand inference (pay per token), and Flex at 50 percent lower cost for variable-latency workloads.
No model training on your data: Bedrock doesn’t use your inputs or outputs to train or improve foundation models (FMs).
High default capacity: Default limits of 100M tokens per minute and 10K requests per minute help reduce throughput bottlenecks as teams scale.
Not every coding task needs the same model. One of the key advantages of using OpenCode with Bedrock is the ability to select and switch between models based on what you’re doing.
Where to evaluate models: The Artificial Analysis Coding Index provides a composite benchmark across real-world software engineering tasks (SWE-Bench, Terminal-Bench, SWE-Atlas). Use it to compare model performance, cost per task, and latency. For evaluations against your own prompts and data, you can use Amazon Bedrock Evaluations to run side-by-side comparisons with automatic scoring, LLM-as-a-judge, or human review.
reasoning_config (low, high, or max). You trade latency for correctness on hard problems.For this post, we feature three models that cover the spectrum:
| Model | Strength | Best for |
| Moonshot AI Kimi K3 | Always-on reasoning with a 1M-token context | Debugging, architecture analysis, complex logic |
| OpenAI GPT-OSS 120B | Frontier code generation (120B parameters) | Full-service generation, multi-file implementations |
| NVIDIA Nemotron 3 Super 120B | Optimized throughput, enterprise-validated | High-volume generating workloads |
The architecture has two parts: OpenCode runs as a terminal user interface (TUI) on your local machine and calls the Amazon Bedrock Converse API for inference. Bedrock hosts the models as fully managed, serverless endpoints.
Figure 1: Solution architecture for OpenCode with open weight models on Amazon Bedrock. The developer interacts with OpenCode in the terminal, which sends requests through the Bedrock Converse API to open weight models. AWS IAM authenticates each request, and AWS CloudTrail logs API activity
OpenCode’s agent architecture supports assigning different models to different roles: a reasoning model for planning and a faster model for code generation. This creates a multi-model workflow within a single session.
Before you start, make sure you have:
This section walks through installing the OpenCode CLI and configuring it to authenticate with Amazon Bedrock.
Choose one of the following methods:
# Install script (recommended)
curl -fsSL https://opencode.ai/install | bash
# Or via npm
npm install -g opencode-ai
# Or via Homebrew (macOS/Linux)
brew install sst/tap/opencodeVerify the installation:
$ opencode --version
1.18.20OpenCode’s Bedrock provider uses the standard AWS credential chain. Configure one of the following options:
aws sso login --profile my-bedrock-profile
export AWS_PROFILE=my-bedrock-profile
export AWS_REGION=us-west-2export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_REGION=us-west-2Replace these with your own credentials. For production use, prefer IAM Identity Center or IAM roles over long-lived access keys.
export AWS_BEARER_TOKEN_BEDROCK=your-bedrock-api-key
export AWS_REGION= us-west-2You can also store these in a .env file in your project root for persistent configuration.
With OpenCode’s agent system, you can assign different models to different roles. Create or edit opencode.json in your project root:
{
"$schema": "https://opencode.ai/config.json",
"model": "amazon-bedrock/us.openai.gpt-oss-120b-1:0",
"agent": {
"plan": {
"model": " amazon-bedrock/global.moonshotai.kimi-k3"
},
"build": {
"model": "amazon-bedrock/us.nvidia.nemotron-super-3-120b"
}
}
}This configuration routes planning and architecture tasks (which benefit from deep reasoning) to Kimi K3, while code generation and implementation go to Nemotron 3 Super 120B for optimized throughput. The top-level model field sets GPT-OSS 120B as the default for other context.
To browse available Bedrock models interactively, launch OpenCode and enter /models.
The following examples show how to match each model to the kind of task it handles best.
GPT-OSS 120B is OpenAI’s 120-billion parameter open weight model. It combines strong reasoning with code generation, making it well-suited for architecturally complex implementations that span multiple files. With the multi-model configuration from the previous section, you can override the default model inline or use /model to switch explicitly:
$ opencode
> /model us.openai.gpt-oss-120b-1:0
> Build an event-sourced CQRS order service in Python (FastAPI) with:
> - Command side: append-only event store in DynamoDB, idempotent handlers
> - DynamoDB Streams triggering a projection Lambda that builds a read-model
> - Query side: denormalized read-model optimized for "orders by customer"
> and "orders by status" access patterns
> - Event replay CLI to rebuild projections from scratch
> - Snapshotting every 50 events per aggregate to bound replay time
> Include CDK infrastructure.OpenCode routes this to GPT-OSS 120B through the Bedrock Converse API with IAM authentication. The model generates the full service structure, including handlers, event store, projections, and CDK stack, directly into your local file system. CloudTrail logs the invocation, and your prompts and responses remain within your AWS account.
Kimi K3 excels at reasoning tasks that require tracing through multiple execution paths. It reasons on every turn, and the reasoning_config field sets how deep the reasoning goes: low for quick passes, max for the hard ones. If you configured Kimi K3 as your plan agent, it’s already the default for analysis tasks. You can also switch explicitly with /model:
> /model amazon-bedrock/global.moonshotai.kimi-k3
> This Step Functions workflow hangs ~2% of the time under load.
> The pattern: Task A does a DynamoDB conditional put that expects
> status="PENDING", Task B (triggered by SQS) sets status="READY"
> but only after Task A's callback confirms receipt. Both tasks
> wait on each other. Trace the deadlock, explain why it only
> manifests under concurrency, and propose a fix that doesn't
> require redesigning the state machine.
> @order_workflow.asl.json @task_a_handler.py @task_b_handler.pyThe @file references inject your local code as context without manual copy-paste. Kimi K3’s Mixture-of-Experts architecture activates only 104B of its 2.8T total parameters per token, delivering frontier reasoning at efficient throughput. You watch the model trace through the concurrency paths live. Because the reasoning is visible, you can judge whether the analysis holds before accepting the proposed fix.
You don’t need to commit to a single model. Enter /models during a session to switch. A practical pattern: use Nemotron 3 Super 120B or GPT-OSS 120B for fast code generation and boilerplate, then switch to Kimi K3 when you hit a complex debugging problem or need to reason about architectural trade-offs.
With the multi-model opencode.json configuration shown earlier, this routing happens automatically. The plan agent uses Kimi K3 for reasoning, while the build agent uses Nemotron for implementation.
Some coding work is interactive. Batch refactoring across a large codebase, generating test suites for existing modules, or producing documentation from code. These tasks are latency-tolerant and can run asynchronously. The Amazon Bedrock Flex tier offers 50% lower cost than Standard for these workloads.
You can combine this with OpenCode’s CLI mode to script batch operations:
# Process multiple files through GPT-OSS 120B for test generation
for file in src/**/*.py; do
opencode run -m amazon-bedrock/us.openai.gpt-oss-120b-1:0 \
"Generate comprehensive unit tests for @${file}. Use pytest with fixtures."
doneStack your tiers: Standard for interactive sessions, Flex for batch processing, and Priority for latency-sensitive production use.
The OpenCode + Bedrock pattern shown in this post is a single-developer workflow. For teams and production systems, the same multi-model principle extends to a routing architecture where an orchestrator directs each sub-task to the optimal model:
Figure 3: Multi-model routing architecture using open weight models on Amazon Bedrock. A router analyzes incoming requests and dispatches sub-tasks to the optimal model tier to help reduce total cost of ownership compared to routing all tasks through a single model
In this pattern:
This routing can help reduce overall total cost of ownership (TCO) compared to sending everything through a single expensive model without degrading quality. The Amazon Bedrock unified API makes this practical: switching models is a parameter change, and the models share the same authentication, logging, and guardrails infrastructure.
For teams ready to go beyond single-developer use, combine OpenCode’s local agent routing with a server-side orchestration layer (Amazon Bedrock Agents or AWS Step Functions) to create a full multi-model coding pipeline.
Ethara.AI, an AWS customer, deploys this architecture in production, using OpenCode as the foundational runtime for AI engineering and research workflows. Oh-My-OpenAgent serves as the orchestration layer for working with specialized AI agents. Rather than relying on a single coding assistant, Ethara.AI operates a fleet of agents optimized for different tasks such as planning, execution, code review, architecture analysis, knowledge retrieval, multimodal understanding, and benchmarking. Through Oh-My-OpenAgent’s category-based routing system, engineers request a capability (such as deep reasoning, rapid execution, visual engineering, or writing assistance), and the system delegates the work to the most suitable agent. This abstraction helps teams focus on outcomes rather than model management, while maintaining flexibility across evolving AI frameworks.
Amazon Bedrock and OpenCode’s provider-agnostic architecture powers Ethara.AI’s model selection strategy. OpenCode supports access to a broad range of foundation models, while Amazon Bedrock provides secure access to frontier models. Instead of a single model, they dynamically route workloads based on factors such as reasoning complexity, latency requirements, cost efficiency, and task type. The combination of Amazon Bedrock, OpenCode, and Oh-My-OpenAgent helps them separate agent capabilities from the underlying model layer. This helps make sure that the most appropriate agent-model combination executes each task, while retaining the ability to evaluate and adopt new models as the model landscape evolves.
Looking ahead, Ethara.AI is investing in self-improving agent systems that draw on research such as SkillClaw. They are developing mechanisms that help skills and agent behaviors evolve based on successful and unsuccessful execution trajectories, creating an infrastructure where agents continuously improve through real-world usage. This vision builds on OpenCode’s extensible architecture, Oh-My-OpenAgent’s delegation framework, and the Amazon Bedrock model catalog to create AI systems that become more capable, adaptive, and efficient over time.
Open weight models on Bedrock inherit identical enterprise controls as proprietary models. The provenance of the weights doesn’t change your security posture. Restrict which models users can invoke with IAM policies that follow least-privilege:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": [
"arn:aws:bedrock:us-west-2::foundation-model/openai.gpt-oss-120b-1:0",
"arn:aws:bedrock:us-west-2::foundation-model/nvidia.nemotron-super-3-120b",
"arn:aws:bedrock:us-west-2:{account-id}:inference-profile/global.moonshotai.kimi-k3",
"arn:aws:bedrock:*::foundation-model/moonshotai.kimi-k3"
]
},
{
"Effect": "Allow",
"Action": ["bedrock:CallWithBearerToken"],
"Resource": ["*"]
}
]
}You can further layer Amazon Bedrock Guardrails for content filtering and personally identifiable information (PII) redaction across invocations. CloudTrail records every InvokeModel call for audit, and Bedrock does not use your inputs or outputs to train models.
This walkthrough doesn’t create persistent AWS infrastructure beyond model access enablement. If you enabled model access solely for testing, you can disable it in the Amazon Bedrock console under Model catalog. No other resources require cleanup, and you incur no charges when you’re not making API calls.
We showed how to configure OpenCode with open weight models on Amazon Bedrock to build a secure, flexible, pay-per-use AI coding workflow. You get the cost efficiency and customization potential of open weight models, the enterprise security and managed infrastructure of Bedrock, and a terminal-native experience that fits into existing developer workflows with the ability to route different tasks to different models automatically.
To get started, install OpenCode, configure your AWS credentials, and set up a multi-model configuration. Explore the full Amazon Bedrock model catalog to find models suited to your workloads, and use the Artificial Analysis Coding Index to compare model performance on coding tasks.
For more information about Amazon Bedrock security and compliance, refer to the Amazon Bedrock User Guide. If you’d like to discuss how Amazon Bedrock can support AI-assisted development in your organization, contact an AWS Representative.
Aris is a Senior Specialist Solutions Architect for Generative AI focusing on open source models on Amazon Bedrock and the broader generative AI open source community. Alongside his professional role, he is pursuing a PhD in Machine Learning Engineering at the University of Regensburg, where his research focuses on applied natural language processing in scientific domains.
Sainath is a Senior Technical Account Manager at AWS, where he partners with automotive enterprises to accelerate autonomous driving initiatives that advance road safety at scale. He specializes in architecting large-scale distributed systems powered by AI/ML — helping customers translate complex technical requirements into production-ready solutions. Outside of work, Sainath enjoys spending time with family and friends.
Saurabh is a Senior Product Manager for Amazon Bedrock and Amazon SageMaker Inference. He is passionate about working with customers and partners, motivated by the goal of democratizing AI. He focuses on core challenges related to deploying complex AI applications, inference with multi-tenant models, cost optimizations, and making the deployment of generative AI models more accessible. In his spare time, Saurabh enjoys hiking, learning about innovative technologies, following TechCrunch, and spending time with his family.
Suryansh is the Co-Founder and CEO of Ethara.AI, where he leads the company’s work in reinforcement learning and training increasingly capable AI systems. A UCLA-trained engineer, he previously developed electric vehicle power electronics and battery systems at Canoo and Romeo Power. He brings a hands-on engineering mindset to building reinforcement learning environments that help AI models reason, use tools and complete complex tasks. His focus is on closing the gap between what AI can generate and what it can reliably accomplish.
An AI coding agent’s patch can pass tests yet fail when the server loads a real model and handles requests. Evaluating changes to inference-serving software therefore requires checking the full serving path, including whether the system returns correct results through its public interface.
Developed with input from the SGLang team, SWE-Serve evaluates this gap with 53 tasks derived from merged changes to SGLang, an open-source system for serving large language models. Across 19 tasks with live-serving checks, the same patches passed 69.4% of the time when those checks were excluded, but only 45.9% with the complete verifier. About one in three patches that passed the other checks failed live-serving tests.
Quick links: Read the paper | Explore the leaderboard | Run SWE-Serve on GitHub
Existing repository-level benchmarks evaluate coding agents across general software-engineering tasks, while inference benchmarks often concentrate on kernel generation or performance optimization. SWE-Serve instead tests repository-scale changes across the inference-serving stack, including model enablement, decoding, caching, scheduling, serving APIs, and runtime performance.
To evaluate this broader engineering work, SWE-Serve turns 83 merged SGLang pull requests into 53 executable tasks across six inference-engineering families.
| Engineering family | Tasks |
|---|---|
| Speculative and advanced decoding | 14 |
| Model and backend enablement | 12 |
| Kernels, quantization, and performance | 8 |
| Serving APIs and runtime correctness | 8 |
| Caching and runtime state | 7 |
| Distributed execution and scheduling | 4 |
Twelve tasks run on CPU, while 41 use a single NVIDIA H100. This first release doesn’t evaluate other inference engines, multi-GPU execution, or multi-node serving.
Thirty-seven tasks come from a single upstream pull request. The other 16 combine two to six related changes. In total, the benchmark draws on 83 merged SGLang pull requests. The SGLang team, a launch partner for SWE-Serve, contributed ideas for identifying challenging tasks, suggested particularly demanding pull requests, and helped shape our approach to verifying correctness.
Each task gives the agent an instruction and a containerized SGLang checkout from before the target change. The agent’s patch passes if it satisfies the task’s hidden verifier on the declared hardware. It is never compared with the reference implementation.
These are substantial changes. The median reference solution modifies 553 lines across seven files. A typical verifier has seven tests for the new behavior and 10 regression tests. Nineteen tasks start a real server, and three enforce a calibrated performance gate on an H100.
One task asks the agent to add serving support for dense and mixture-of-experts (MoE) Qwen3.5 models. Starting from a revision without Qwen3.5 support, the agent must make both the 0.8B dense model and the 35B-A3B MoE model load and serve through the normal SGLang interfaces on one H100.
Its verifier checks model registration, configuration and weight loading, image and video inputs, OpenAI-compatible requests, native batched generation, log probabilities, and execution through the MoE model’s routed experts.
Some failures only appear once a real server starts. The SGLang team contributed ideas for end-to-end verification, including recommendations for specific model-serving tests. SWE-Serve includes 19 tasks that load the required model and test the agent’s patch through a live serving interface.
Across these tasks, the same 627 patches pass 45.9% of the time under the complete verifier. When the live serving tests are excluded, that rate rises to 69.4%. In other words, 147 patches changed from fail to pass when the live serving tests were excluded.
The 19 tasks contain 276 live serving tests. Of those, 242 are sourced or adapted from SGLang. The remaining 34 cover behavior introduced by the corresponding merged changes when no suitable upstream test was available.
The Gemma 4 MoE task makes the result concrete. Sixteen of 33 patches passed every other check but failed at least one live serving test. Those tests cover model loading, expert routing, text and image serving, and batched generation with correct ordering and log probabilities.
A SWE-Serve pass has a narrow meaning: the patch satisfies the benchmark verifier. SWE-Serve’s tests are not SGLang’s upstream review process. They don’t establish that an agent patch or benchmark reference solution is deployable, ready to merge, or endorsed by SGLang maintainers.
We divided the request-to-output path into four runtime domains: request handling and I/O, scheduling and request lifecycle, model execution, and KV-cache and runtime-resource management.
Across the best setting for each of the 11 models, the 26 tasks confined to one runtime domain have a 69.0% pass rate. The 27 tasks spanning more than one runtime domain have a 47.7% pass rate, a difference of 21.3 percentage points. Every model setting shows the same direction of difference.
Model performance varies substantially. Across each model’s best tested configuration, mean pass@1 ranges from 34.6% to 75.5%. No model scores highest in every task category, and configurations with similar overall scores can have very different costs and runtimes.
We evaluated 11 models and 31 model-effort configurations with mini-swe-agent, a minimal software-engineering agent that uses only Bash, under closed-book conditions. We tested the two Claude and three GPT-5.6 models at five effort levels; the other six models ran at one setting each.
Each configuration ran the complete 53-task benchmark three times. Each agent session was capped at 210 minutes and 350 steps. A task counts as solved only when the agent’s patch passes the complete verifier on the task’s declared hardware. The table reports the highest-scoring effort setting for each model.
| Model | Reasoning setting | pass@1(mean ± SD, 3 runs) | Mean cost/task | Mean wall time |
|---|---|---|---|---|
| Claude Opus 5 | max | 75% ± 3% | $17.40 | 57.5 min |
| GPT-5.6 Sol | max | 75% ± 6% | $12.26 | 29.5 min |
| Claude Sonnet 5 | xhigh | 64% ± 3% | $6.61 | 40.6 min |
| Kimi K3 | max | 64% ± 5% | $7.24 | 99.9 min |
| GPT-5.6 Luna | max | 64% ± 4% | $0.95 | 28.9 min |
| GPT-5.6 Terra | max | 64% ± 4% | $5.06 | 25.5 min |
| DeepSeek V4 Flash (0731) | max | 55% ± 4% | $0.69 | 36.4 min |
| GLM-5.2 | max | 48% ± 2% | $2.10 | 34.0 min |
| Gemini 3.6 Flash | high | 48% ± 6% | $4.84 | 37.3 min |
| Laguna S 2.1 | max | 46% ± 5% | $0.33 | 56.9 min |
| Inkling S | xhigh | 35% ± 3% | $0.44 | 17.6 min |
Native harnesses didn’t improve the two leaders: GPT-5.6 Sol scored 73.6% in Codex and Claude Opus 5 scored 69.8% in Claude Code, versus 75.5% each with mini-swe-agent.
Cost doesn’t map cleanly to performance. Among the four models tied at 64%, mean cost ranges from $0.95 to $7.24 per task, while mean wall time ranges from 25.5 to 99.9 minutes.
No model leads all six engineering families, and models with the same overall score can have different strengths. The top score shows that many SWE-Serve tasks are within reach of current agents; the spread shows that performance is far from uniform.
We screened 786 potential task sources, built 156 executable candidates, and admitted 53.
Every admitted task was tested on its declared hardware. The unmodified repository had to fail the tests for the new behavior while continuing to pass regression tests. A reference patch had to pass the complete verifier. We also challenged the verifiers with agent-created patches, repairing, narrowing, or excluding tasks when we found a concrete problem.
Reported evaluations are closed-book. We block the public web and upstream source repositories, while allowing Hugging Face access for model weights, because an open-network pilot showed models retrieving task-specific upstream code. We audited all 1,749 trials behind the leaderboard; 196 prohibited retrieval attempts were blocked, and none succeeded. The paper describes the full qualification and evaluation-integrity process.
SWE-Serve includes the task environments, verifiers, and baseline configurations. It makes the gap between passing local checks and working through the full serving path measurable.
Explore the SWE-Serve leaderboard, then run SWE-Serve on GitHub to evaluate your coding agent on SGLang inference-engineering tasks.
Agents run in a loop: an LLM decides what to do, a tool executes, a model evaluates the results, and then continues in that loop until the task is complete.
Agents and LLMs were initially difficult to integrate into software applications, which depend on structured data and predictable interfaces. Two primitives emerged that made this much easier:
But even with those in place, the agent loop is still slow and costly: every decision requires another model call.
Enter, Jev. Jev is a new model released from TypeSafe AI. The company reports up to 200x faster inference and 400x lower cost than comparable LLMs on classification tasks.
This post covers how Jev works, where it fits into the agent loop, and how to use it with LangChain.
Jev is actually not a traditional LLM, it doesn’t generate text. It’s what the TypeSafe AI team calls a System One model:
📖 System One models are a class of AI models built to make fast, structured decisions that software can use directly. A System One model evaluates a state and returns typed answers and probabilities.
It’s trained using reinforcement learning for calibrated decisions (RLCD). Your code uses those results to guide what an agent does next, without a full chat LLM call for each decision.
To invoke a Jev model, you send it a state (the context) and questions about that state. Here’s a single-question version of the support-ticket example in their docs:
{
"model": "jev-latest",
"state": "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.",
"questions": {
"is_urgent": {
"type": "noul",
"instructions": "The message conveys urgency or time-sensitivity"
}
}
}The docs’ example gives this urgency answer, shown here without the rest of the response:
{
"is_urgent": {
"type": "noul",
"noul": 0.999
}
}That’s a 99.9% probability that the message is urgent, which your application can use to prioritize the ticket.
There are three types of supported questions:
One key feature here is that you can ask multiple questions about the same state in one request.
💡 System One models evaluate every question in a request in parallel. Adding questions barely changes the response time and costs only the tokens for the extra questions, which are cheap.
For an example of asking multiple questions about a support ticket, see the TypeSafe Quickstart.
In sum, unlike traditional LLMs, Jev is neither constrained by text generation or sequential decision making!
LangChain's provider agnostic model is well suited for supporting Jev alongside thousands of other integrations and model providers.
The LangChain integration exposes Jev through TypeSafeClassifier. You pass your state and questions to .invoke(), and get classification results rather than a chat response.
Install langchain-typesafe and set your TYPESAFE_API_KEY, then make a call:
from langchain_typesafe import Noul, TypeSafeClassifier
classifier = TypeSafeClassifier()
response = classifier.invoke({
"state": (
"The deploy failed twice and customers are seeing 500s. "
"Can someone look now?"
),
"questions": {
"urgent": Noul(
instructions="Does this need attention right now?"
),
},
})
urgency = response.nouls["urgent"].noulThe state can be text, structured data, or LangChain messages. That makes it straightforward to call Jev from a node or middleware hook using the context your agent already has.
You can build this into custom middleware or tools!
Jev isn’t a drop-in replacement for an LLM. It doesn’t generate text, but it can handle classification tasks we often use LLMs for today, without the same latency and cost. That makes it a promising complement to the model driving your agent: use an LLM for open-ended reasoning and generation, and Jev for fast, structured decisions along the way.
A simple lookup doesn’t need the same model as a difficult debugging task. Model-routing middleware lets Jev assess the request and choose a model based on criteria you define, so fast and inexpensive for straightforward tasks, more capable for complex ones.
from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
ModelChoice,
ModelRouterMiddleware,
)
router = ModelRouterMiddleware(
choices={
"fast": ModelChoice(
model="openai:luna",
criteria="Direct lookups, extraction, and localized changes.",
),
"powerful": ModelChoice(
model="openai:sol",
criteria="Architecture and high-stakes decisions.",
),
},
instructions="Choose the least costly model that can complete the task.",
)
agent = create_agent("openai:gpt-5.6-luna", middleware=[router])The router selects a model from the latest user message and uses it throughout the run. The probabilities and confidence remain available in agent state, too.
Agents are still inherently untrustworthy. An agent can receive bad instructions (either naturally or from a motivated enough attacker) which can persuade it into taking actions we didn’t want it to.
Coding harnesses like claude, codex, cursor have shipped some kind of way to classify dangerous actions before they’re taken which has slowly helped to build trust in agents. Up until now, this classifier step has been locked away in the closed source parts of the harness.
Now that a cheap and performant classifier model exists, we can take the same pattern and adopt it to all agents!
from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
AutoModeMiddleware,
)
guardrail = AutoModeMiddleware(tools=["bash"])
agent = create_agent("openai:gpt-5.6-luna", middleware=[guardrail])AutoModeMiddleware uses Jev to check tool calls for risky decisions it may take, and block calls before the tool executes.
We're pretty thrilled about Jev and the possibilities that come with it. A few cool projects that we’ve seen already: Kyle Jeong from Browserbase is powering browser use agents for fractions of a cent, Jarrod Watts built a live trading agent, and Ryan Vogel is doing email triage at scale.
New models drop every week at this point, but this one had a pretty outsized response. We’re excited to see what you build with LangChain and Jev.
Let us know what you think on the forum, tag us on X and share what you’re building, or engage with LangChain issues!
September 23, 2026 by Julia Gong, Shengjie Ma, Ben Liggett, and Ulugbek Abdullaev
Completion-style ghost text, next edit suggestions near the cursor, and edits farther away were previously powered by separate models. We built one model for all three, and learned that the best results come from training, evaluation, and editor design evolving together.
In the first part of this blog post, we shared the first phase of the unified modeling effort: unifying the NES and long-distance edit behavior into a single 2-in-1 model.
Figure 1. We first unified the NES and long-distance NES models to yield the 2-in-1 unified model, followed by unifying the 2-in-1 model with the completions model to yield the 3-in-1 model.
The next step was to bring completion behavior into the 2-in-1 model. Previously, an edit opportunity could involve a completion request followed by another request for a richer 2-in-1 model edit. With the fully unified 3-in-1 model, one request can consider the full range of code editing behaviors and decide the best response at any given moment, or chain many of them together in a single response.
This greatly reduces model calls and serving complexity, but the biggest benefit isn't even the efficiency. A single model can optimize for the best edit that fits the moment instead of preserving the artificial boundaries that resulted from the system architecture.
Case in point, in our final 3-in-1 model candidate, we found that while the proportion of less-intrusive ghost text decreased in the 3-in-1 model compared to the 2-in-1 model setup, there were actually significant measurable improvements in user satisfaction. We will talk more about this later in this post.
To train the 3-in-1 model, we applied what we learned while developing the 2-in-1 model. As the goal was to add completion behavior to the 2-in-1 model, the distribution of the training data needed to change. We gathered ghost text completions data through a variety of techniques, including distilling ghost text from the original completions model and filtering them for quality using an LLM judge. We leveraged the original training recipes from the 2-in-1 model while adding this new ghost text data.
We also had to carefully rebalance the distribution of the multi-edit data inherited from the 2-in-1 model recipe. Models that showed too little ghost text or were too eager to jump away from the user's cursor often resulted in more disruptions to the user flow and thus higher dismissal rates. This naturally gave rise to a dedicated subset of multi-edit data where the first patch was a ghost text completion at the user's cursor. During RL, as mentioned earlier in part one, we also kept the grader design that encouraged the model's edit sequences to begin with the most immediate continuation of the developer's work, then move outward to related follow-up changes, which made edits least intrusive and kept more logical flow and continuity.
Early model candidates produced ghost text more often and more suggestions overall as a result. We measured these results offline with three benchmarks discussed in part one: STests, Output View Kind, and Pseudo-Online Evaluation. We also noticed that the HumanEval benchmark results improved because of having stronger completions capabilities.
When we flighted these v4 models against the now-production setup of the 2-in-1 model and the completions model, results were quite promising: no statistically significant changes in acceptance rate, dismissal rate, shown rate, and user engagement metrics. But we did find a marginally statistically significant regression in accumulated retained characters, or the number of characters that were not reverted by the user within a certain time interval after accepting a suggestion.
We considered several potential causes of this regression, including poorer suggestion quality, but because of the lack of movement in other metrics, we ultimately pulled the thread on one hypothesis in particular: suggestion length.
Our hypothesis was that because the model was producing slightly shorter ghost text suggestions compared to the production completions model, users might be accepting promising but incomplete suggestions. As a result, they might be more frequently reverting or editing the suggestions. For instance, writing the function signature and docstring is nice, but if we don't follow through with the full implementation, then the user might remove the function, make edits, or rewrite it manually.
To test this hypothesis, we applied what we learned from the 2-in-1 model's insertion challenges and gathered a subset of data that specifically targeted this type of longer-output scenario and performed a similar upweighting of the data. We further added another auxiliary grader dedicated to ghost text completeness and gated it to these specific curated samples. To measure the offline impact of this change, we also introduced a second version of the Output View Kind dataset that measured the character lengths and proportion of ghost text in model responses compared to the original distribution of the production model.
We flighted our most promising model candidate and were pleased to see that the regression in accumulated retained characters had decreased but was not eliminated, and the dismissal rates were trending higher. However, there were several observations made during the dogfooding experience that led us to look beyond the model itself. This reinforced an important lesson we had learned through iterating on our previous models: client behavior is just as important as the model.
The ultimate user experience is much more than just the model's capabilities: a great inline suggestions experience is the fusion of a great model with a great end-to-end system. Careful client orchestration is also necessary to show edits to the user at the best time and in the best way.
For example, when dogfooding the most promising 3-in-1 flight candidate, we noticed an important past client behavior decision for NES that was worth revisiting. When ghost text was shown to the user but ignored, if the user moved their cursor to a different location, that cached ghost text suggestion would be re-shown as an NES edit. Upon deeper investigation, we estimated that this client behavior alone could potentially be responsible for inflating the 3-in-1 model's dismissal rate by 7-8%. Once we ablated this issue in an A/B flight, we found the impact was even larger than we had estimated—the dismissal rate went from a 15.9% increase to a 10.1% decrease—a 26% difference—compared to the production 2-in-1 setup.
The reason for this behavior requires some historical context. Originally, the standalone NES model's suggestions could arrive asynchronously as the user moved their cursor, whereas the standalone completions model's ghost text disappeared when the cursor moved away. As a result, the ghost text from the standalone NES model was intentionally made persistent so it could be shown as the user moved their cursor. However, because the 3-in-1 model identified itself to the client as an NES model, it inherited this behavior despite its ability to also provide completion-style ghost text at the cursor. By ablating this NES ghost text persistence behavior through a client setting, nesMimicGhostTextBehavior, we were able to revisit and change this behavior to make a choice that best fit the new model behavior. This underscores the importance of experimenting with the feature end-to-end in the client—new models have new behavior, which might warrant different client settings.
Beyond this ablation study, we also did extensive testing to make the best client-side choices for the 3-in-1 model's edits, including optimizations in edit caching, speculative decoding, how to render the edit to the user, and progressive reveal of longer edits. Through multiple rounds of internal dogfooding and A/B testing, we discovered that each of these client choices made a big impact on the user experience.
We evaluated five client-side optimizations as a joint configuration that resulted in the best end-to-end unified model experience:
Speculative decoding: The prior standalone NES model already used speculative decoding to reduce suggestion latency, but the new unified model output format required us to revisit what output should be speculated. We ablated several variants of the speculated text with progressively stronger guidance toward completion behavior: only the file path header (without specifying any line, the default used for the 2-in-1 models), replacing the current line (specifying the current line but not a particular view kind), and completing the current line (specifying both the current line and the ghost text view kind). After load testing, latency analysis, view kind distribution analysis, and online flights comparing these options, we chose to speculate completion at the current line, which yielded the lowest suggestion latency without negatively impacting the user experience.
Ghost-text progressive reveal: Progressive reveal shows the immediately relevant portion of a longer ghost text suggestion first, then progressively reveals the remainder. This prevents the user from navigating a long block of text while making incremental progress on their work. Progressive reveal of ghost text was a critical client feature that underwent several rounds of optimization for the original standalone completions models, and its functionality was extended later to the standalone NES models—we thus ported over this technique to the unified model client behavior.
Diff-based edit rendering: For the standalone NES models, the client would take parse the code in the rewritten window to determine the code diff and thus the corresponding view kinds to show the edits to the user. This was required because raw rewritten code had to be parsed to show a minimal edit to the user. However, for the unified models with diff patch output formats, we had a choice—should we display the view kinds implied by the raw patches themselves, or perform this diff-based rendering per-patch as well? Earlier, we found that the model may generate suboptimal-efficiency patches in exchange for better overall suggestion quality (see the discussion on patch validity in part one of this post). For instance, the model might produce the following patch:
aquarium.py:2
-cuttlefish
+axolotl
+blobfish
+cuttlefish
+dolphin
If we used the raw patch, we would show a side-by-side "diff" view kind to the user, where cuttlefish is replaced by axolotol, blobfish, cuttlefish, and dolphin. However, if parsed, this could be optimized so that the user first sees this insertion ghost text:
aquarium.py:2
-cuttlefish
+axolotl
+blobfish
+cuttlefish
Followed by this insertion on a different line:
aquarium.py:4
-cuttlefish
+cuttlefish
+dolphin
Thus, we hypothesized rendering might serve as another safeguard in case there were an even stronger way to present the edit to the user than what was implied by the model output. Through online A/B experimentation, we found this was indeed a positive change for the user experience. Thus, we first interpret each change in the patch and then present it through the appropriate rendered interaction (for example, ghost text, a nearby rewrite, a farther rewrite, a cross-file suggestion, or no suggestion), rather than rendering edits based on the model's raw patch structure.
Cache delay and debounce settings: Immediate presentation of suggestions can be disruptive when they arrive too quickly, which results in a worse user experience. A slight delay before presenting a cached suggestion better matches the developer's typing rhythm to help them stay in-flow, especially with the decreased suggestion latency from optimizing the speculative decoding. This setting was already in use, and we ablated several configurations of cache delay durations to arrive at a configuration tailored to the unified models. We did the same for debounce settings—whether to delay calling the model for a suggestion while the user is still typing—and found that immediate model invocation while the user was still typing did not result in a worse experience.
Cross-mode ignored suggestion suppression: As discussed above, this prevents ignored ghost text from immediately resurfacing as a next edit suggestion after the cursor moves. To the system, these may be different views, but to the developer, they are the same unwanted suggestion.
This work underscored a point that applies across interactive AI systems: ultimately, a good model is just one component of a complex system—UX, client logic, networking, and server-side logic, to name a few—and all parts of that system must be designed with care and optimized intentionally to create a great experience for the user. The end-to-end experience is the product.
Folding completions into the unified model had a much higher development velocity because it built directly on the 2-in-1 foundation, showing how compounding learnings accelerate each strategic phase. A few lessons to call out:
Beyond the individual model results themselves, another broader lesson we learned is the value of reflection and learning across the end-to-end system.
From a science perspective, the 2-in-1 model took our team over 15 SFT training runs, 170 RL training runs, and several times as many checkpoint-selection runs, resulting in 14 A/B flight candidates before the final shipping candidate. In contrast, the 3-in-1 model was developed using only RL on top of the 2-in-1 model, and it took us only 35 RL training runs and 10 A/B flight candidates. Each modeling milestone has benefited from, built on, and been accelerated by our learnings in previous ones.
This iteration extended far beyond the model itself. Over the same period, the VS Code client team made 113 PRs improving and supporting the NES experience, including the changes described earlier in this post. These spanned experimentation and configurability, model integration, prompting and context construction, serving and request orchestration, caching and rebasing, rendering and interaction, telemetry and observability, correctness and robustness, and the extensive ongoing work of keeping the experience running smoothly as it shipped to millions of users each week.
This is the reward of building the product as an end-to-end system: the model and the surrounding experience evolved together, with learnings from each continually shaping the other. The acceleration we see in the end-to-end system getting better at learning and improving is just as, if not more, exciting than any individual milestone itself.
Figure 2. The learning loop between the model and the full end-to-end experience is a two-way street, and evaluation and reflection at every step benefits all components of the experience.
We flighted the 3-in-1 model compared to the then-production baseline of the 2-in-1 model and completions model. We were excited to see a 10.1% decrease in dismissals with the new unified model without statistically significant regressions in any key metrics, including accumulated retained characters. This was especially notable given that the proportion of ghost text at the cursor decreased by 13% compared to the control, adding nuance to our prior learnings that increasing the proportion of less intrusive ghost text lowered dismissal rates.
This underscores the benefit of a unified model. The model can choose the best edit for the given context, often more effectively than complex client logic that orchestrates independent models. We consider this to be a promising data point to show that the unified experience can be greater than the sum of its parts.
We are excited for you to try this unified system that treats inline editing as one continuum, from finishing the current line to carrying a change through the rest of the file.
We hope you enjoyed this two-part deep dive into training the new unified inline suggestions models (and in case you missed it, you can find part one here: Building the new GitHub Copilot Inline Suggestions Model: Part One). Please also stay tuned for a more detailed technical report!
We are also working on expanding this model's capabilities to make the code editing experience even more seamless and intuitive. This includes personalized model eagerness, which builds on our work in model quality by tailoring how proactively the model suggests edits to user preferences.
The unified Inline Suggestions experience is available now for paid GitHub Copilot users in VS Code. Update to the latest version of VS Code, then make sure next edit suggestions are enabled.
Give it a try the next time you are in the editor, whether you're working on a refactor or writing a new function. We hope you enjoy the tab-tab-tab experience, and we'd love to hear your feedback!
In the second part of this blog post, we'll dive deeper into how we moved to the 3-in-1 model for inline suggestions. Stay tuned!
Happy coding! 💙
Special thanks to Luciana Abud, Alexandru Dima, Yu Hu, Simona Liao, Gaurav Mittal, Elsie Nallipogu, and Nick Trogh for their thoughtful feedback, insights, and contributions to this blog post.
We extend our deepest gratitude to our developer community for the ongoing feedback that pushes us to deliver the best possible experiences with VS Code and GitHub Copilot. Huge thanks to the researchers, engineers, product managers, and designers across GitHub and Microsoft who curated the training data, built the training pipeline, evaluation suites, and serving stack, and to the VS Code and GitHub Copilot teams for smooth model releases.
Last week Rishi Raj Jain built an app to search Hacker News posts and comments using Postgres full-text search, hosted on Neon Lakebase. It's a good app and a good demo video except for one thing: this tilde.
Why can't Lakebase provide an exact count? For that matter, why should it take over 3.6 seconds to rank what Lakebase claims is just 3,400 documents?
I knew TIN could do a better job than that, so I forked Rishi's app and started building. We've already shown that TIN is really fast, but sometimes being fast gives you the space to build more interesting features, too. Let me show you what I built.
First thing to do is stop debouncing keystrokes. Rishi's original app, on the left, waits 250ms after each keystroke before it even begins the search. The TIN version, on the right, searches immediately after every keystroke. Postgres with TIN can comfortably handle immediately sending off the query at every change, since it is so much faster.
TIN supports wildcard searches. So in my app, I made any incomplete final word a wildcard: typing planetscale datab in the textbox actually searches for planetscale datab*, which naturally matches planetscale database. (Also: datablindnes, Databall, and DATAbEEF, but not in a conjunction query with planetscale.)
Autocomplete is nice if you don't know how to spell a word or just want to avoid typing the whole thing.
TIN supports fuzzy matching, specifically Levenshtein edit distance. In my version of the app, if the search as you've typed it returns too few results overall, the app performs a COUNT(*) search for each term individually, then allows an edit distance of two for the least popular term. If that still doesn't find enough results, it repeats the process until all terms are fuzzed, if necessary. So here, I've typed planetscale databse, and because databse isn't a real word (it's found ten times across the whole history of Hacker News), we end up searching for planetscale databse~2. The results show you which word(s) got fuzzed.
Most index implementations, including Lakebase, encourage developers to configure their index to ignore the hundred-or-so most common English words, called stop words. That's a trade-off: smaller, faster indexes, but no way to search for common words. TIN is fast enough that it can afford to just index everything. Good luck searching for "to be or not to be" or "The Who" if your index contains none of those words.
Blink and you might miss it: as I was typing, TIN searched for to b, counted 1,054,718 results, and ranked the top 30 of them in 11ms.
TIN is especially well optimized for COUNT(*) queries. For even very large numbers of results, even on complicated multi-term queries, TIN can provide an exact count in just a few milliseconds. Here's a search for show hn, the example from Rishi's original demo. Lakebase takes 3.6 seconds and estimates there are 3,400 results. Actual count: exactly 213,447.
Rishi's original app provisions a Neon instance with 32 CUs, which requires the Scale plan. It also configures the instance to never sleep, so the cost continues all month long: 32 × 730 × $0.222 ≈ $5,186.
The TIN version uses an HA cluster of three M-160 instances, specifically M-160s instances with x86-64 CPUs and 118 GB of NVMe storage each.
| Lakebase | TIN | |
|---|---|---|
| Instance size | Scale: 32 CU | M-160 |
| CPU cores | 32 | 2 |
| RAM | 128 GB | 16 GB |
| Monthly price | $5,186 | $609 |
TIN is faster and can implement many more useful features, for less than 1/8 the price.
If you like better, cheaper, faster, we have the technology. Get started with TIN search for Postgres.
For file uploads in a React app, the best setup is a service where the browser uploads files straight to storage. Your server signs a short-lived permission, and the browser uses it to send the file directly to storage. Upstash Blob does this with one server handler and one React hook, and costs $0.02 per GB stored and $0.02 per GB served.
Where the file bytes go matters more than which library draws the drop zone. When the browser sends them straight to storage, the choice comes down to how much of the upload flow you want to build yourself and what you pay per GB.
If you deploy your app to a serverless platform (e.g. Vercel), your API route can only take a small request body, and a video or a large photo goes over that limit fast. A Vercel Function accepts at most 4.5 MB in the request body. AWS Lambda stops at 6 MB.
The simplest upload form puts the file in a FormData, sends it to /api/upload, and lets the route write it to storage. It works on your laptop with a small test image. In production, a screen recording goes over the limit, and the platform rejects the request.
But even if a file is within the limits, it's quite expensive. When the file goes through your server, your function receives every byte and then sends every byte again to storage. You pay for that time, and a 2 GB video can't go through a function at all.
A better way is for the browser to send the file straight to storage. Your server still decides who can upload what, but its only job is to sign a short-lived permission:
In a direct upload, your server checks the request and signs a short-lived URL, then the browser uses that URL to send the file straight to storage. When the upload finishes, your server gets a callback so it can save the file in your database.
With Upstash Blob, you write one handler on the server and one hook in React. Here's a Next.js App Router setup based on the quickstart. It uses one secret, UPSTASH_BLOB_TOKEN, which stays on the server.
In the handler, you set which files the route accepts and where each file goes:
// lib/uploads.ts
import "server-only";
import { uniquePath, uploadHandler } from "@upstash/blob";
export const uploads = uploadHandler({
constraints: { maxSize: "20mb", contentTypes: ["image/*", "application/pdf"] },
onBeforeUpload: ({ file }) => ({ path: uniquePath`uploads/${file.name}` }),
onUploadComplete: ({ path, url }) => console.log({ path, url }),
});
onBeforeUpload runs before anything is signed, so this is where you check the session and block users who shouldn't upload. onUploadComplete runs once the file is in storage. It can run more than once if the browser retries, so an upsert is the safe way to write to your database.
The route file mounts the handler:
// app/api/upload/route.ts
import { uploads } from "@/lib/uploads";
export const { GET, POST } = uploads;
The hooks take the handler's type, so the client knows what the server allows:
// lib/upload-hooks.ts
"use client";
import { uploadHooks } from "@upstash/blob/react";
import type { uploads } from "./uploads";
export const { useUpload } = uploadHooks<typeof uploads>();
And the component picks a file, starts the upload, and shows progress:
// app/upload-form.tsx
"use client";
import { useUpload } from "@/lib/upload-hooks";
export default function UploadForm() {
const { start, upload, accept } = useUpload();
return (
<>
<input type="file" accept={accept}
onChange={(e) => start({ file: e.target.files?.[0] })} />
{upload && <>
<progress aria-label="Upload progress" value={upload.percent} max={100} />
<p>{upload.percent}%, {upload.status}</p>
</>}
{upload?.status === "done" && (
upload.blob.url
? <a href={upload.blob.url}>{upload.blob.url}</a>
: <p>Uploaded: {upload.blob.path} (private; no public URL)</p>
)}
{upload?.status === "error" && <p>{upload.error.message}</p>}
</>
);
}
Large files work with the same code. Past 16 MB, the SDK cuts the file into parts and uploads four at a time. A failed part retries automatically, and if the signature expires mid-upload, the SDK gets a new one. If the user closes the tab and picks the same file again, the upload continues where it stopped. One object can be up to 5 TB.
A large file takes this path:
A React upload UI needs a progress bar, a drop zone, file type and size checks, and a way to recover when a large upload fails halfway. Some upload services include these, while with raw object storage like S3 or R2 you build all four yourself.
| Piece | What it needs | Upstash Blob | Raw S3 or R2 |
|---|---|---|---|
| Progress | Bytes sent, a status, pause and cancel | useUpload returns percent, status, and pause, resume, cancel, retry | You track upload progress yourself |
| Drag and drop | A drop target that hands over a File | Pair the hook with react-dropzone | Same, react-dropzone |
| Type and size checks | Checks in the browser and again on the server | Declared once on the server, the picker follows | You write both sides |
| Large files | Parts, retries, resume | Automatic past 16 MB | You build multipart or use a library like Uppy |
For drag and drop, react-dropzone works with any backend. It gives you the dropped file, and you pass it to the same start function from the upload hook:
// app/drop-zone.tsx
"use client";
import { useDropzone } from "react-dropzone";
import { useUpload } from "@/lib/upload-hooks";
export default function DropZone() {
const { start, upload } = useUpload();
const { getRootProps, getInputProps, isDragActive } = useDropzone({
onDrop: ([file]) => start({ file }),
});
return (
<div {...getRootProps()}>
<input {...getInputProps()} />
<p>{isDragActive ? "Drop it" : "Drop a file here"}</p>
{upload && <progress value={upload.percent} max={100} />}
</div>
);
}
File checks have to run in two places. The browser check gives the user a fast error, but anyone can skip it by editing the page. With Upstash Blob, the route's size and type limits are sent to the client as JSON, so the file picker only offers allowed types and rejects a file that's too big before any request goes out. The server checks again before it signs anything. The browser also sends the file's first bytes, and the server refuses the file when those bytes clearly don't match the declared type.
Even with those checks, every uploaded file is still untrusted. The byte check catches honest mistakes, but a client can send a clean sample and then upload something else. Nothing in the flow scans for malware.
For most React and Next.js apps, a managed upload service with a React SDK fits best. Upstash Blob gives you typed hooks, automatic multipart and a global CDN. Raw storage like Cloudflare R2 costs less when you serve a lot of data, but you build the upload flow yourself.
Here's a small overview:
| Service | Type | What you get |
|---|---|---|
| Upstash Blob | Managed | S3-compatible storage, typed React hooks, automatic multipart, files up to 5 TB, a CDN for public buckets |
| UploadThing | Managed | Storage, a server file router, and ready-made upload button and dropzone components |
| Vercel Blob | Managed | Storage with client uploads, files up to 5 TB, caching only for files up to 512 MB |
| Cloudflare R2 | Raw storage | S3 API storage with free egress |
| AWS S3 | Raw storage | S3 storage, with CloudFront added separately as the CDN |
| Bunny Storage | Raw storage | Storage with Bunny's CDN billed on top |
| Cloudinary | Media platform | Storage plus image and video transformations, paid in credits |
File upload costs depend on what you pay per GB stored each month and what you pay per GB your users download (egress). Upstash Blob charges $0.02 for each, Vercel Blob charges $0.023 and $0.05 respectively, and Cloudflare R2 charges $0.015 with free egress.
| Service | Storage per GB-month | Egress per GB | Free tier |
|---|---|---|---|
| Upstash Blob | $0.02 | $0.02, CDN included | 1 GB storage, 10 GB bandwidth, no credit card |
| Vercel Blob | $0.023 | $0.05 | 1 GB storage, 10 GB transfer on Hobby |
| Cloudflare R2 | $0.015 | Free | Not covered here |
| AWS S3 + CloudFront | $0.023 in us-east-1 | $0.085 for the first 10 TB, US and EU | Not covered here |
| UploadThing | Flat plans: 100 GB for $10, 250 GB for $25, then $0.08 per GB | No per-GB charge on the pricing page | 2 GB |
| Cloudinary | 1 credit per GB | 1 credit per GB | 25 credits a month |
Upstash Blob also charges per request: $0.30 per million simple operations and $4.50 per million advanced ones. Uploads count as advanced operations but use free bandwidth, and deletes are free. Upstash bills storage on the average bucket size over the month.
Here is the Upstash Blob pricing page:
Take an app that stores 100 GB and serves 1 TB (1,000 GB) of downloads in a month. On the four services that bill per GB, that app costs:
Upstash Blob: 100 × $0.02 + 1,000 × $0.02 = $22.00
Cloudflare R2: 100 × $0.015 + 1,000 × $0 = $1.50
Vercel Blob: 100 × $0.023 + 1,000 × $0.05 = $52.30
AWS S3 + CloudFront: 100 × $0.023 + 1,000 × $0.085 = $87.30
These figures only cover storage and egress. Request charges are extra on all four, and Vercel can also bill edge requests on cache misses.
R2 is by far the cheapest because egress is free. In exchange, you write presigned URLs, progress, multipart and retries yourself. Upstash Blob costs less than half of Vercel Blob and about a quarter of S3 with CloudFront, and it includes the upload flow.
UploadThing's $10 plan covers exactly 100 GB of storage, so it's cheap for this app. Past 250 GB it charges $0.08 per GB stored, four times Upstash Blob's rate:
Cloudinary is in a different price range. At one credit per GB, this app uses 1,100 credits a month (100 for storage plus 1,000 for bandwidth), compared to 25 on the free plan. It's worth paying for if you need its image and video transformations.
Software is eating the world, and agents are eating software engineering. It is imperative that software engineers develop an understanding not just of the agent software that is now their most important tool but also of how the intelligent core of agent software works, through inference by large generative models of language — if not out of the engineer’s need to understand and control their tools, then at least because inference is poised to consume more computing power and produce more benefit than all other uses of computers.
The central fact about inference services for coding agents is that they must operate at extremely high relative and absolute performance.
By relative performance, we mean large fractions of the peak rate or “speed of light” of the hardware that it uses. By absolute performance, we mean that the scale of that peak rate and the amount of work done per request is large. Contemporary matrix math accelerators like Tensor Cores operate at the petaFLOP per second scale. Large generative sequence models with sufficient intelligence to automate software development have trillions of floating point parameters, and each of them must be accessed many times per second, even when serving just a single request.
Due to these requirements, economically viable coding agent inference services are currently only feasible by operating at a scale sufficient to amortize hardware and engineering costs — roughly, at the scale of trillions of input and output tokens.
We’ve done this, and we’d like to share how.
At Modal, we operate a number of such inference services for coding agents at this scale and work with a number of customers who do the same. You can use our services indirectly via inference routing platforms like OpenRouter or Vercel AI Gateway or directly through our Shared Endpoints.
In this blog post, we will walk through how we optimized inference performance when serving inferences from Moonshot AI’s Kimi K2.6 model to power coding agents. Though this model is “old” by this field’s standards (literally hundreds of days old!), the fundamentals of sequence modeling, hardware, and scaling change slowly enough that the core story and many of the details match what we have done for more recent models that have superseded K2.6 in intelligence and cost-performance, like Kimi K3.
Our optimizations allowed us to scale per-replica performance of inference replicas by 2.8x per user and 5.6x across users on the replica:
This chart relates the individual user’s experience (decode tokens per second per user, aka interactivity) on the x-axis with the cost-performance of the overall system on the y-axis (total tokens per minute per GPU, aka token throughput), with the number of concurrent users indicated at each point.
More intuitively, that’s the difference between a ruinously expensive service with the UX on the right below and a price-competitive service with the UX on the left:
We then scaled those single-container replicas into deployments and services. One particular service processed hundreds of billions of tokens a day and trillions in aggregate:
Below, we aim to make this performance engineering legible to a general software engineering audience. By sharing how we, and our customers, are able to operate these services, we hope it enables you to do the same — perhaps by deploying a Dedicated Endpoint on Modal.
We break this down into two sections: understanding the sequence model that infers the response to each request and understanding workload structure across requests.
Contemporary coding agents are powered by probabilistic generative models of unicode sequences pre-trained mainly via unsupervised masked sequence prediction and post-trained mainly by reinforcement of output software correctness. Like the parser of a compiler, they operate not on raw strings but on tokenized sequences, so we call their inputs and outputs tokens. Because we are, in the end, guessing what output tokens should be, this is called inference. If you prefer deduction, stick to databases and operating systems.
The underlying sequence models these days are hybrid-attention, mixture-of-experts Transformer neural networks. These networks apply computations both per token in the sequence and across tokens in the sequence.
Attention has evolved into a generic term for cross-token computation. Mixture-of-experts refers to the dynamically routed block-sparse matrix multiplication that applies the majority of the per-token computation. These computations iteratively update the network’s internal, or latent, representation.
For insight into what’s going on inside these sequence models, see Anthropic’s “A Mathematical Framework for Transformer Circuits” (2021, but still undefeated).
A single forward pass through such a neural network produces both substantial internal state and a probability distribution over the next token(s) in the sequence for each sequence position. Because we predict (”regress”) based on our own outputs (”auto”), this is autoregressive sequence modeling.
To respond to a client request, we generally chain multiple forward passes together like this:
Forward passes are expensive, so we want to amortize this work as much as possible. Much of the work in per-token computation amortizes by batching several sequences together. Much of the work in cross-token computation amortizes by caching the internal state. For historical reasons, this is called the key-value cache (KV cache or just KV), even though contemporary models like Kimi don’t have distinct keys and values. You can read more about the “napkin math” here in Kipply’s excellent “Transformer Inference Arithmetic” blogpost (2022, but still undefeated).
When a forward pass processes a request’s input tokens, we call it a prefill, because it is “prefilling” the KV cache. When a forward pass produces a response’s output tokens, we call it a decode, because we are “decoding” the model’s “encoding” of past state into predicted future. What about forward passes that do both? Yeah, we don’t like the terminology either.
Prefill performance is mostly tracked by the latency to complete all prefills for a request, aka time-to-first-token (TTFT). Decode performance is mostly measured by the rate at which output tokens are produced after that, aka output tokens per second (TPS). Both can be measured client-side or server-side, causing no end of confusion.
The particular sequence model covered in this post is Kimi K2.6 by Moonshot AI. This model parametrizes its matrix multiplications with approximately one trillion numbers (weights in its matrices), the majority of which are stored as four bit integers (INT4).
We serve the model, however, with four bit floating point numbers (FP4). Four bits only gives you sixteen distinct values, so you further need a micro-scaling format to scale individual blocks within tensors independently. We chose the NVFP4 micro-scaling format, which has native hardware support at the petaFLOP/s scale in the Tensor Cores of Blackwell Streaming Multiprocessor Architecture GPUs like the B200 and B300. Because we operate a dynamic GPU fleet in a time of constrained compute supply, we prepare our deployment to run on both B200 and B300 GPUs. Results below are all for B200 GPUs; B300s are substantively similar but operate at higher request concurrency because they have more high-bandwidth memory (HBM) available for caching.
We chose the SGLang inference engine as our base. We found several opportunities to improve performance by patching the engine. As contributors to the SGLang project, we upstreamed these patches, described and linked in the post below.
When you serve such models on coding agent traffic naïvely, you get bad results.
This chart indicates that throughput and interactivity rapidly collapse above 6 concurrent users. Furthermore, even before that peak, the interactivity is below user expectations and the system is below acceptable efficiency.
So from here, you need to increase interactivity and throughput to deliver better outcomes to users while decreasing your own costs. To do that, you need to understand the sequences in this workload deeper than just “tokens in and tokens out”.
Individual requests for output tokens are created in “sessions”: the user, the generative model, and the tool calls chain together iteratively to construct a tower of input sequences, accumulating context — and value — over time. The iterative process of meaning construction, information discovery, and sense-making strikes us as fundamental to the nature of sequence modeling and sequential action, so we expect this pattern to far outlast “coding agents”.
Concretely, a single session looks something like this:
That is, the input sequence (green) for each turn T is the entire session history up to T (darker green), plus something new (lighter green). This has two key consequences.
First, it means requests inherently have long input sequences relative to their output sequences (pink, above) — there are T-1 past output sequences in the input to turn T, and T is in the dozens. For the core workload we used in optimization and served in production, this ratio was 200:1; requests contain roughly 100k input tokens and produce roughly 500 output tokens. That means the majority of processed tokens will be input tokens (just check the token usage numbers in your coding agent software).
Second, it means the input sequences have high overlap with previously processed input sequences — the ones from turns 1 to T-1. That means that on the way to serving turn T, the tokens in turn 1 are processed T times. This makes caching absolutely critical — we can avoid linearly-scaling recomputation to save effort, but we introduce linearly-scaling state that must be managed and has its own performance characteristics. Navigating this tradeoff is the core engineering problem we’ll tackle in this post.
With this picture of the workload in mind, we turn to optimization.
To optimize performance, build a working system, identify the bottleneck, then lift it. Repeat as needed until you’ve won.
Though our ultimate goal was to optimize an entire service, we decomposed that problem into two simpler problems: optimize a single replica first, then scale from one to many replicas.
We further split the problem of single replica performance into two sub-problems: first maximize interactivity, then maximize throughput without losing interactivity.
Interactivity primarily impacts request latency. Request latency and throughput interact through concurrency, the number of in-flight requests, by a rearrangement of Little’s Law:
Our key bottlenecks for latency, concurrency, and throughput started in the GPU HBM.
Our key bottleneck on latency was HBM bandwidth during decode. We lifted it by parallelizing matrix multiplication across GPUs (tensor parallelism, TP) and by applying custom DFlash speculative decoding — doing more computation per memory load, even when that computation may not be needed.
That created a bottleneck on concurrency through HBM capacity: how much work can we keep in a cache that loads faster than we could just recompute results. We lifted it by clearing up intermediates in HBM, quantizing intermediates to lower floating point precision, and extending the cache hierarchy to CPU RAM with HiCache. We used the cache hit rate (CHR) as a targeted metric of improvements to caching. CHRs between one and two 9s are very much feasible for most coding agent workloads.
Increasing interactivity increases the system performance as observed by individual users. We chose to work on this first. We made that choice for several reasons.
First and simplest, we found that coding agent users enjoy and will pay more for tokens that come to them faster, so high interactivity was key to building the service that our and our customers’ users wanted.
Second, interactivity is particularly amenable to improvement by speculative decoding. Because it is a simple, learning-based technique, its performance benefits scale with compute and data: machine learning’s famous “bitter lesson”, returning in performance engineering for ML systems. And we know how to scale training.
This choice to interactivity-maxx had two additional benefits, one operational and the other for throughput, which were especially salient because we operate a dynamic, autoscaling fleet of thousands of GPUs.
Maximum interactivity replicas are smaller and therefore easier to serve.
Using multiple processors together requires an interconnection network (interconnect) for communication. The lowest latency, highest bandwidth interconnect for Nvidia GPUs is NVLink. NVLink operates across a group of processors in a “domain” of some size.
A single host operating system can support an NVLink domain of up to 8 GPUs. The largest NVLink domains that are generally available comprise 72 accelerators (in a multi-node IMEX domain). Using more accelerators would require a slower interconnect (IB/RoCE or, worse, standard Ethernet). That means that for maximum interactivity we should not expect to use more than 72 accelerators per replica — the communication overhead will almost surely dominate any per-request latency wins.
But that doesn’t mean we must use 72 accelerators.
Consider these results from SemiAnalysis’s InferenceX benchmarks for this same NVFP4 Kimi K2.6 model, which show throughput per GPU as a function of interactivity, annotated with GPU count, for a variety of deployments:
The highest interactivity is achieved by a deployment with just eight GPUs per replica. Furthermore, that interactivity is achieved with comparable throughput per GPU, which means that by choosing a smaller domain, we are not obviously forgoing peak throughput cost-performance (subject to our interactivity constraint).
To keep the chart legible, we selected only a small subset of deployments most similar to ours, but the pattern holds across more accelerator types and across more models in the InferenceX benchmarks (explore them here). Generally, you can achieve the highest interactivity at comparable per-GPU throughput with only four or eight GPUs. You can then achieve the same aggregate throughput by scaling smaller replicas. The core Modal serverless platform makes this scaling performant and reliable.
This is a huge operational win. Smaller, simpler units make for easier scaling. Eight GPUs can be driven by a single host OS kernel. An NVL72 domain, on the other hand, is comprised of nine such subsystems sharing an address space (yes, you should be shuddering). Availability is constrained and contracts are long and inflexible.
Eight-GPU Blackwell systems, on the other hand, are standard enough to be available via on-demand and spot markets, which makes it much more cost-effective to handle variable load. Replicas with one, two, or four GPUs can furthermore be packed inside of a single physical eight-GPU machine — which already has all the resources required to start another replica (model weights, JIT artifacts).
Of course, as and if the compute supply and user demands change, we will happily revisit this choice.
Increasing interactivity indirectly increases throughput.
By reducing the latency of individual requests, we indirectly improve throughput by freeing up resources for new requests.
Agentic coding workloads are approximately “closed-loop” per session. Sessions are almost always chains — of user-written tokens, of tool call responses, and of model outputs. The next request in the session, therefore, almost always arrives some time after the previous response has finished generating. The session’s next request is therefore latent for some time, outside the inference system — for tool calls, 10s of ms to seconds with a tail of minutes; for user responses, seconds to minutes, with a tail of hours or more.
During that time, other requests can be processed on the same node. When you have sufficient load for the active capacity, there are always requests ready for a node to process. When you have sufficient capacity for the active load, there are always nodes to map requests onto. Both of these are guaranteed by our fast autoscaling system. We’ll talk more about request routing in the section on scaling to multiple replicas.
Interactivity measures output tokens per second per user. Naïvely, autoregressive sequence models like Transformers produce these tokens sequentially. Amdahl’s heartbreaking Law strikes again.
Each time a token is produced, gigabytes or more of model weights and KV cache must be loaded from GPU HBM to Streaming Multiprocessor L1 caches, which generally takes longer than actually computing the KV state and output for a single next token. This creates a bottleneck on that memory bandwidth. Parallelism helps create more bandwidth, but this is more useful for per-token calculations than for cross-token calculations, which arise as a bottleneck for long sequences, as observed in coding agent workloads.
So we instead elevated that bottleneck by applying speculative decoding.
Fundamentally, speculative decoding makes the same trade that speculative execution in processors makes: when you have spare operational bandwidth due to serial dependencies between operations, you can use that bandwidth to run operations that may not end up being used. Effective operational throughput increases if you can guess operations that will be used with high probability, and the name of the game is increasing that probability with the least work possible.
For autoregressive sequence model inference, the “trick” to run more operations per iteration is to guess what the next several tokens will be using another, faster language model (the “speculator” or “draft”), and then validate the guesses in parallel with the served model (the “target” or “verifier”).
As with speculative execution, this acceleration happens without changing program behavior, i.e. the probability distribution of the target sequence model.
As we describe in our blog post releasing speculators for the Qwen 3.5 and 3.6 model family, the gains from speculative decoding are large — integral factors, not a few tens of percent.
Counterintuitively, it is fairly easy to produce a speculator that predicts four, eight, or even more of the next tokens in the output, on average, especially for coding agent workloads. Roughly, there are two reasons this is the case: the target model sets speculators up for success and the majority of tokens do not use the full intelligence of the target model.
First, the target language model has already produced extremely useful representations of the sequence during its forward passes — starting from the static embedding of each token, each layer of the model progressively enriches this representation, up until the final “language modeling head” layer turns that representation into a distribution over next tokens. Even better, these representations are already stored in KV cache. State-of-the-art speculator architectures like DFlash (and derivatives like DSpark) re-use this state as their inputs, so they can be orders of magnitude smaller (and faster) than the target: standing on the shoulders of giants, pointing to where they might go next.
Consider the following sample coding agent output:
Anyone who has used recent models can give you a good guess for what comes after You’re absolutely (it's never wrong). And the quotation is from previous user input, so once the quote opens, the next tokens become highly predictable.
Looking a layer deeper, consider what this sequence looks like once it has been formatted with the special control tokens in the model’s “chat template”:
This sequence has substantial structure that does not require high intelligence to produce. Of course, the details within that structure still matter for correctness, so the target model’s capabilities are still important!
Most of the capacity of the target model, then, is likely going to enrichment of the representations of these tokens for use in predicting tokens many steps ahead. If you already know what the next several tokens are, you can compute their representations in parallel.
This is not a quirk or a hack: providing dual parallel and sequential forward passes is a fundamental feature of modern sequence models relative to traditional recurrent neural networks. It is present in both “classic” Transformers and linear/hybrid attention models, so we can expect it to persist.
For this and other reasons, we have invested heavily in speculative decoding, and we suggest you do the same.
The fastest speculators are trained not just to predict the general behavior of the target model but to predict its behavior on specific datasets. Because they are small, their modeling capacity is limited, and you want to use that capacity only for what will actually occur in production. For the ML ‘heads: the loss for a speculator is Kullback-Leibler divergence from the target model, which encourages mode-seeking, rather than mode-covering.
But as with neural networks in general, our experiments have indicated that it’s better to start from a strong foundation and then adapt the speculator to the specific task — aka fine-tuning. So we first trained a DFlash speculator for Kimi K2.6 on a generic data mixture and then fine-tuned it on coding traces that were output by the target model. The draft model can then be continually trained on the target model’s outputs when serving production traffic.
We ran into one issue when operating on live traffic: mapping tokens to a string and then re-tokenizing is not an identity map, because tokenization is fundamentally a cursed hack. But typical logging, e.g. of HTTP requests, operates on strings, not tokens. We therefore patched SGLang to emit raw token ids through sglext and contributed the work upstream.
Fine-tuning gave us an increase in accept length from 5.00 to 5.84 tokens per step on representative traces, for an incremental speedup of 20%.
Adding more engineers to a slow task makes it take longer, but computers have no such weakness — if you parallelize work and shard data correctly.
The primary parallelism strategies for sequence model inference split work:
Of these choices, only context parallelism, expert parallelism, and tensor parallelism split work within a single request and so directly improve interactivity. Tensor parallelism (TP) is the lowest level of parallelization — besides the parallelism within kernel execution, which is legion but out of scope (we’ve shared some of our work on that elsewhere). That means TP optimizations compose better with other strategies and therefore make a good first target.
In more detail: tensor parallelism takes an input to a matrix multiplication and splits the output processing work across parallel workers, which can therefore shard the matrix data needed for that processing, aka the model weights. For more, see the Megatron paper (2019, but still undefeated).
Despite this first-principles argument, we still investigated multiple other parallelism strategies, because 1) interactivity can be indirectly affected by optimizations elsewhere and 2) you never know what you don’t know. However, we found that Tensor Parallelism Is All You Need™ to interactivity-maxx. For instance, we found that data-parallel attention allowed us to achieve higher concurrency by sharding KV cache, but latency was worse. In fact, it was so much worse that it caused overall throughput per GPU to drop, even though concurrency increased.
Along with choosing a parallelism strategy, you also need to choose the number of parallel workers. For the Kimi K2.6 model running on B200 GPUs on sequences that may have hundreds of thousands of tokens, the feasible configurations are with four GPUs (TP4) and with eight (TP8).
Some quick napkin math there: a B200 has 180 GB of HBM, and Kimi K2.6 has 595 GB of weights (over a trillion, one nybble per weight). Spilling weights to CPU RAM or disk would wreck latency, so TP1 and TP2 are both infeasible. With four or eight GPUs to shard weights over, we have about 125 or 845 GB for KV. Each KV entry has 576 elements, stored in two-byte BF16 format, and there are 61 layers, each with their own KV entry per token, and so a single token consumes ~72 KB = 576×2×61 bytes. That gives you space for about half a million tokens of KV in TP4, or about three million in TP8 — six times the cache capacity with twice the hardware.
| Config | Total HBM | HBM minus weights | Approx. KV size per GPU | Approx. KV capacity |
|---|---|---|---|---|
| TP1 | 180 GB | -415 GB | - | - |
| TP2 | 360 GB | -235 GB | - | - |
| TP4 | 720 GB | 125 GB | 31.3 GB | 0.45 Mtokens |
| TP8 | 1.44 TB | 845 GB | 105.6 GB | 3.0 Mtokens |
This makes TP8 look pretty appealing. However, we found that on the target workload and at concurrencies compatible with our interactivity goal, TP8 running only prefill achieved roughly the same throughput per GPU as TP4 running both prefill and decode — an unfair comparison in TP8’s favor, which it failed.
However, choosing TP4 left us extremely constrained on KV cache capacity.
So from here, we turned to strategies to alleviate this constraint.
At ~100k max input tokens per request and running TP4, only around 4 users’ conversations could be scheduled onto a single replica without tanking interactivity. Past that, cache hit rate (CHR) plummeted and interactivity/throughput collapsed as long inputs were recomputed. Recomputation is far, far slower than loading their KV entries from HBM. So we went about creating more space for KV.
The most direct optimization was to find wasted HBM and give it back to the KV cache.
We took a look at the implementation of the DFlash draft model architecture in SGLang and noticed that it incurred twice the necessary HBM usage.
Specifically, the target model intermediates used as input to the draft model were first collected as a list of pointers and then copied into contiguous memory at the end of the forward pass. We rewrote it to instead pre-allocate that contiguous memory as a buffer and push intermediates to it during the forward pass, cutting the peak load on HBM in half. And we did it without changing the append-based logic, thanks to a bit of Python magic. We upstreamed our changes to SGLang in this PR.
The second easiest way to free up space is to use lower-precision floating point numbers everywhere.
But unlike speculative decoding or reducing waste, lowering precision is not a free lunch. Model outputs can change dramatically, and usually not in a way that is good for application outcomes. You can get an intuition for the impact of block quantization techniques with the visualizer in our LLM Engineer’s Almanac (sample below; block-quantized on the left, original on the right).
Being able to confidently make changes that are in principle lossy but which don’t impact outcomes for the target application is critical — and a differentiating capability for custom, self-hosted inference applications versus generic, multi-tenant model API providers.
As usual, speculative decoding is the easier case, and so quantizing the draft model is an easy win. The target outcome for the model, decode speed, degrades smoothly, unlike intelligence, and drafter correctness doesn’t impact application outcomes outside of performance. We upstreamed FP8 support for the DFlash speculator architecture to SGLang in this PR.
But there are inevitably appealing optimizations that do impact application outcomes, which is why we’re investing heavily in building our capacity to evaluate the modeling capabilities of inference servers (more on that soon!). We also massively appreciate and support initiatives like Moonshot’s Kimi Vendor Verifier that help consumers of models consistently assess quality. Evals, evals, evals!
Based on our evals, we found that we could quantize the model’s KV cache from BF16 to FP8. This doubles the cache capacity, counted in tokens. Furthermore, most of the expert matmuls were already in NVFP4, the most compact format with native hardware support (for now!). But not the critical “shared” experts that are activated on every token. We found that we could quantize the shared experts from FP8 to NVFP4, freeing up additional HBM for cache. Neither of these changes meaningfully degraded model quality (relative to run-to-run non-determinism) in our evaluations.
Finally, what if the cache was bigger, even if that meant it was slower?
So far, we’ve only considered GPU HBM for storing KV. That means our two options when handling input sequences are either “keep it in ultra-fast, ultra-expensive storage on the GPU” or “chuck it in the bin”. This leads to a very sharp degradation in replica performance when the KV cache size we need to service the workload exceeds what fits in HBM due to a reduction in cache hit rate (CHR):
That’s why all good caches are multilayer! Each layer of the cache adds another, gentler step down in CHR with load. SGLang’s HiCache expands KV cache capacity by adding “L2” and “L3” cache tiers, allowing KV to be stored in host memory (L2) and distributed storage (L3).
Higher cache tiers are still slower (or else we’d just use them as the lower tier!), so they can easily harm latency and potentially hurt throughput. We got a lot from using just the CPU RAM-based L2 cache. Even then, we essentially only use it to handle excess load.
That is, without HiCache, rapid degradation in performance with concurrency above the level that supported peak performance prevented us from trying to serve at that peak. With it, replica behavior was smoother when an individual replica’s load transiently exceeded the peak.
Replicas don’t always have exactly the target request load because of nondeterminism in upstream user/agent behavior and because of routing of requests across replicas, which we consider next.
After optimizing single-replica performance, we scaled up to a larger deployment — after all this effort, we want to serve significantly more than six concurrent users! At a high level, we do this by serving an autoscaling pool of inference engine replicas behind a Modal Server.
Because our core platform’s autoscaling infrastructure handles all of the typical problems that bedevil autoscaling and entangle it in spaghetti Kubernetes YAML — deciding when to scale, acquiring resources, spinning up a host environment, setting up replicas quickly, recovering from faults, providing observability, releasing resources — essentially the entirety of our work was in the routing layer.
Routing is “easy” except when replicas have state, and the KV cache adds state to the replicas. Luckily, it’s the good kind of state, an ephemeral cache: it’s not necessary for application correctness and can be readily recomputed on a miss. But recomputing incurs a performance penalty, so routing becomes an important part of performance optimization.
We observed two performance problems that caused us to look closer at our routing:
The underlying cause of both of these issues was “regrettably cold prefills” — input sequences that overlapped with sequences we’d seen before, but for which we ended up recomputing the entire KV. The underlying cause of that was inefficient request placement by our original stateless routing algorithm, driven by both concurrency within sessions and unlucky hashing.
Based on this work, we’ve updated our routing layer to support stateful and KV cache-aware routing algorithms. Modal Servers can use it via the kv_aware_routing experimental option:
By default, Modal Servers use uniform random routing for all requests. To make certain requests “stick” to a particular container, clients can provide a header, Modal-Session-Id. This is then hashed and mapped onto a replica, something like this:
Though not exactly the circular hashing in the diagram — we use consistent hashing to get better behavior when the replica count or identity changes. See this code sample for details.
Coding agent clients of inference services on Modal can therefore create and re-use session IDs within the same agent session to map requests onto replicas that have already seen their previous inputs and so may have their KV representations in cache, resulting in better performance — usually.
A stateless, uniform-random session routing algorithm works reasonably well for achieving balance when the number of concurrent users is large and when tolerance for variability in concurrent session count is high, but it has some issues with tight tolerance on low concurrencies — the exact regime that high interactivity coding agent inference operates in.
Here’s the math, in sketch. The distribution of session counts for each server using any uniform random hashing algorithm is binomial, with N equal to total session count and p equal to one over server count. The binomial distribution converges quickly to a Poisson distribution with rate parameter equal to Np, aka number of sessions divided by number of servers. Focusing on steady state dynamics, we can treat this as fixed and equal to the target concurrency, thanks to autoscaling. That’s good! But the Poisson distribution has variance equal to this fixed rate parameter, which means the spread in session count per replica does not decrease with increasing scale. It stays fixed, and that gives you predictable tail behavior.
Concretely: if you are targeting five sessions per replica and you have fifty replicas serving 250 sessions, a uniform random routing algorithm will produce a replica serving ≤1 sessions with probability ~4%, which shows up as reduced aggregate efficiency. It will furthermore produce a replica serving at least 12 sessions with probability ~0.5%, which shows up as tail latencies. These rates are independent of scale.
So these routing algorithms can only work in cases where this level of dispersion in load is tolerable — which is not the case for coding agent workloads.
And the situation in practice is in fact worse than the modeling predicts. Deviations from the model (and there are always deviations!) cause extra variance in concurrency counts. We observed this directly. Variance was often several times the mean, with a very heavy right tail that led to occasional very high latencies. See the load-per-replica observations below (from a deployment with its target set to five concurrent requests, for increased interactivity).
This was the root cause of our observed tail TTFT latencies and throughput shortfall.
The final router system achieved strongly sub-Poisson dispersion of load (variance under half the mean). A sample load distribution is shown below, again for a deployment targeting five concurrent requests.
To get there, we investigated the causes of tail latencies and over-dispersion of load and added new routing algorithms to address each of them.
The single biggest cause of over-dispersion and tail latencies was violation of our model of ID’d sessions as “closed-loop”, aka one request at a time per session.
In our system, clients control session IDs, so there’s no way to prevent clients from submitting multiple concurrent requests with the same session ID. And if session IDs are always mapped onto the same container, then the number of concurrent requests per container is no longer bounded. One replica gets “hot”, with very high load, even though overall load is not increased.
Typical coding agent sessions, even with sub-agents, don’t need to share the session ID across concurrent requests, because the typical session proceeds one turn at a time. But there are cases where multiple concurrent input sequences share a prefix. This happens when coding agent sessions are tree-structured, rather than chain-structured — like when you use /btw.
Here’s a point-in-time sample of request count by session ID across a number of replicas, with the session with the largest request count in red. The largest session on replica 6 has a number of concurrent requests several times in excess of the target load. Not good!
When there are such “thicc sessions”, trying to preserve perfect locality results in worse perf than duplicating some cache and spreading concurrent work across replicas. To fix this, our router intentionally breaks up highly concurrent sessions into multiple containers. That is, before sending a session to one of its assigned replicas, we check a load threshold for that session. We send this request to a different replica when the threshold is exceeded.
As described above, uniform-random algorithms are subject to a fixed rate of “unlucky” containers/users.
Even worse, though, the model above assumes that request processing time is fixed as a function of load. But more work means it takes more time to process the work in-flight, and so requests on loaded servers take longer, their load is elevated for longer — thicker tails than in the modeling.
And on top of that, it assumes sessions require equal amounts of work. But some coding agent sessions are long, and the requests in those sessions have many hundreds of thousands of tokens, while others are short and only have a few thousand tokens.
To avoid this, we assign new sessions to replicas according to finer-grained load-based signals, such as running requests (not just assigned sessions!) and KV utilization. We still maintain session affinity after session placement to preserve CHR.
During increases in load, we need to increase the number of replicas. The existing replicas have warm cache for the sessions already in-flight, so we’d prefer to keep routing those sessions there.
But with rendezvous hashing, changing the set of containers causes a re-balancing of in-flight sessions, not just new sessions. It’s not a total free-for-all — the point of using consistent hashing-style algorithms is to reroute only the ~1/N of the sessions you need to achieve balance when you add a new target. But even this requires lots of KV recomputation and slowdowns for certain sessions.
This ends up becoming another form of load-aware routing. Especially during load increases, new sessions are generally being created regularly, and mapping more of these onto replicas with less load leads to them preferentially landing on newer replicas: those replicas haven’t accumulated any load yet! There often still needs to be some balancing when the rate of new sessions and new replicas doesn’t match. We again solve this by load-awareness, this time in session reassignment, not just session/request assignment.
With all these changes to the routing in place, the behavior of our multi-replica deployment was much closer to what we expected from extrapolating single-replica results. TTFT was substantially more stable, and per-replica throughput stayed much closer to the single-replica performance, even as the deployment grew.
Taken together, these optimizations allowed us to operate multiple Kimi K2.6 inference services at the scale of hundreds of billions of tokens per day and at an interactivity and cost-performance substantively in excess of both our baseline and other offerings.
We have since repeated this basic motion — much faster, because we are much wiser and because we built reusable tools and infra! — for a number of additional models. That includes Moonshot’s updated Kimi K3 model. We repeatedly served the plurality of Kimi-K3 tokens on the competitive OpenRouter marketplace, which routes demand to providers based on the quality of their supplied inference service.
Throughout this work, we spent almost as much time on understanding our benchmarks and workload as we did on optimizing the service itself. Benchmarking is hard!
Nearly every metric we cared about was a function of both the system and the workload we fed it. Changing the data could change the results we observed without any changes to the system.
The simple answer to that is to always benchmark on the same data, and for that data to exactly match the workload from production. But production data is sensitive, which limits access. And production data has variability, both across requests and across time, and to do proper performance engineering, it is critical to understand how the system behaves in specific scenarios, not just in aggregate.
So we also want to sometimes run “controlled experiments” outside of the behavioral regime exercised regularly by production to 1) theory-build and 2) clearly isolate and measure the impact of performance interventions. As one example, already mentioned, we ran TP8 in prefill-only mode to clearly demonstrate it was inferior to TP4 in our setting. Consider this analogous to how scientists study systems not merely by observation of natural behavior, but also by intervention in controlled laboratory settings.
A few cases where data-dependence showed up:
This work began with optimizing coding agent workloads for a specific model for one customer. But we’ve also worked on a variety of inference workloads, like high-latency/throughput-sensitive analytical processing (more on that soon). We continue to partner closely with some of the world’s leading companies deploying inference to production, and we’d love to work with you too! Contact us here.
As indicated by the genericity of the performance discussion in this post, this work readily translated to supporting other customers and to serving other models too. It has also motivated longer-term improvements to our inference serving and evaluation stack, including our own eval platform, new routing systems, and better benchmarking techniques, which we’ll talk more about soon.
You may have noticed that we’re quite fond of our core autoscaling cloud infrastructure platform, and we credit it with making possible many of the lower-level engineering choices in this post.
That’s why the post exists at all — if we truly believe our platform is the best for running high-performance inference, why hide inference perf “alpha”? And that’s why we’re committed to open source. Not only did we upstream our work on the inference engine, we also released the code and configuration as the backing source for Modal Auto Endpoints. Spin up a Dedicated Endpoint for Kimi K2.6 right now with modal endpoint create and you’ll be able to inspect our setup — or modify it for your own purposes.
Finally, if you made it this far, we bet you’re interested in and capable of pushing the frontier of inference performance. Check out modal.jobs if you’d like to do that with us! We’d love to hear from both systems engineers with an interest in inference and from inference specialists.
This work would not have been possible without the amazing work of open weights model providers like Moonshot AI, open source inference engines like SGLang, and the entire community of researchers and engineers who share their work for others to build on.
Active-Active Redis distributes data across multiple regions, allowing each regional database instance to serve both reads and writes. What kind of magic allows that? Redis uses conflict-free replicated data types (CRDTs) to resolve concurrent updates and ensure that the instances eventually converge to a consistent state.
A standard redis-py client connects to a single configured endpoint. In order to be able to quickly fail over between instances, in case of a failure, an application could create separate clients for multiple regional database instances, but it must then implement (and maintain!) health monitoring, endpoint selection, failover, and failback. Having countless applications around the world implementing the same logic, some with more success than others, doesn’t really make a lot of engineering sense, so we decided to come up with a single “canonical” implementation which provides a client API that manages these responsibilities, thereby enabling client-side geographic failover.
This API is exposed on MultiDBClient - a wrapper over the regular single and cluster client instances. The MultiDBClient routes traffic to one selected endpoint (active database) - while monitoring the health of all configured endpoints. If the active database is considered unhealthy, the client selects another healthy endpoint according to the configured weights and redirects traffic to it. When automatic failback is enabled, the client periodically evaluates the unavailable endpoints and can return to the highest-weighted healthy endpoint. MultiDBClient does not broadcast commands or replicate data across the endpoints. Data replication remains the responsibility of the Active-Active database layer letting the CRDTs shine.
MultiDBClientEach DatabaseConfig defines one endpoint and its weight. Create one for every regional database instance. MultiDbConfig collects these endpoint configurations and defines behavior that applies to the overall MultiDBClient setup, including health checks, retries, failover, and failback. As we’ll see later, there are plenty of knobs to allow for different scenarios and setups.
MultiDBClient delegates endpoint communication and connection management to an underlying Redis or RedisCluster client. You can select tone or the other globally with MultiDbConfig.client_class: use Redis for standard endpoints and RedisCluster for endpoints exposing the OSS Cluster API. Pass endpoint-specific client options through DatabaseConfig.client_kwargs.
The MultiDBClient uses a circuit-breaker pattern to control traffic to each endpoint. When the active endpoint is considered unhealthy, its circuit opens and the client fails over to the highest-weighted healthy endpoint. The concept comes from electric circles, where circuit breaker elements protect an electric circle from the damage of the excess of what the equipment can actually carry.
The circuit breaker is only one of two complementary mechanisms that detect failures. The proactive background health check periodically evaluates every endpoint using the configured health-check policy. The reactive FailureDetector we described above, observes command successes and failures within a sliding window and opens the circuit when the configured failure thresholds are reached. Together, they allow the client to detect and respond to failures more quickly.
When the active endpoint’s circuit opens, the failover strategy selects the highest-weighted healthy endpoint and routes subsequent commands to it. It’s worth noting that commands already in flight against the previous endpoint cannot be redirected. If an in-flight write succeeds there, its result might not be immediately visible through the new endpoint until Active-Active replication catches up.
Eligible command failures are handled by the global retry policy. Before each retry, MultiDBClient checks the currently active endpoint, allowing the command to be retried against the newly selected database.
Automatic failback periodically checks whether a higher-weighted endpoint has recovered. Once the endpoint is healthy, MultiDBClient can switch traffic back to it. Weights can be configured to prioritize the endpoint closest to the application.
This system would allow you to always prefer endpoints closest to the application, while being prepared for outages.
For more control, you can disable automatic failback by setting auto_fallback_interval to -1 and dynamically selecting a healthy endpoint explicitly with set_active_database().
A health check can run several probes before deciding whether an endpoint is healthy. The policy determines how their results are combined, balancing fast failover against tolerance for transient failures:
In short: choose HEALTHY_ALL to prioritize endpoint quality, HEALTHY_ANY to prioritize stability and avoid unnecessary switching, or HEALTHY_MAJORITY when you need a balance between the two.
For example, the following configuration runs five probes and considers an endpoint healthy when at least three succeed:
Health-check interval: Frequent checks improve failure detection when application traffic is absent, but every application instance runs them, which sometimes can be too wasteful.
For high-traffic systems, let the reactive detector lead and use health checks as a slower safety net. For low-traffic systems, use more frequent health checks because the reactive detector may not receive enough commands.
Failure-detection window: A short window reacts quickly and forgets old failures sooner, which suits high traffic. Low-traffic applications need a longer window; otherwise they may never collect enough samples to reach min_num_failures.
Failover attempts: failover_attempts × failover_delay defines the approximate recovery window when every endpoint is temporarily unavailable. A latency-sensitive API should keep this window short and return control to the application. A background worker can wait longer for an endpoint to recover.
This preset favors throughput and fast failure detection:
MultiDBClient provides runtime notifications whenever failover or failback changes the active database. Applications can use custom event listeners to record the transition, update metrics, trigger alerts, or synchronize external state. These listeners are registered through an EventDispatcher supplied to MultiDbConfig.
When redis-py observability is enabled, MultiDBClient records the redis.client.geofailover.failovers counter. It includes the attributes:
db.client.geofailover.fail_fromdb.client.geofailover.fail_todb.client.geofailover.reason, such as automatic or manualTo see how MultiDBClient handles regional failover and failback end to end, explore the redis-py geographic failover example.
Active-Active Redis takes care of syncing your data across regions, while MultiDBClient helps your Python application stay connected. It brings health monitoring, weighted endpoint selection, failover and failback, retries, and observability together behind a familiar Redis client API.
There’s no one-size-fits-all configuration for high availability. The defaults offer a balanced starting point, but you should still tune them for your workload, expected failover speed, and tolerance for false positives.
So, when a region goes down, your Python application has a clear path to keep running.
Stella Ma
Product Marketing Manager
Arti Arutiunov
Product Manager
Datadog Real User Monitoring (RUM) SDK settings live in your application code, so changing how the SDK collects RUM data has traditionally required shipping a new application version. These configuration changes can include adjusting sampling rates, enabling Session Replay, or changing which events the SDK collects. For mobile teams, this means that updates often sit in app store review for days or weeks before users start adopting the new version. Full user adoption can take weeks or months longer. These delays make it hard to react when an incident or performance regression calls for more RUM data.
RUM Remote Configuration lets you change supported SDK settings directly from Datadog, without modifying or redeploying your frontend application code. Once an eligible browser, iOS, or Android SDK has RUM Remote Configuration enabled, you can publish new SDK settings from the Application Management page in RUM or Product Analytics. Supported SDK versions in your frontend applications will pick up those settings the next time they initialize. RUM Remote Configuration requires minimal changes to implement and works alongside your existing SDK setup, so you can quickly adopt it without revisiting a configuration that’s already working.
In this post, we’ll show you how you can:
RUM Remote Configuration supports scenarios where the RUM data you need changes faster than your release cycle. For example, if an application starts showing slow page loads or unresponsive interactions, you can increase the profiling sample rate from the Datadog UI without shipping any new code. Increasing the profiling sample rate lets you see what’s happening at the method level during key moments, like page loads, to help you investigate and resolve the issue.
For mobile applications, the ability to update SDK settings remotely is critical. RUM Remote Configuration removes app store review and adoption lag for supported settings. Once an SDK version that supports a given setting is deployed, you can adjust it independently of your release cycle.
RUM Remote Configuration is supported for new and existing RUM applications, and it requires only a small change to your RUM SDK setup. Each RUM application has a remote configuration ID that its SDKs use to retrieve the latest published configuration for that RUM application. New applications have the remote configuration ID included in the RUM initialization snippet, with Remote Configuration enabled by default. For an existing application, add the ID to your initialization code and enable Remote Configuration.
After setup, review your first configuration and decide which settings should override the SDK ones. Datadog won’t apply the values shown in the UI over your existing SDK settings until you save and publish, so you can confirm the settings before RUM Remote Configuration becomes the source of truth for those supported parameters.
Settings for each RUM application are organized separately by browser, iOS, and Android platforms. You can manage sampling rates like rum.sessionReplaySampleRate and rum.traceSampleRate, privacy settings, event tracking, and app attributes, with specific settings varying by platform.
Separate permissions for viewing versus editing and publishing let you open up visibility to more engineers while limiting who can actually change SDK behavior. The UI will also display who last modified settings for easy auditing purposes.
With RUM Remote Configuration, you can adapt RUM data collection to an incident or investigation and have your applications reflect the new settings without waiting for a new release or for app store review. You also don’t have to wait for users to adopt a new app version.
To learn more, see the RUM Remote Configuration documentation. If you don’t already have a Datadog account, sign up for a 14-day free trial to start monitoring your applications with RUM.
General-purpose agents handle a broad range of tasks, but you still need them to follow the procedures that run your business: compliance checks, document-processing workflows, escalation policies, engineering conventions. Encoding all of that in one system prompt or in application logic gets hard to maintain and update. Skills are a modular alternative. A skill is a reusable set of instructions, usually stored in a SKILL.md file, that teaches an agent a domain-specific task like redacting a contract, reconciling an invoice, or following a team’s pull-request conventions. Because skills follow the open Agent Skills standard, they are portable across compatible harnesses, and the agent loads only the skill it needs at runtime instead of carrying every procedure in its core instructions.
A skill packages one or more tools with the context an agent needs to use them correctly:
This modular approach helps teams specialize agents faster, reuse proven procedures across agents and workflows, keep behavior consistent, and update domain-specific guidance without fine-tuning the underlying model or rewriting the agent’s core logic.
This skill composability in agents introduces two failure modes that general output-quality metrics can miss: the agent invokes a skill that is not appropriate for the task and the agent invokes the right skill but skips or only partially follows its instructions. Both failures can produce a fluent, plausible response without having used your pre-determined domain knowledge. An evaluation therefore cannot examine the final response alone.
To make these failures measurable, Strands Evals SDK and Amazon Bedrock AgentCore Evaluations, a capability of Amazon Bedrock AgentCore, add skill-focused evaluators:
In this post, you will learn how to evaluate skill selection and instruction following from a recorded trajectory in Strands Evals, add deterministic routing checks to a test suite, evaluate skill behavior from OpenTelemetry traces with AgentCore Evaluations, and interpret per-skill results to choose the right fix, all through the AgentCore CLI.
An agent receives a task and a catalog of skills, chooses a skill, loads it, and acts. The run is recorded as a trajectory in Strands Evals or an OpenTelemetry trace in your observability layer. This record can now be used for all three skill evaluators. Skill Selection Accuracy checks whether each invoked skill fits the task and whether the agent invoked the correct skill. The following figure shows how the agent chooses a skill from the 1:n skills provided to it. Skill Selection Accuracy then scores whether the selected skill is the correct one for the task. You can find the prompt template and the rubric of this evaluator in the prompt template documentation.
Skill Instruction Following asks how fully the agent followed that skill’s prescribed steps. You can find the prompt template and the rubric for this evaluator in the prompt template documentation.
SkillInvoked is deterministic. It calls no model and is specific to Strands Evals.
An overview of these skill evaluators is demonstrated diagrammatically in the following figure.
Consider an HR assistant agent with skills for paid time off (PTO) planning and discussing employee benefits. An employee asks about their dental and vision benefits. If the agent invokes the benefits skill, it may produce a more polished response. If the tool call succeeded but the agent chose the wrong playbook, Skill Selection Accuracy isolates that routing decision.
Now suppose the agent correctly invokes the PTO-planning skill for a related request. The skill instructs the agent to identify the employee_id, check the PTO balance, check the rollover rules against the latest HR policy, and then submit a PTO request if the conditions allow. If the agent checks the PTO balance but skips the rollover rules, the agent might still return a plausible response while violating the prescribed process. Skill Instruction Following isolates that execution failure and identifies the skipped step.
The failures require different fixes. An inappropriate selection often points to overlapping or ambiguous skill descriptions. Incomplete instruction following might call for clearer steps, a different skill structure, or a more capable agent model.
| Evaluator | Availability | Score | Question answered |
| Skill Selection Accuracy | Strands Evals and AgentCore Evaluations | Binary, per invoked skill | Was invoking this skill appropriate for the task? |
| Skill Instruction Following | Strands Evals and AgentCore Evaluations | Five levels, per invoked skill | How fully did the agent follow this skill’s prescribed steps? |
| Skill Invoked | Strands Evals | Binary, deterministic | Was this named skill successfully loaded? |
Because judge-based evaluators return per-invoked-skill results, multi-skill runs remain diagnosable: you can identify which selection or instruction-following result lowered the aggregate score. If no skill is invoked, the judge-based evaluators don’t produce a score. Pair them with SkillInvoked when a regression test has a known routing requirement.
InvokeModel permission for the judge model.To follow the Strands Evals section, install the SDKs:
pip install strands-agents-evals strands-agentsYou also need a recorded agent run. The skill evaluators accept either a Strands Evals Session or a raw message list as the trajectory. At launch, skill extraction recognizes signals from the Strands AgentSkills plugin, Claude Code, Claude Agent SDK, OpenAI Agents SDK, Codex, Gemini CLI, OpenHands, Google ADK, and generic SKILL.md file reads.
To follow the AgentCore Evaluations section, you need:
The examples in this post use the AgentCore CLI:
npm install -g @aws/agentcoreStrands Evals is useful when you control the test cases and can rerun the agent during development or continuous integration. Check out the complete Strands evals code sample created for the HR assistant agent in the complete code sample.
from strands_evals import Case, Experiment
from strands_evals.evaluators import (
SkillInstructionFollowingEvaluator,
SkillInvoked,
SkillSelectionAccuracyEvaluator,
)
case = Case(
name="q3-revenue-tables",
input="Summarize the revenue tables in q3-report.pdf",
)
evaluators = [
SkillSelectionAccuracyEvaluator(),
SkillInstructionFollowingEvaluator(),
SkillInvoked(skill_name="pdf-table-extraction"),
]The skill evaluators read the run’s trajectory. TracedHandler collects the agent’s spans and attaches them as the trajectory:
from strands import Agent
from strands_evals import TracedHandler, eval_task
@eval_task(TracedHandler())
def task_function():
return Agent(...) # your skill-equipped agent
experiment = Experiment(cases=[case], evaluators=evaluators)
report = experiment.run_evaluations(task_function)
report.run_display()Suppose the agent loaded pdf-table-extraction, ran pdftotext -layout, but never opened the extracted file or located the table boundaries. A simplified report:
SkillSelectionAccuracyEvaluator: score=1.00, pass=True
pdf-table-extraction: The skill directly matches the request.
SkillInstructionFollowingEvaluator: score=0.50, pass=False
pdf-table-extraction: The extraction phase was completed,
the table-boundary phase was skipped.
Steps:
- Extract text with layout preservation: covered
- Locate table boundaries: skipped
- Summarize each table's headline figure: partial
SkillInvoked: score=1.00, pass=True
skill 'pdf-table-extraction' was invokedSkill Instruction Following uses five ratings: Fully Followed (1.0), Mostly Followed (0.75), Partially Followed (0.5), Minimally Followed (0.25), and Not Followed (0.0). It passes at Mostly Followed or better.
SkillInvoked for every critical skill that a specific regression case must invoke. Because it doesn’t call a model, it is a fast routing assertion.report.test_passes: if not all(report.test_passes): raise SystemExit(1).AgentCore Evaluations works directly with existing OpenTelemetry traces, supporting on-demand evaluation, batch processing of stored sessions, and continuous sampling of live traffic. Telemetry is organized into sessions, traces, and spans. Because skill evaluators operate at the tool-call level, each result includes a spanContext with the sessionId, traceId, and spanId of the recognized skill invocation.
A skill invocation is recognized through either a SKILL.md filesystem read, which works across frameworks, or a native skill-loading tool in Strands Agents, LangGraph Deep Agents, Google ADK, or the Claude Agent SDK.
AgentCore Evaluations includes two built-in judge-based evaluators for skills. To score something the built-ins don’t cover, create a custom evaluator at the TOOL_CALL level. Tool-level templates can reference skill placeholders:
| Placeholder | Contents |
{invoked_skill} |
Name of the skill loaded on this tool call |
{skill_content} |
The loaded SKILL.md body |
{available_skills} |
The catalog offered at runtime, or “(not recorded by this harness)” |
{user_message} |
The request that triggered the invocation |
{context} |
The conversation record |
For example, a template that checks one specific property of a skill run:
## Skill instructions
{skill_content}
## Conversation record
{context}
## Evaluation Question
Did the agent complete every numbered step in the skill instructions above, in the order given? Answer Yes or No.The placeholders you reference also decide when the evaluator runs. A template containing {invoked_skill} runs only on skill-invocation spans, and one containing {skill_content} additionally requires the loaded body.
The following commands target the separate skill-enabled runtime. Substitute the runtime name from skills-evaluation/agent_config.json (agent_id or agent_arn) for <skill-runtime>. Strands.SkillInvoked is client-side only and has no CLI equivalent.
Use on-demand evaluation to investigate a session, validate a recent change, or evaluate staged traffic:
agentcore run eval \
--runtime <skill-runtime> \
--evaluator Builtin.SkillSelectionAccuracy Builtin.SkillInstructionFollowing \
--session-id <session-id>Use batch evaluation to score many stored sessions at once, for example to establish a baseline before a skill-catalog change:
agentcore run batch-evaluation \
--runtime <skill-runtime> \
--evaluator Builtin.SkillSelectionAccuracy Builtin.SkillInstructionFollowingOnline evaluation samples live traffic so you can detect behavior that a curated test set didn’t anticipate:
agentcore add online-eval \
--name HRSkillsProductionEval \
--runtime <skill-runtime> \
--evaluator Builtin.SkillSelectionAccuracy Builtin.SkillInstructionFollowing \
--sampling-rate 100 \
--enable-on-createThen deploy to provision the online evaluation configuration:
agentcore deployContinuous evaluation is particularly useful for detecting catalog drift (when a new skill overlaps with an existing description), unanticipated phrasing (when real requests differ from curated test prompts), and long-session failures (when instruction following degrades as context grows).
When you’re evaluating agent skills, the first thing to internalize is that routing and execution are two different failure modes, and your evaluation strategy needs to separate them cleanly. If you see a high selection score but a low instruction-following score, that indicates the router picked the correct skill but it was not completely executed. The opposite pattern means the skill would have worked fine if only it had been invoked. Running these two evaluations together, rather than collapsing them into one pass/fail number, is what lets you tell those two stories apart.
Before you build anything custom, start with built-in evaluators to establish a baseline, so any custom logic you add afterward can cover the gap the baseline actually missed. For requirements where you already know the correct routing behavior, don’t rely on a judge model to catch it. Add a deterministic SkillInvoked assertion for every skill that must fire.
After you’re running evaluations, resist the urge to only look at the aggregate score. Per-step evidence is where the real diagnosis happens. It tells you whether an instruction was fully covered, partially completed, or skipped outright, and that level of detail is what turns a failing eval into an actionable fix. This is also why you should evaluate at every lifecycle stage instead of waiting for the final output. A failure at the end doesn’t tell you whether the router sent the request to the wrong place or the right skill executed poorly, and you need both signals to know what to fix.
Skill-level and end-to-end evaluation should be paired, because a skill can execute perfectly and still be the wrong skill for the request in front of it. You will reduce a lot of this ambiguity upstream by writing skill descriptions that are genuinely discriminative. The same logic applies to scope of a skill. A skill built to handle six unrelated functions doesn’t have a single definition of correct behavior, which makes it nearly impossible to evaluate consistently.
State clearly that only the path actually taken in a given run should be evaluated, otherwise untaken branches get miscounted as skipped steps and quietly corrupt your pass rates. And before you trust any of these scores, validate that your extraction pipeline is actually working against live traces.
As you scale this across tools, thresholds may not transfer cleanly. Each evaluation surface needs to be calibrated on its own terms, since a passing score on Strands Evals and a passing score on AgentCore Evaluations aren’t guaranteed to mean the same thing. And ultimately, none of this should live outside your deployment pipeline. Skill quality regressions need to block a release the same way a failing unit test would, or the evaluation work you’ve done up to that point isn’t actually protecting production.
Skills make agents inexpensive to specialize, but a plausible final answer does not prove that the agent selected the right procedure or followed it. Skill Selection Accuracy and Skill Instruction Following separate those failure modes and return evidence for each invoked skill. In Strands Evals, SkillInvoked adds a deterministic guard for known routing requirements. In AgentCore Evaluations, the two judge-based metrics can run on demand for a specific session, as a batch over stored sessions, or continuously over sampled traffic.
Use Strands Evals when you have test cases and recorded trajectories you can rerun. Use AgentCore Evaluations when you want to evaluate OpenTelemetry traces from staged or live agents. Many teams will use both: deterministic and judge-based gates before deployment, followed by trace-based monitoring in production.
Thank you to Ritvika Pillai, Vincent Chen, Qiaoxuan Xue, and Shoaib Javed for the AgentCore Evaluations implementation, to Po-Shin Chen for the Strands Evals review, to Anwesan Pal for early discussions on skill evaluation, to Ben Coombs for product guidance, and to everyone else who helped make this work possible.
Sangmin is an Applied Scientist at AWS AI Labs, where he conducts research and develops machine learning solutions for agentic AI, with a focus on evaluation frameworks and advancing agent behavior and performance. His interests include agentic AI, generative models, and multimodal AI. Outside of work, he enjoys traveling and exploring new places.
Bharathi is a Generative AI Data Scientist at AWS. She is passionate about Responsible AI to increase the reliability of AI agents in real-world scenarios. Bharathi guides internal teams and AWS customers on their responsible AI journey.
Shruthi is a Solutions Architect at AWS based in Chicago, Illinois. She works with startups in the US East region, helping early-stage and growth-stage companies design and build scalable cloud architectures on AWS, with a focus on generative AI, agentic AI workflows, data, and migration workloads. Outside of work, she enjoys walking, yoga, and discovering new food and coffee places.
Visakh is a Solutions Architect at AWS, working with customers and internal teams to bring legibility, trust, and reliability to production artificial intelligence (AI). His work on agentic reliability and AI safety has been presented at machine learning conferences. Outside of work, he enjoys music, birding, and sports.
Renu is a Software Development Engineer at Amazon Web Services, where she works on Amazon Bedrock AgentCore. She previously helped build AgentCore Memory and now focuses on developing scalable systems for AgentCore Evaluations & optimization, helping customers assess and continuously improve the quality of their agentic applications.
Vinayak is a Sr. Applied Scientist at Amazon Web Services. With several years of experience, he has worked on various domains of AI like computer vision, natural language processing, recommendation systems etc. Currently, Vinayak helps build new capabilities on the AgentCore and Strands, enabling customers to evaluate their Agentic applications with ease, accuracy and efficiency.
Haibo is a Principal Applied Scientist and Manager working on agentic AI at Amazon. He holds a Ph.D. from the University of Utah. His work focuses on large language models (LLMs) and AI agents, where he leads research in areas such as agent evaluation, agent tool optimization, prompt optimization, and model routing. He has served as an area chair for conferences such as AAAI and ACL, and previously as Program Chair for KDD 2025 Workshop on Prompt Optimization.
Jonathan is a Senior Software Engineer at AWS. He builds agent environments, evaluation frameworks, and post-training infrastructure that help turn advances in agentic AI into reliable production systems.
In healthcare, the best judges of whether an AI system works correctly are the people whose time it was built to protect. A clinician can quickly tell if a generated note correctly attributes a symptom or a patient was recommended the most appropriate level of care. But they cannot perform this review across thousands of encounters indefinitely. At scale, expert validation becomes the limiting factor.
Healthcare teams have approached this challenge in different ways, but many share one principle: they treat clinical review as infrastructure rather than a recurring operational cost. This blog covers how two organizations, building AI healthcare products in different ways, converge on evaluation practices. Using LangSmith, they convert expert input into durable assets such as labeled datasets, calibrated evaluators, and automated release gates. The goal is not to eliminate human judgment, but to make its value compound.
You’ll explore:
Healthcare AI agents operate across various workflows, such as patient care decisions and visit documentation.
Included Health built Dot, an AI guide powered by LangGraph and Deep Agents. It interprets ambiguous member needs, answers coverage and billing questions, routes people to appropriate care, and detects emergencies. A question about whether a scan is covered may reveal, several turns later, that the member actually needs to speak with a primary care physician. Evaluating Dot means both checking its accuracy and whether it used the member's full context to recommend a safe and appropriate next step.
Abridge transforms patient-clinician conversations into clinical notes. With a patient's consent, a physician records the visit, and Abridge converts the conversation into a note that becomes part of the longitudinal health record and supports billing. In this setting, attribution is critical. If a patient's observation is presented as a physician's conclusion, a symptom can become a billable diagnosis. Hallucinations create a different risk: a medication or dosage that was never prescribed can enter the record. A trustworthy note must preserve who said what, capture what matters clinically, and introduce nothing the conversation does not support.
These systems fail in different ways, but both teams use LangSmith to address the same constraint: accuracy is determined by someone outside the engineering team, and that person's time is often the scarcest resource in the system.
That makes expert review both indispensable and a potential bottleneck. As the Abridge team puts it: “trust is earned in drops and lost in buckets.” The goal is to make each expert judgment reusable across future tests, releases, and iterations.
Before clinical judgment is encoded into automated evaluations, teams must first define exactly what experts are evaluating. Three properties make that judgment difficult to scale.
Ground truth is rarely singular. A clinical note does not have one canonical form. What belongs varies by specialty, encounter, and clinician, and reasonable experts can disagree. Reference notes are useful, but treating a single reference as the only correct answer can penalize valid variation while still missing clinically important errors.
Correct inaction matters. Healthcare teams must evaluate whether a system acted correctly and if it recognized when not to act. For instance, Included Health's reviewers check that emergency guardrails trigger when appropriate and remain inactive in benign cases. For example, Abridge tests whether its agent stays within its boundaries and selects the tools a clinician would expect.
Reviewer expertise is part of the specification. Abridge determines upfront whether an evaluation requires a board-certified physician or a particular specialist. A judge is only as good as the judgments that calibrated it.
None of this makes clinical judgment impossible to automate. It simply defines the requirements for doing so responsibly: tolerance for valid variation, attention to what did not happen, and the right expertise behind every label.
Once teams have defined the judgment they need to preserve, they can begin converting expertise into infrastructure. Abridge converts clinician input into labeled datasets and calibrated judges that keep working after an individual review ends.
Abridge begins with known failure modes from clinician and user feedback. The team ranks them by prevalence and severity, groups them into categories such as accuracy, compliance, style, and completeness, and builds a separate judge for each. Rather than produce one general quality score, the evaluators test for specific ways a note can fail.
The time savings come from automating what happens after they've provided their judgment. Previously, a clinician wrote an annotation guide and labeled encounters, then someone manually adjusted the judge’s prompt until its scores matched those labels. Abridge now feeds the same guide and examples into an automated prompt optimization framework that generates the judge.
Because no single reference can capture every valid note, Abridge layers two approaches with complementary strengths:
Together, they balance breadth and precision: one provides scalable coverage across encounters, while the other captures the specialty-specific nuance that clinical review demands.
An optimized judge still must be validated against clinician annotations. LangSmith’s Align Evaluator gives teams an interface for comparing the two and investigating disagreements. Abridge separately asks annotators to explain their decisions, even when the output is correct. Those explanations help resolve inconsistencies and confirm the labels reflect careful review.
The result is an evaluation system that can be inspected, recalibrated, and reused. By turning individual judgments into durable evaluators, teams reserve scarce clinical expertise for the cases where it adds the most value. Production review supplies the new cases and feedback that keep those standards current.
Calibrated judges apply judgment the team has already captured. Production review supplies the next round of that judgment, revealing how the system behaves in real conversations and generating evidence for what to fix.
Included Health shows how that new evidence enters the loop. Conversations go into a LangSmith annotation queue, where clinical reviewers assess whether Dot directed the member to the right care setting, whether its emergency guardrails behaved appropriately, and whether the case requires follow-up.
Those decisions become structured labels that are exported to Included Health's data warehouse, where the data science team uses them to build operational dashboards. They also feed back into the skill definitions that govern how Dot navigates members. Each review is spent once and used three times.
The result is a powerful feedback loop. Clinical judgment becomes data, the data guides product changes, and those changes are tested against the same standards before the next release. Every review contributes to both the case at hand and the system’s future behavior.
The feedback loop pays off at release time. Instead of evaluating every candidate change from scratch, teams can test it against evidence they have already captured.
At Abridge, a model change moves through progressively more realistic stages: offline evaluations, backtesting against historical encounters, a limited A/B test, full release, and continuous production monitoring. Each stage adds a different kind of evidence.
The A/B test is the most unusual step in this process. Some of Abridge’s partners agree to be among the first 10 to 15% of customers included in a silent rollout. This lets Abridge observe signals automated judges cannot provide: whether clinicians edit the generated notes, how they rate them, and what qualitative feedback they share. Offline evaluations establish whether a change is ready for limited exposure; production behavior determines whether the rollout should expand. That process reduced Abridge’s release cycle from one or two months to a matter of days.
Included Health applied the same principle to an architectural change. Moving Dot’s supergraph to Deep Agents affected four product teams, all wary of breaking changes. The team ran its existing multi-turn simulation suite, confirmed that performance held, and completed the migration in under two weeks without significant regressions.
In both cases, release confidence became cumulative. Rather than re-establish trust with every change, teams could build on evidence they had already collected.
A faster release cycle matters only if the system performs reliably once it reaches real users. Included Health measures performance across three dimensions: adoption, routing quality, and safety.
Each metric answers a different question: Will members use the product? Does it direct them to appropriate care? Does it recognize situations that require urgent attention? Looking at them together gives the team a more complete picture of production reliability.
Following Dot's launch, Included Health reports a 75% lift in chat engagement. Among the graded conversations, clinician agreement with Dot’s care recommendations remains above the team’s 95% target, and clinical audits show that Dot identifies more than 99% of high-risk situations.
At Abridge, labeled encounters calibrate judges that run against future releases; at Included Health, clinical labels outlive the conversation that produced them. The artifacts still require review and recalibration, but the expert judgment behind them is no longer consumed by a single decision.
The same artifacts that make clinical judgment reusable—encounter traces, conversation histories, and clinician annotations—can also contain protected health information. Once teams begin storing and reusing them, security and deployment architecture become part of the evaluation design.
Abridge treats self-hosting, access controls, and auditability as requirements for its evaluation infrastructure. They also remove identifying information from conversation data before using it for learning. These are not controls to add after the evaluation pipeline is built; they shape what data can enter it in the first place.
LangSmith supports managed cloud, bring-your-own-cloud, and self-hosted deployments. Teams must decide where evaluation data will be stored, who can access it, which audit and retention controls apply, and how traces containing PHI will be handled.
Trust builds slowly in healthcare AI. It grows with every encounter handled correctly, every guardrail, and every regression caught before it reaches users. Yet one change that escapes those checks can undo it.
Healthcare teams move fast by ensuring each careful review continues working long after the review itself is complete.
For the complete customer stories, watch Building Clinical AI Agents with LangGraph: Abridge's Eval Stack for High-Stakes Healthcare and read How Included Health Built Federated Agents for Healthcare Navigation with Deep Agents and LangGraph.
When applications crash in production, how much do we actually know about what happened? And more importantly, how easy is it to debug what happened so that we can fix the bug? Let’s learn how easy it is to capture a memory dump so that we can debug it.
You know that moment when you’re in a desktop application and suddenly it hangs, the screen greys out, and it’s clear the application has stopped responding. Or when you go to a website and you’re sure you clicked on that link, but the browser is just spinning.
While the developer might have added logging and telemetry to that application and be able to follow the execution pathways from that, actually understanding the state of the application can be a lot harder.
This is where a memory dump can be useful. A memory dump can capture the application state, and depending on whether it’s a full or partial dump, you can get a view of objects in memory that are waiting for the garbage collector to clean up, including out-of-scope state that can still provide insights into the broader application behavior.
For this scenario, we’re going to look at an application that is becoming unresponsive, and a common culprit for this kind of issue is how we are using asynchronous code and tasks.
The pattern that we’re going to use to monitor the thread pool is that we’ll periodically add our own Task to it, observe how long that task takes to complete, and if it took longer than an allowed threshold, we’ll know that the thread pool is likely saturated and probably something we want to capture a dump of.
We’ll create a ThreadPoolWatcher class that will encapsulate this logic:
internal class ThreadPoolWatcher(string name = "ThreadPool Watcher", int interval = 3_000)
{
private static readonly object DumpLock = new();
private static int dumpCount;
private readonly Thread thread = new(() => Watcher(interval))
{
Name = name,
IsBackground = true
};
private static void Watcher(int interval)
{
while (true)
{
Thread.Sleep(interval);
Stopwatch stopwatch = Stopwatch.StartNew();
Task task = Task.Run(stopwatch.Stop);
if (!task.Wait(interval))
{
Console.WriteLine($"Task did not complete within {interval} ms");
}
if (stopwatch.ElapsedMilliseconds <= interval) continue;
lock (DumpLock)
{
if (dumpCount++ > 0)
{
Console.WriteLine("Dump already created for this run; skipping additional dumps.");
continue;
}
}
// Took over the interval to complete
Console.WriteLine($"Task took too long: {stopwatch.ElapsedMilliseconds} ms");
string path = Path.Combine(AppContext.BaseDirectory, $"fulldump-{Environment.ProcessId}-{DateTime.Now:yyyyMMdd-HHmmss}.dmp");
if (OperatingSystem.IsWindows())
{
WindowsDumper.WriteCurrentProcess(path);
}
else if (OperatingSystem.IsLinux())
{
LinuxDumper.WriteCurrentProcess(path);
}
}
}
internal void Join() => thread.Join();
internal void Start() => thread.Start();
}There are a few things going on in this code, so let’s dissect it a bit.
First, we’re creating a new Thread (which we’re providing a name so we can identify it while debugging) that, when run, will continually invoke the Watcher method. The watcher uses Thread.Sleep to pause for the specified interval between each check.
When the thread wakes up, it adds a new task to the thread pool and measures how long it takes to complete. If the task takes longer than the allowed threshold, it indicates that the thread pool is likely saturated and we may want to capture a memory dump to investigate further. Otherwise, it goes back to sleep. This is a simple way to observe thread-pool behavior in real time by exploiting task timing.
For production use, you should also guard against repeated dump generation. Full dumps can be large and may include credentials, tokens, connection strings, or other sensitive data. Storing them in a restricted directory, adding a cooldown, or limiting the number of files generated is a safer pattern than dumping on every delayed probe.
Then, if the task took longer than the specified interval, we’ll dump the memory of the current process, using either Windows or Linux APIs.
On Windows, to create a memory dump of the current process, we’re going to need to call into a native library, dbghelp.dll, and have Windows generate the dump for us.
[SupportedOSPlatform("windows")]
internal static class WindowsDumper
{
[Flags]
private enum DumpType : uint
{
Normal = 0x00000000,
WithDataSegs = 0x00000001,
WithFullMemory = 0x00000002,
WithHandleData = 0x00000004,
WithUnloadedModules = 0x00000020,
WithFullMemoryInfo = 0x00000800,
WithThreadInfo = 0x00001000,
WithTokenInformation = 0x00040000,
}
[DllImport("dbghelp.dll", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool MiniDumpWriteDump(
IntPtr hProcess,
uint processId,
SafeHandle hFile,
DumpType dumpType,
IntPtr exceptionParam,
IntPtr userStreamParam,
IntPtr callbackParam);
/// <summary>
/// Writes a full memory dump of the current process.
/// </summary>
public static void WriteCurrentProcess(string path)
{
Write(Process.GetCurrentProcess(), path);
}
/// <summary>
/// Writes a full memory dump of <paramref name="process"/> to <paramref name="path"/>.
/// </summary>
public static void Write(Process process, string path)
{
ArgumentNullException.ThrowIfNull(process);
ArgumentException.ThrowIfNullOrEmpty(path);
string? directory = Path.GetDirectoryName(Path.GetFullPath(path));
if (!string.IsNullOrEmpty(directory))
{
Directory.CreateDirectory(directory);
}
using FileStream stream = new(path, FileMode.Create, FileAccess.ReadWrite, FileShare.None);
// Full memory dump: entire address space (including the heap), handles, modules and thread state.
bool success = MiniDumpWriteDump(
process.Handle,
(uint)process.Id,
stream.SafeFileHandle,
DumpType.WithFullMemory |
DumpType.WithFullMemoryInfo |
DumpType.WithDataSegs |
DumpType.WithHandleData |
DumpType.WithUnloadedModules |
DumpType.WithThreadInfo |
DumpType.WithTokenInformation,
IntPtr.Zero,
IntPtr.Zero,
IntPtr.Zero);
if (!success)
{
throw new Win32Exception(Marshal.GetLastWin32Error(), $"MiniDumpWriteDump failed for process {process.Id}.");
}
}
}This is a dump class, and because it only works on Windows, we’re annotating it with the SupportedOSPlatform("windows") attribute. Next, there’s an enum that defines the different types of memory dumps that can be created, such as full memory dumps, dumps with handle data, and dumps with thread information. The MiniDumpWriteDump function from dbghelp.dll is then imported to actually perform the dump, and the class provides convenient methods to write a dump of the current process or any specified process.
For this example, we’re adding everything to the memory dump that is generated, which means it will be quite large. In our sample, this produces a dump of approximately 125 MB, although the size depends on the process’s memory usage and selected dump contents.
To create an equivalent memory dump on Linux can be a little more difficult as it will depend on the distribution that is used, whether it’s running in a container, and the permissions the process has. Here’s an example of creating a full memory dump using the createdump utility that ships with the .NET runtime.
[SupportedOSPlatform("linux")]
internal static class LinuxDumper
{
// Yama LSM (see /proc/sys/kernel/yama/ptrace_scope). With the default scope of 1
// ("restricted ptrace"), a process may only be ptraced by its own descendants unless
// it explicitly designates another process (or PR_SET_PTRACER_ANY) as an allowed
// tracer via prctl(PR_SET_PTRACER, ...). "Yama" spelled out in ASCII.
private const int PR_SET_PTRACER = 0x59616d61;
private static readonly IntPtr PR_SET_PTRACER_ANY = new(-1);
[DllImport("libc", SetLastError = true)]
private static extern int prctl(int option, IntPtr arg2, IntPtr arg3, IntPtr arg4, IntPtr arg5);
public static void WriteCurrentProcess(string path)
{
AllowAnyProcessToPtraceSelf();
Write(Process.GetCurrentProcess(), path);
}
/// <summary>
/// Best-effort: on distros using the Yama LSM (e.g. Ubuntu/Debian) with the default
/// ptrace_scope of 1 ("restricted ptrace"), a process may only be ptraced by its own
/// descendants - not the parent that spawned it. createdump attaches to us as our
/// child, so we explicitly allow any process to ptrace us. This is a no-op (and
/// harmless) on distros where Yama isn't enabled (e.g. many Fedora/RHEL setups), and
/// is swallowed entirely if "libc" or prctl can't be resolved at all, which can happen
/// on musl-based distros like Alpine that don't ship an unversioned libc.so.
/// </summary>
private static void AllowAnyProcessToPtraceSelf()
{
try
{
_ = prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero);
}
catch (Exception ex) when (ex is DllNotFoundException or EntryPointNotFoundException)
{
// libc/prctl isn't resolvable this way on this platform (e.g. musl/Alpine) -
// fall through and let createdump itself report any real permission failure.
}
}
public static void Write(Process process, string path)
{
ArgumentNullException.ThrowIfNull(process);
ArgumentException.ThrowIfNullOrEmpty(path);
string? directory = Path.GetDirectoryName(Path.GetFullPath(path));
if (!string.IsNullOrEmpty(directory))
{
Directory.CreateDirectory(directory);
}
string createDumpPath = FindCreateDump();
using Process createDump = new()
{
StartInfo = new ProcessStartInfo
{
FileName = createDumpPath,
// --full: entire address space (analogous to MiniDumpWithFullMemory).
// -f: explicit output path (createdump would otherwise pick its own name/location).
ArgumentList =
{
"--full",
"-f", path,
process.Id.ToString(),
},
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
},
};
createDump.Start();
string stdout = createDump.StandardOutput.ReadToEnd();
string stderr = createDump.StandardError.ReadToEnd();
createDump.WaitForExit();
if (createDump.ExitCode != 0)
{
string hint = process.Id != Environment.ProcessId
? " Dumping another process typically requires running as root, the " +
"CAP_SYS_PTRACE capability, or /proc/sys/kernel/yama/ptrace_scope set to 0."
: " If this is a container, ensure ptrace isn't blocked by seccomp " +
"(add --cap-add=SYS_PTRACE) or by an SELinux/AppArmor policy.";
throw new InvalidOperationException(
$"createdump failed for process {process.Id} with exit code {createDump.ExitCode}.{hint}{Environment.NewLine}{stdout}{stderr}");
}
}
private static string FindCreateDump()
{
string runtimeDirectory = RuntimeEnvironment.GetRuntimeDirectory();
string candidate = Path.Combine(runtimeDirectory, "createdump");
if (!File.Exists(candidate))
{
throw new FileNotFoundException(
$"Could not find the 'createdump' utility next to the runtime directory '{runtimeDirectory}'.",
candidate);
}
return candidate;
}
}This class does a couple of extra things. It uses prctl from libc to allow the child createdump process to attach to its parent under Yama’s restricted ptrace policy, and it locates the createdump utility next to the runtime directory so it can create a full memory dump. In our sample, this produced a dump of approximately 800 MB, although the size depends on the process’s memory usage and the dump configuration.
Now that we can capture memory dumps of our processes, let’s simulate a problem by intentionally causing an issue in our application that we can then analyze using the memory dump.
internal static class ApplicationRunner
{
public static void DoLotsOfWork() => Parallel.For(0, 1000, DoSomeWork);
private static void DoSomeWork(int i)
{
Console.WriteLine("Running task {0}", i);
Thread.Sleep(10_000);
}
}This code is going to simulate running a lot of parallel tasks, each of them “doing something” that will take a long time to complete, but there’s no restriction on the number of tasks that can be run on the thread-pool, potentially saturating it and causing performance issues or the appearance of the application hanging.
Then we can run our application by creating the ThreadPoolWatcher instance, starting it, and running the workload while the dedicated watcher thread waits for the next probe.
var tpw = new ThreadPoolWatcher();
tpw.Start();
ApplicationRunner.DoLotsOfWork();
// The application keeps running until the process exits or the watcher is stopped.After a while, our application will start to become unresponsive and generate the dump file.
The .dmp files that are generated can be opened in Visual Studio with managed debugging, allowing us to walk the call stacks, inspect available variables, and view the state of the application at the time the dump was created.
If you want to learn more about analyzing memory dumps and using the parallel stacks view in Visual Studio, you can read the companion article on the Visual Studio blog.
In this article, we’ve seen how easy it can be to have our application create memory dumps when it encounters performance issues or an unresponsive thread pool, allowing us to analyze the state of the application at the time of the problem instead of relying only on logging and reproducing scenarios. Combining this with the Visual Studio tools for analyzing memory dumps, we can gain deep insights into the behavior of our application and more effectively diagnose and resolve complex issues.
By incorporating memory dump generation into our development and monitoring practices, we can proactively address potential performance bottlenecks and hangs, ultimately leading to more robust and reliable applications.
The post Creating a memory dump in C# appeared first on .NET Blog.
Code quality is shaped by countless decisions, from how developers manage complexity to how quickly they detect regressions. But common concepts such as technical debt, cognitive complexity and unit testing are not always clearly understood.
In a new code quality Q&A session, we asked members of the Qodana team to answer some frequently searched questions about maintaining code quality. Here are their concise, practical explanations.
“Poor code quality describes the code itself, while technical debt describes a compromise that creates future work. For example, “Let’s use hardcoded values instead of configs, just because we need to write POC ASAP” is intentional technical debt.
By contrast, “I don’t know why I need to use configs, that’s why I hardcode values” is poor code quality caused by a lack of knowledge. Technical debt may lead to poor code quality, but poor code quality is not always caused by technical debt”. – Alexandr Kugushev
“Best practice to minimize complexity in nested loops – is avoiding them: extract functions, double check conditions (maybe it’s possible to simplify them with different logic rules), explicitly name intermediate results, use iterators, generators, map, filter, reduce functions, try to flatten data before processing and so on”. – Anastasia Lavrenko
“Code coverage and mutation testing serve different but related purposes. Code coverage measures how much production code is executed by unit tests. Mutation testing evaluates the quality of those tests by deliberately introducing small changes to the code and checking whether the tests detect them. If the tests still pass, the mutation survives, indicating a possible gap in the test suite.
There is no universal ideal mutation score, as results vary by project and testing strategy. However, scores of around 40–60% are generally considered a reasonable starting range. Higher scores can indicate a stronger test suite, but the goal should be meaningful test quality rather than reaching a specific number”. – Aleksander Movsesov
“Unit tests provide the fastest feedback on code changes. By testing individual units of behaviour in isolation, they catch regressions close to the point where they are introduced, make refactoring safer and confirm that the code continues to behave as expected”. – Arman Ayvazyan
“I think the best way to track if code quality is actually improving is to look at the trends over time. Static analysis can give you signals like how many new issues are being introduced, how severe those issues are, and whether technical debt is going up or down. But you should also be able to see the impact outside of static analysis.
If code quality is improving, you should start seeing fewer Sev 1 issues making it into production and a faster MTTR when issues do happen. That gives you a better picture of whether the changes you are making are actually improving the quality and reliability of the software”. – Alex Costa
“First of all, we should draw a distinction between vulnerability scanning and static analysis. What Qodana does is not always vulnerability scanning – a lot of the issues it finds cannot be exploited by a third party. Sure, Qodana does find vulnerabilities, but it also finds regular bugs, dangerous functions, dead code, and so on.
That being said, eliminating false positives is done in a similar way regardless of the issues – by giving the tool you use more information about your repository. The most common case of a false positive is the automated tool not knowing that you intended something to be written in a particular way, for example when you disagree with established practices, or when you use an older standard of the language that doesn’t support a safer alternative, or when you are forced into an unsafe code pattern by a third-party dependency.
All of this can be addressed: for repo-wide false positives, you can exclude the inspections in qodana.yaml. For individual lines and blocks for code, you can explicitly silence inspections with comments like // NOLINT(<INSPECTION ID>) and // NOLINTNEXTLINE(<INSPECTION ID>)“. – Anna Zhukhova
Maintaining code quality requires more than fixing individual issues. Teams need to make technical compromises consciously, keep code understandable and create fast feedback loops that catch problems early.
Automated code analysis can support these practices by identifying quality issues continuously and helping teams apply consistent standards throughout development. With Qodana, teams can bring JetBrains inspections into their CI/CD pipelines and address problems before they become more difficult and expensive to resolve.
Need answers to your most pressing code quality and security questions? Leave a comment below for the next round or find out how Qodana can help you secure and improve your codebase.
GPU acceleration can speed up compute-intensive robotics workloads, but a fast CUDA kernel alone does not guarantee a fast ROS 2 graph. As messages move between nodes, they may continue to be serialized or copied through CPU memory, eroding the benefits of keeping perception and AI workloads on the GPU (Figure 1).
With the upstream rosidl::Buffer abstraction and the CUDA buffer backend that NVIDIA recently contributed to ROS Lyrical, ROS 2 nodes can exchange GPU-resident payloads through zero-copy transport when runtime conditions allow, while preserving standard ROS 2 messages and node boundaries. All nodes in NVIDIA Isaac ROS 5.0 have been updated to use the CUDA buffer backend and benefit from the more efficient data movement enabled by rosidl::Buffer.
Existing ROS 2 nodes can adopt rosidl::Buffer with minimal changes. The more challenging task is identifying the correct boundaries to update. This requires a careful audit of allocations, serialization, stream ownership, and fallback behavior.
This tutorial walks you through how to turn that audit into an agent-driven workflow. An AI coding agent uses the purpose-built migrate-node-to-rosidl-buffer skill to inspect an existing CUDA-accelerated node, trace data movement, plan a minimal interface-preserving refactor, and verify that the CUDA transport path is actually enabled. You’ll learn how to use the agent skill to update the node to adopt the CUDA buffer backend. The resulting accelerated workload can then be deployed on NVIDIA Jetson AGX Thor.
rosidl::Buffer and CUDA buffer backendIn ROS 2 Lyrical, variable-length primitive array fields such as uint8[] are represented in generated C++ code by rosidl::Buffer<uint8_t>. The default CPU-backed rosidl::Buffer behaves like the std::vector<uint8_t> interface existing ROS 2 code expects, preserving source compatibility. The pluggable abstraction also allows platform vendors to support externally managed storage without defining a separate ROS message type.
NVIDIA contributed the CUDA buffer backend for ROS 2 Lyrical. It implements rosidl::Buffer<uint8_t> storage with CUDA Virtual Memory Management (VMM). When publisher and subscriber meet backend runtime requirements, the payload can move between co-located nodes without serialization or host copies. Otherwise, ROS 2 automatically falls back to the CPU path that’s compatible with any existing ROS 2 nodes. The optimized path requires the same host, CUDA device, Linux user, and a supported RMW implementation (for example, rmw_fastrtps_cpp and rmw_zenoh_cpp).
Together, rosidl::Buffer and the CUDA buffer backend move memory sharing and data-lifetime management behind a standard ROS 2 field. This means the upstream capability is easier to adopt in GPU-accelerated robotics applications, so you can focus on node logic while retaining CPU fallback for incompatible peers.
This tutorial uses the Depth Anything 3 (DA3) TensorRT ROS 2 node as the example. The DA3 model predicts spatially consistent geometry from an arbitrary number of visual inputs, with or without known camera poses.
We aim to update this node to adopt the introduced CUDA buffer backend to take advantage of the performance improvement offered by the rosidl::Buffer feature. The node is particularly useful as a migration example because its algorithm is already GPU-accelerated.
This node’s callback converts the incoming ROS image to an OpenCV view, runs monocular metric-depth inference with NVIDIA TensorRT, converts the resulting cv::Mat back to a ROS image, and publishes it as a floating-point depth image.
The code is straightforward, but the CPU-backed ROS boundary surrounds a GPU-native algorithm. That CPU boundary is appropriate for a CPU producer or consumer, but it is unnecessary when the nodes on both sides can already produce and consume CUDA memory. In that case, the two payload-sized host transfers, host allocation, and serialization work become an optimization opportunity at the interface.
The goal is therefore not to redesign the model or replace its standard messages; rather, it is to preserve the existing ROS contract while allowing the output Image.data field to carry storage from an appropriate backend.
An AI coding agent is well suited to investigative work: following payloads through callbacks and helper libraries, finding host-device boundaries, preserving the node contract, and coordinating source, dependency, launch, and test changes.
The migrate-node-to-rosidl-buffer skill turns this analysis into a repeatable workflow. Rather than replacing the node with a template or rewriting code automatically, it directs the agent to:
rosidl::BufferUsing the rosidl::Buffer migration skill, the agent updates the node’s dependencies and interfaces to adopt the CUDA buffer backend. Most changes adapt the TensorRT wrapper to accept CUDA buffer handles for input and output data while preserving its existing API. The ROS transport change remains small: one subscription option, one CUDA allocation, two stream-aware handle extractions, and one publish. No custom message, duplicate CUDA topic, or CPU/CUDA publisher branch is required.
The following sections explain the key changes you can expect from the skill for the node migration.
First, the skill helps add CUDA buffer backend packages (cuda_buffer and cuda_buffer_backend) as additional dependencies. The message definition does not change—the node continues using sensor_msgs/msg/Image.
The subscriber is then updated to accept messages with CUDA-backed buffers. CPU remains an acceptable fallback by default, so the node-level callback does not need separate CPU and CUDA implementations.
rclcpp::SubscriptionOptions options; options.acceptable_buffer_backends = "cuda"; sub_image_.subscribe( this, image_base_topic, image_transport, rclcpp::SensorDataQoS().get_rmw_qos_profile(), options);
The existing image_transport and message_filters topology remains in place. The subscription options are simply forwarded through it.
The subscriber callback still accepts bgr8, preserves the header, dimensions, encoding, and byte stride, and converts with cv_bridge only when a different input encoding requires it. With the update, the TensorRT inference now directly writes the results to the CUDA buffer allocated in the output message, ready to publish right after the GPU work is enqueued.
The following excerpt contains the essential changes that leverage CUDA buffer APIs:
auto depth_msg = std::make_unique<sensor_msgs::msg::Image>();
depth_msg->header = bgr_image_msg->header;
depth_msg->height = bgr_image_msg->height;
depth_msg->width = bgr_image_msg->width;
depth_msg->encoding = sensor_msgs::image_encodings::TYPE_32FC1;
depth_msg->is_bigendian = false;
depth_msg->step = depth_msg->width * sizeof(float);
depth_msg->data = cuda_buffer_backend::allocate_buffer(
static_cast<size_t>(depth_msg->step) * depth_msg->height);
const cudaStream_t stream = tensorrt_depth_anything_->getCudaStream();
{
auto input = cuda_buffer_backend::from_input_buffer(
bgr_image_msg->data, stream);
auto output = cuda_buffer_backend::from_output_buffer(
depth_msg->data, stream);
tensorrt_depth_anything_->doInferenceCuda(
input.get_ptr(), bgr_image_msg->width, bgr_image_msg->height,
bgr_image_msg->step, *camera_info_msg,
reinterpret_cast<float *>(output.get_ptr()),
node_param_.point_cloud_downsample_factor,
node_param_.colorize_point_cloud,
node_param_.publish_point_cloud,
node_param_.enable_debug);
} // Release the CUDA event-tracked handles before publishing.
pub_depth_image_->publish(std::move(depth_msg));
Each line has a narrow purpose:
allocate_buffer() gives the standard Image.data field CUDA buffer-backed storage.from_input_buffer() supplies a CUDA buffer handle that is safe to consume on the TensorRT stream for read-only operations. CUDA input is used directly. CPU input is promoted to CUDA when necessary.from_output_buffer() supplies a CUDA buffer handle that is safe for write operations. The existing CUDA postprocess writes its final 32FC1 result directly into the buffer assigned to the outgoing message through the write handle, avoiding both a device-to-host copy and an intermediate device-to-device output.publish() as it normally does with the same message type while the underlying data field is now backed by the CUDA buffer backend. The CUDA memory sharing and compatibility with its downstream subscribers are handled automatically by the ROS 2 middleware as well as the backends.The skill keeps the non-CUDA route intact. Point-cloud construction and debug visualization are local CPU consumers in the original node. When enabled, they may still require a device-to-host copy and synchronization. They do not determine the representation delivered on the depth topic, so the migration leaves them as explicit optional boundaries rather than complicating the optimized publication path.
The rosidl::Buffer feature was introduced in ROS 2 Lyrical, so the migrated node is expected to work with Lyrical and above with supported RMW implementations (rmw_fastrtps_cpp and rmw_zenoh_cpp).
During the migration, the core functions and boundary message types are kept the same and add cuda_buffer and cuda_buffer_backend as additional dependencies to the package for enabling CUDA buffer backend. As a result, the overall build process and setup remain similar to the original node.
To enable CUDA buffer backend, build the packages from source. Start by cloning the source from the rosidl_buffer_backends repository where all the currently supported backends and companion packages are hosted:
git clone https://github.com/ros2/rosidl_buffer_backends.git
Note that the core functions of rosidl::Buffer are already built in ROS 2 Lyrical, so there is no need to rebuild the ROS 2 core packages.
The rosidl::Buffer backends are designed to be ROS 2 plugins. Building and sourcing the CUDA buffer backend packages in the same workspace is sufficient to make the backend available to the nodes at runtime.
colcon build --symlink-install --packages-up-to cuda_buffer_backend source install/setup.bash colcon build --symlink-install --packages-up-to depth_anything_v3 source install/setup.bash export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
You can then follow the same model preparation process and run the same launch file with the updated TensorRT node as instructed in the original repository.
The migration leaves the TensorRT computation unchanged and targets the transport around it. To inspect GPU activity and memory transfers, use NVIDIA Nsight Systems. On an eligible CUDA path, the migrated node should not show payload-sized host-to-device or device-to-host transfers at its ROS boundary. Record comparable latency measurements before and after the change.
You can also validate backend negotiation from the subscriber. When both endpoints meet the CUDA backend requirements, msg->data.get_backend_type() should report "cuda". This is useful for tests that confirm the CUDA transport path is active.
rclcpp::SubscriptionOptions options;
options.acceptable_buffer_backends = "cuda";
subscription_ = create_subscription<sensor_msgs::msg::Image>(
"/depth_anything_v3/output/depth_image", rclcpp::QoS(1),
[this](sensor_msgs::msg::Image::ConstSharedPtr msg) {
const std::string backend = msg->data.get_backend_type();
RCLCPP_INFO(get_logger(), "received backend=%s", backend.c_str());
if (backend != "cuda") {
throw std::runtime_error("CUDA transport was not negotiated");
}
auto input = cuda_buffer_backend::from_input_buffer(msg->data, stream_);
consume_on_cuda(input.get_ptr(), stream_);
},
options);
Note that the production code will often try to accept CPU fallback without throwing the error.
With the provided CUDA buffer APIs, from_input_buffer() automatically handles the CPU fallback internally. Users don’t have to distinguish the CPU path and GPU path in the callback for incoming messages. All the CUDA memory sharing and CPU-to-GPU conversion, if needed, are taken care of by the CUDA buffer backend.
The skill also contains a verification step that helps produce custom source and sink nodes for testing and validation. This is done by creating two pipelines based on the generated source and sink nodes to test the same migrated node working under both CPU and GPU setup without code changes.
In the CPU control setup, a source node that publishes messages with CPU-based data is used. The messages arrive at the TensorRT node with a buffer that is backed by plain CPU storage. The CUDA buffer APIs used in the subscriber callback automatically detects the buffer backend type and do the conversion (CPU to CUDA in this case) when needed, so the same code functions as expected to accept CPU-based messages.
In another setup, a source node that publishes CUDA buffer-based messages is used. With the migrated TensorRT node, the CUDA buffer-aware subscriber can receive the message and obtain the CUDA handle by using the CUDA buffer APIs without additional CPU-GPU copies.
The same workflow can be applied to other CUDA-accelerated ROS 2 nodes with variable-length primitive message fields. The key is to treat optimization as an end-to-end systems task. The AI agent traces data movement, identifies which fields benefit from GPU-backed storage, preserves standard ROS 2 interfaces, and verifies both the optimized path and CPU fallback. That makes the migration repeatable instead of a one-off refactor.
NVIDIA Isaac ROS 5.0 brings this workflow into an accelerated robotics software stack, while NVIDIA Jetson AGX Thor provides the edge compute platform for running demanding ROS 2 perception, inference, and autonomy workloads on the robot.
Accelerating a ROS 2 node requires optimizing GPU computation as well as data movement. With rosidl::Buffer, the NVIDIA CUDA buffer backend, and an Isaac ROS 5.0 AI-guided migration skill, existing CUDA-enabled nodes can exchange GPU-resident data with minimal code changes. This avoids unnecessary serialization and CPU copies while preserving standard ROS 2 message interface.
To get started, follow these steps:
Every tool you've ever set up has an onboarding screen you just straight click past. Usually the defaults are chosen by someone who thought about them harder than you have time to.
Engram's setup has one of those screens too, and it takes a couple of minutes to get through:
name a project → pick a template → click past topics that are already filled in for you → generate an API key.
Engram is our fully managed memory and context service purpose-built to help agents remember, learn, and improve over time.
Then you call memories.add with your raw conversation data, memories.search before your model call, and it works.
It keeps working, too. That's the interesting part, because nothing ever nudges you back to that screen. But somewhere between "it works" and "it works the way I meant it to", four decisions turn out to be yours rather than Engram's:
In this post we walk through all four, with real outputs from a project mostly running the stock personalization template.
Engram runs an asynchronous pipeline over whatever raw data you send it. By default, it:
Victoria's walkthrough video covers that end-to-end, including the console setup and the SDK:
memories.add hands back a run, not a memory, and nothing you send is searchable until that run finishes. The run status guide covers the states, and why you usually shouldn't wait on one.
For how the pipeline itself works, the Engram: Memory by Weaviate blog goes through it step by step. Which memories it pulls out of the data you send, though, is decided by your topic descriptions, so that is where we start.
When you create a new Engram project in the console, the User Personalization template sets up UserProfile and UserKnowledge for you, and offers ConversationSummary as an optional third. I took all three, then sent it something about me:
"I'm Prajjwal, a developer advocate at Weaviate. I write all my demos in Python, and I have a hard rule that they stay under 100 lines."
Five memories came back across the three topics:
[UserProfile] The user's name is Prajjwal.
[UserKnowledge] Prajjwal works as a developer advocate at Weaviate.
[UserKnowledge] Prajjwal writes all his demos in Python.
[UserKnowledge] Prajjwal enforces a hard rule that his demos stay under 100 lines.
[ConversationSummary] Prajjwal introduced himself as a developer advocate at Weaviate,
mentioned that he writes demos in Python, and follows a rule to
keep demos under 100 lines.
Other ready-to-use project templates exist as well, like Coding Assistant and Personal Claw Agent. And you can always start from a blank slate and fully customise the topics to your domain.
UserKnowledge broke that one sentence into three atomic facts, each retrievable on its own. ConversationSummary kept it whole as a flowing narrative. Same input, same pipeline, same moment, and two completely different shapes, because the two topics describe themselves differently.
We didn’t write any routing logic or a formatter. The descriptions did both, because a topic description is the memory extraction prompt, and it controls three things:
UserKnowledge is the template's catch-all topic, and anything personal about the user that might change over time belongs there. That makes it useful when you want broad coverage. But when you want a focused, niche memory, the description needs to do more than say what belongs - it also needs to define what doesn't.
Take a throwaway message like:
"Two hours lost to a Docker rebuild, my headphones died mid-call, and it started raining right as I stepped out. Anyway, I finally swapped the demo over to qwen3-embedding-8b."
Four memories came back. Three were weather, hardware, and other passing events. The agent now knows it rained on Thursday, and short of a delete call, that fact can stick around indefinitely.
To fix this, the broad topic can be replaced with a focused one by just specifying what you don't want to be remembered. I created a new topic called UserFacts to replace UserKnowledge and its description ends with a new rule:
"Do not record events, incidents, or passing conditions."
That one line made the difference as it gives the model general criteria for what to ignore. I tested it with three new messages: a late train, a cat on the keyboard, and a stolen lunch, each paired with one lasting decision. Across 24 runs, none of the noisy incidents reached memories created by UserFacts, and the decisions were captured in all of them.
So, for a focused topic, a rule about what to exclude does more work than a list of what to include. Name the kind of thing you want left out, rather than every example you can think of, and the rule can generalise to cases you never anticipated. Also, UserFacts was created only for this experiment and from here on, I switched back to the stock UserKnowledge topic.
You're almost certainly going to read your memories into a prompt, so you should describe the form you want, not just the subject. For example, "Two or three sentences of plain prose, second person, no headings or bullet points" gives you something you can drop into context untouched. Or ask for atomic facts instead, and you can get rows you can retrieve one at a time.
A description is applied to one input at a time, alongside whatever related memories the pipeline pulls in for it. That makes it good at judgements it can settle on the spot, but it cannot help with anything that depends on what came before.
Accumulating is a separate step. A pipeline chains extract, transform and commit step by default, and a fourth kind of step, a buffer, can sit anywhere among them. It holds memories or raw inputs back until a trigger fires which could be a count, or a timer. A rule about something building up over time belongs there, not in the wording of your description.
A default project runs extract, transform and commit. Adding a buffer, or reordering the steps around it, is configured per project and is currently available on enterprise plans.
Also, topics are editable in the console at any time, so they can be added, removed, or reworded without recreating the project. This makes it easy to iterate until they work as expected.
Description is one field. The rest are configuration rather than wording, and they carry their own effects. This is the form you see when you add a topic of your own in the console:
| Field | What it decides | If you get it wrong |
|---|---|---|
| Name | how you address the topic in code, topics=["UserProfile"] | - |
| Description | the extraction prompt: what gets pulled out, and how it's written | the topic keeps everything, or nothing you can use |
| User scoped | memories belong to one user_id | uncheck it and every user shares one pool |
| Property scopes | extra partition keys like conversation_id or repo | every write must carry them, or nothing reaches the topic |
| Bounded | at most one memory per scope | leave it off and nothing guarantees a single memory to read back |
The topics docs cover all these concepts in more depth.
Bounded caps how many memories a topic may hold. Scope decides which of them a given read or write request can reach. Both are about what happens when a new message arrives for a topic that already holds memories.
Continuing the 100-line demo example from earlier, let's say four days later I send:
"Update: the demo is 400 lines now. The 100-line rule is officially dead."
Zero memories created, two updated. UserKnowledge memories afterwards:
[UserKnowledge] Prajjwal works as a developer advocate at Weaviate.
[UserKnowledge] Prajjwal writes all his demos in Python.
[UserKnowledge] Prajjwal no longer enforces his previous hard rule of keeping demos
under 100 lines; as of 5 Sep 2026 his demos are 400 lines long.
The memory about the 100-line rule was rewritten in place and the rule is gone. The other two were left alone, since nothing in the new message contradicted them. That is the transform step from the Engram pipeline doing its job, and every topic gets it, bounded or not. Note that deleted was zero, as reconciliation generally supersedes rather than erases. If you want a memory gone explicitly, you delete it with memories.delete().
What bounded adds is a promise about the count. A bounded topic holds at most one memory per unique scope. Engram derives the memory's ID from the topic name and the scope, so every later write lands on that same ID and updates it instead of adding another.
The update above was one run. Send the introduction and then the update message to five fresh users, each starting from an empty store, and count how many memories land:
run 0 run 1 run 2 run 3 run 4
UserKnowledge (unbounded) 2 4 4 3 3
UserProfile (bounded) 1 1 1 1 1
The unbounded topic landed anywhere between two and four memories: sometimes the retraction became one memory, sometimes two, and so on. UserProfile always held exactly one memory, five times out of five, because it is bounded. So if your code needs to read a single standing memory for a topic, you should always bound the topic rather than trusting the count. The template bounds ConversationSummary for the same reason. A conversation should have one running summary that gets rewritten, not a new one per message.
Scope partitions a topic, and there are two kinds. User scope is a hard wall as every write and every read has to carry a user_id. Property scopes like conversation_id or repo are required on writes but optional on reads, so you can read one partition or all of them at once.
That optionality is the reason to use a property. The template scopes ConversationSummary by conversation_id, so each thread keeps its own summary and you can still ask for every summary a user has. If two partitions are never meant to be read together, don't use a property and give them separate user_ids instead. And if a fact should follow the user everywhere, leave it at user scope and add nothing.
A write has to carry every key the topic declares, or it's rejected like this:
insufficient scope: missing required scope properties [conversation_id] to write memories
insufficient scope: missing required user_id to write memories
invalid scope property: [repo] not configured on any topic (configured properties: [conversation_id])
An empty string counts as missing, and a property no topic declares gets an error naming the ones the project has. Reads only insist on user_id. An empty conversation_id is treated as absent and searches every conversation, even though a write would have rejected it, and a conversation_id that was never written simply returns nothing from the conversation-scoped topic.
There are four retrieval modes. vector, bm25 and hybrid all are for search: you give them a query, they score every memory against it, and you get the best ones back in ranked order. hybrid is what runs if you don't pick one, while fetch is designed for direct, non-ranked memory retrieval.
search ranks by relevance to a query. In a chat app, that query is usually the user's current message:
from engram import HybridRetrieval
hits = client.memories.search(
query=user_message,
retrieval_config=HybridRetrieval(limit=3),
user_id=uid, properties=props,
)
context = "\n".join(f"- {m.content}" for m in hits)
Set limit to the number of memories you actually want in the prompt. The default is ten, and you get ten whether or not the tenth has anything to do with the question. Ask for too many, and you pay for irrelevant memories on every turn. Ask for too few, and the agent misses the one fact that mattered.
fetch is closer to a listing operation than a search: name the topic, get its memories back, no ranking involved.
from engram import FetchRetrieval
profile = client.memories.search(
query="unused", # fetch ignores this, but the API insists
retrieval_config=FetchRetrieval(limit=1),
topics=["UserProfile"],
user_id=uid, properties=props,
)
Nothing comes back with a score as the results are not ranked. It suits bounded topics well, as "which memory" has only one answer there.
If you already hold a memory's ID, use get to simply read it back:
memory = client.memories.get(
memory_id,
user_id=uid,
)
So, to read memories from Engram: search when you want what is relevant to a query, fetch when you want everything a topic holds, and get when you need to view a memory by its ID.
The API specifics can change, therefore, always refer to the current definitions in the search memories and manage memories docs. The latter also covers deletion, which is permanent.
The obvious thing to do with search results is to paste them into the system prompt - search memories on every turn, rebuild the system prompt, send. It works fine, and in a short chat you will never notice anything wrong with it.
The bill shows up in long sessions as most LLM providers cache prompts from the front. If a request starts with the same text as an earlier one, that shared opening is read from cache at a fraction of the price. The cached part runs up to a breakpoint. Change anything before that breakpoint, and everything from the change onward is billed in full again.
Memory search results can change every turn, and when you paste them into the system prompt, they sit in front of everything else. So the history behind them is almost never read from cache, and you pay for the whole prompt on every turn.
That means a prompt can have memory in two places:
Engram's two reads line up with them: fetch returns a whole topic without a query, search returns what matches the current message (or query).
A better option is to move the search results to the end of the prompt, after the user's message, in a message of their own. On the next turn, you replace that message with the new search results rather than keeping both. Put the breakpoint on the user message just before it. Now the system prompt and the whole history are read from cache, and you pay in full only for the new messages and the memory block.
The breakpoint is the important part here because if you don't place it, the provider puts it at the end of the last message, which in this layout is the memory block. And if that block is different next turn, the prefix doesn't match. In this situation, GPT-5.6 models end up caching only the system prompt, and Claude models don't get a cache hit at all.
The field for placing the breakpoint also differs by provider:
prompt_cache_breakpoint on the user message's content blockcache_control on the same blockSome memories are needed on every turn, whatever the user asks. Like, in one of our internal agents I built, those were the user's profile and writing preferences. If someone says they write in British English or introduces themselves, every reply in the session should know that, not just the replies where the search happens to bring it back.
Searching for those every turn is wasted work, and it puts text that never changes inside the block that does. The alternative is to fetch them once when the session opens and put the result in a user message right after the system prompt. It stays identical all session, so it is read from cache from the second turn on.
Which topics go in front depends on how often they get rewritten, not on whether they are bounded. In our example, UserProfile rarely changes, so it can sit at the front. ConversationSummary is bounded too, but it is rewritten on every message, so it stays in the per-turn search.
Also, name the topics explicitly in both calls, or the same memory can appear twice. A plain search may return the user’s profile alongside everything else, so in our running example the split would become:
fetch with topics=["UserProfile"] when the session openssearch with topics set to the rest, on every turnThe trade-off here is freshness. Whatever the user says in the current session is in the history anyway, but if another session rewrites one of the fetched memories, the current session won't see the change until the next one opens.
If the whole store is small enough to fit in the prompt, you can also just skip the search entirely and fetch everything at the start. Cached tokens still cost something on every turn, so that only pays off while the store stays relatively small.
These are the four layouts from the test runs behind this post - 25 turns each on gpt-5.6-luna, and what you pay for on every turn after the first:
| Layout | Fully billed every turn |
|---|---|
| memories pasted into the system prompt | the whole prompt |
| memories sent last, no breakpoint placed | everything after the system prompt |
| memories sent last, breakpoint on the user message | the new messages and the memories, including always-on ones |
| always-on topics fetched at start, search results sent last | the new messages and the search results only |
With the always-on topics fetched at the start and the search results sent last, the final request was about 3,500 tokens and only around 100 of them were not read from cache.
So, how much you save depends on how much sits in front of the memory block and how long the session runs. Always measure the efficiency on your own stack and check the cached-token count whenever you touch prompt layout, as nothing errors when caching breaks, the bill just goes up.
The fastest way to fix what your agent remembers isn't more application code. It's opening the topics you clicked past during setup and writing down what you actually want kept - what to leave out, what shape to write it in, and what isn't worth recording at all (unless your setup works just fine using one of our templates).
Then bound the topics that must be singular, scope the ones that need partitioning, set a limit to the number of memories you actually want in the prompt, and keep what you fetched at the front of the prompt and what you searched for at the end. A few minutes on that config screen is usually all it takes! Otherwise you get an agent that remembers your name and not much else.
Get started with Engram in Weaviate Cloud and create your free project today.
For any questions, ideas, or to just chat, feel free to join the conversation on our community forum.
Happy building!
At enterprise scale, even small architecture choices can have outsized consequences. A deployment that works for a handful of teams can become a constraint once thousands of developers, repositories, and pipelines depend on it.
That makes each decision made before rollout especially consequential. For example, your:
Together, those factors determine how well the platform can absorb growth without creating new operational constraints.
In this guide:
Your GitLab deployment model determines which parts of the platform your team must size, secure, monitor, upgrade, and recover. For enterprise deployments, the three core options are:
GitLab.com: GitLab’s multi-tenant software-as-a-service (SaaS) offering. GitLab operates the application and underlying infrastructure, while your organization manages its GitLab configuration, integrations, and any self-managed runners.
GitLab Dedicated: A fully managed, single-tenant SaaS offering hosted on Amazon Web Services (AWS). GitLab operates the underlying infrastructure, including updates, high availability, and disaster recovery; your organization controls user and data access through application-level controls.
GitLab Self-Managed: Your organization installs, administers, and maintains its own GitLab instance. You manage the infrastructure and assume responsibility for operating, scaling, securing, and recovering the environment.
Choose the model based on the control your organization requires and the infrastructure responsibility it can sustain. For example, you may want to choose:
Before you decide, document any requirements that could rule an option in or out. Pay particular attention to data residency, network isolation, recovery objectives, and infrastructure control. Then map the operational work each model leaves with your team, including upgrades, monitoring, capacity planning, backups, and incident response.
That exercise should clarify the central tradeoff: how much infrastructure responsibility your organization needs and can realistically own.
Once you define that operating boundary, it’s time to plan the compute layer that will execute your CI/CD workloads.
GitLab Runner executes CI/CD jobs, and the GitLab application coordinates the pipelines behind them. That separation matters at enterprise scale because application capacity and runner capacity respond to different types of demand. The application handles Git, web, API, and automation traffic; the runner fleet absorbs the volume and concurrency of CI/CD work.
For this reason, you should size the runner fleet based on the workloads themselves rather than on developer headcount. Start by documenting:
Use those inputs to estimate how many jobs must run at once to meet your queued-duration target. Workload data matters more than team size because two organizations with the same number of developers can generate very different CI/CD demand based on pipeline frequency, automation, and job requirements.
Runner scope determines how broadly teams can use each pool. Instance runners can serve projects across the GitLab instance, while group runners limit access to projects and subgroups within a defined group. Project runners provide the narrowest scope and fit workloads that need dedicated credentials, specialized infrastructure, or stronger isolation. Because that capacity is reserved for fewer workloads, project runners may also sit idle when job volume is intermittent, so factor utilization into the decision.
Use the broadest scope that meets the workload’s trust and compute requirements. Broader pools generally improve utilization, while sensitive deployment jobs or specialized workloads may justify dedicated infrastructure.
Autoscaling lets runner capacity expand or contract with demand, but operating that infrastructure also takes platform engineering time. If you manage your own runner fleet, account for how quickly new resources become usable: instance provisioning, cloud quotas, image downloads, and cache availability can all affect queued duration during a spike. Keep enough ready capacity to absorb short-term demand while additional compute comes online.
You can also shift that operational work to GitLab. GitLab-hosted runners are available for GitLab.com and GitLab Dedicated, with GitLab managing the underlying runner infrastructure and autoscaling. For teams that want to reduce the time spent provisioning, patching, and scaling runner machines, that changes the runner strategy from an infrastructure-management decision to more of a capacity and workload-placement decision.
Executor choice determines where CI/CD jobs run and what infrastructure your team must operate. For cloud-native environments, the Kubernetes executor uses an existing Kubernetes cluster. For autoscaled workloads on public-cloud virtual machines, GitLab provides the Docker Autoscaler and Instance executors.
The right choice depends on the environment your jobs need and the infrastructure your team is prepared to manage. With the Kubernetes executor, each CI/CD job runs in its own pod, making cluster behavior part of runner performance. Scheduling delays, resource requests and limits, node capacity, and autoscaling can all affect how quickly jobs start and complete.
Account for those constraints in your capacity plan so the executor does not become a bottleneck as CI/CD demand grows.
Availability planning should begin with the business impact of downtime and data loss. Define:
Together, these targets define the redundancy and recovery capacity the architecture needs.
How much of that work falls to your platform team depends on the deployment model. If you use GitLab Self-Managed, your team owns those architecture decisions. Use the GitLab reference architectures as a production-ready starting point, then adapt the topology to your availability and recovery requirements.
With GitLab Dedicated, GitLab manages the underlying disaster recovery infrastructure and failover process. Customers can choose a secondary AWS region for geo-based disaster recovery, while GitLab maintains replication between the primary and secondary regions and manages failover when required.
For Self-Managed deployments, your recovery design should treat high availability, disaster recovery, and backups as distinct but complementary layers:
In addition, for Self-Managed deployments, turn your recovery targets into architecture requirements. Decide where redundancy is needed, how data will replicate, and how backups will protect critical data. Include dependencies such as identity and networking services in the recovery plan.
GitLab Geo provides an active-passive disaster recovery architecture with secondary sites that synchronize from the primary. For Self-Managed, failover requires customer-managed operational steps, so rehearse the process under realistic conditions and measure the results against your RTO and RPO.
Record any failed dependencies or manual steps that could slow recovery, then use those findings to strengthen the design.
As GitLab adoption grows, performance planning shifts from sizing for expected demand to validating the platform’s behavior under real-world load. The first step is to identify where time is being lost.
GitLab separates queued duration from execution duration, which gives you a useful starting point for diagnosis. A job’s queued duration shows how long it waited to start, while job duration captures execution time. Pipeline duration measures the time spent running the pipeline and excludes pending queue time.
Those metrics point to different constraints. A high queued duration may indicate insufficient runner capacity. Longer execution times, by contrast, can stem from pipeline design, test suites, dependency downloads, or repository transfers.
Test representative projects under both normal and peak demand to see where performance starts to degrade. Include conditions such as release windows, scheduled security scans, and periods of heavy commit activity, then track the signals that surface the bottleneck:
Break the results down by runner pool and workload type so organization-wide averages don’t hide bottlenecks affecting specific teams or workloads. From there, use what you learn to set thresholds for expanding runner capacity, optimizing pipelines, or scaling the GitLab application.
Large repositories and monorepos place distinct demands on GitLab and runner infrastructure. Frequent clones and fetches can increase CPU, memory, disk, and network usage, especially when many pipelines access the same repository simultaneously.
Look beyond repository size when estimating that impact. Clone frequency, concurrent CI/CD activity, branch patterns, and the amount of data each job transfers can all shape platform load.
Pipeline design can reduce demand on the platform itself. Run independent jobs in parallel, avoid unnecessary pipelines, cache frequently downloaded dependencies, and limit artifact retention. For monorepos, trigger jobs only when relevant paths change and reduce the amount of repository data each job needs to transfer.
Continue measuring after rollout as usage evolves. For Self-Managed environments, actual resource utilization and workload patterns provide the clearest signal for when the architecture needs to scale.
Kubernetes can play two different roles in a GitLab architecture. The Kubernetes executor can run CI/CD jobs as pods in an existing cluster, while GitLab Self-Managed can run in a cloud-native architecture on Kubernetes.
These choices affect different parts of the platform and should be evaluated separately. Using Kubernetes for runners changes how CI/CD compute is provisioned and scaled. Running GitLab on Kubernetes changes how your team operates the application and its supporting infrastructure.
With the Kubernetes executor, a runner manager calls the Kubernetes API and creates a pod for each CI/CD job. That makes the cluster itself part of your runner architecture.
Plan the Kubernetes resources and controls those jobs will rely on, including namespaces, service accounts, resource requests and limits, and workload isolation. Sensitive deployment jobs may also require stronger separation from less-trusted build workloads.
Capacity matters just as much as configuration. Test whether cluster autoscaling can add nodes quickly enough to meet your queued-duration targets. Even when the cluster eventually provides enough compute, slow node provisioning can leave jobs waiting during demand spikes.
Running GitLab itself on Kubernetes requires a broader architecture decision. GitLab recommends its Cloud Native reference architecture for new Self-Managed deployments. In this model, GitLab components run in Kubernetes, while PostgreSQL, Redis, and object storage remain external.
Cloud Native Hybrid remains an option when specific components need to stay outside Kubernetes. Teams that require a Gitaly Cluster for repository-level high availability, for example, should evaluate a hybrid or VM-based reference architecture because the standard Cloud Native architecture runs Gitaly in a non-clustered configuration.
Whichever model you choose, include Kubernetes in the operating plan for the wider GitLab platform. Your team will need observability across the cluster and external services, along with an upgrade process that accounts for GitLab and its infrastructure dependencies. Capacity and recovery testing should cover the cluster as part of the production environment.
The GitLab cloud-native overview provides more context on this deployment model. Choose Kubernetes when its operating model fits your infrastructure requirements, and your team has the skills to run it reliably.
Use this checklist before rollout to validate the major architecture decisions across deployment, sizing, runners, recovery, and performance. For each item, document the evidence that supports the decision or assign an owner to close the gap.
| What to validate | Evidence or owner |
|---|---|
| Deployment model | |
| The selected deployment model meets data residency, isolation, networking, and customization requirements. | |
| Responsibilities are clearly divided among GitLab, your platform team, and infrastructure providers. | |
| Upgrades, maintenance, support, and capacity management have named owners and documented procedures. | |
| Application sizing | |
| Expected RPS drives the baseline architecture size for Self-Managed deployments. | |
| Sizing reflects the mix of API, web, and Git traffic. | |
| The design accounts for atypical workloads such as large monorepos or heavy automation. | |
| Runner strategy | |
| Runner sizing reflects job volume, duration, peak concurrency, and compute requirements. | |
| Runner scopes match trust boundaries, privileged access, and workload-isolation requirements. | |
| Autoscaling limits, cloud quotas, startup time, and ready capacity have been tested under peak demand. | |
| Queued-duration and pipeline-duration targets are defined and monitored separately. | |
| Runner-manager architecture avoids a single point of failure for critical workloads. | |
| Availability and recovery | |
| Business and technical owners have approved SLO, RTO, and RPO targets. | |
| Redundancy, backups, replication, and failover procedures address required failure scenarios. | |
| Recovery tests include identity, DNS, secrets, networking, and external integrations. | |
| The latest recovery exercise met its objectives or has assigned remediation work. | |
| Performance and growth | |
| Representative projects, monorepos, security jobs, and release workloads have been tested under expected peak demand. | |
| Dashboards track queued duration, job and pipeline duration, errors, infrastructure saturation, and runner utilization. | |
| Scaling thresholds define when to add capacity or optimize workloads. | |
| The architecture has a defined review cadence for changing usage patterns and organizational requirements. |
Enterprise scale puts every early architecture decision under pressure. The strongest GitLab environments reflect how the organization actually operates and leave enough room for demand to change.
Those conditions will evolve as adoption expands. Keep measuring, revisit the architecture as demand shifts, and let evidence drive the next decision. That discipline turns GitLab from a platform that simply supports more users into one that can keep pace with the organization around it.
GitLab Duo Agent Platform orchestrates and automates complex tasks through agentic flows. A key part of the platform is the Flow Registry, a declarative configuration framework, built from reusable components, that compiles YAML into fully functional LangGraph flows. By using Flow Registry, agent builders — both our GitLab engineers and our customers can use declarative YAML configurations instead of repetitive, ad-hoc Python implementations. Flow Registry turns bespoke state management and agent wiring duplicated across agents into a set of reusable components and primitives available to agent builders.
Using Flow Registry has reduced our own code-per-agentic-flow by 45%. What is this translating to?
All of these benefits are available to customers orchestrating and building agents on GitLab Duo Agent Platform.
In this article, we share the architectural principles and lessons from this effort and how to apply them in your environment.
After releasing GitLab Duo Code Suggestions and Duo Chat, we dug into a then-novel technology, autonomous agents. We researched available AI frameworks and selected LangGraph, an agent runtime and low-level orchestration framework from LangChain, as the foundation for GitLab Duo Agent Platform.
LangGraph's rich feature set, which includes a broad range of model adapters, durable execution, and traceability, combined with an excellent level of engineering autonomy, brought all the necessary building blocks we looked for to start GitLab Duo Agent Platform development.
During the initial months, GitLab engineers, empowered by LangGraph, swiftly built the foundations of Duo Agent Platform, and before long the team shipped four agentic flows:
We also quickly realized a critical gap that low-level frameworks such as LangGraph do not address: a lack of structure to support consistent development at scale.
With just four flows present, and a small engineering team working on GitLab Duo Agent Platform at that time, the codebase was growing rapidly. Every flow was implemented as an ad-hoc directed graph, turning into a web of interconnected nodes and edges. The early Duo Agent Platform codebase had no reusability, no composability, and little in the way of shared standards. It became very difficult to develop new features, and any horizontal platform-wide change seemed like an impossible task.
With every flow taking at least 450 lines of ad-hoc Python code and looking like this example, the team's velocity slowed down as engineers struggled to introduce changes, overwhelmed by complexity and coupling.
The graph's complexity spilled into the test suite, as well. Each test case depended on an execution propagating through a whole graph, which changed tests from a quality assurance safety net into a boogeyman that nobody wanted to look at.
It became clear to us that graphs used as an atomic building block at this low abstraction level are not a good match for a platform implementation. To support the scale we envisioned, it was necessary to introduce smaller units, that break down the complexity and reduce cognitive load put on platform engineers maintaining the project.
Furthermore, graphs with low-level nodes managing model API calls or executing function calls produced by said models, were not the right abstraction for AI engineers either, as they are more accustomed to terms like agents and agent orchestration.
Looking for a way out of that maze, we decided to separate those two concerns — AI engineering from platform development — with the introduction of a new layer of abstraction. To do so, we reviewed existing graphs and identified and extracted repeated structures (for example, cycles going between large language model (LLM) calls and tool execution, implementing agent loops). The refactor brought some relief, as the most complex files had been broken down into smaller pieces that formed the new abstraction layer.
However, the platform was still far from a scalable state. The extracted graph pieces unfortunately operated with their own state structures, tightly coupled with the flow from which they originated. This prevented us from reusing extracted entities between different flows, and we were concerned that at that point every new flow would be more likely to create its own set of pieces, rather than be composed from ones that already existed. The system was neither collaborative nor efficient, and it was not sustainable in that state for a longer period of time.
That realization made it apparent — we had to put more effort in, continue to evolve the architecture, and provide clear development guidelines. At that time we already knew that Duo Agent Platform flows wouldn't be exclusively built by other product teams, but that a wider GitLab community would be invited to contribute as well.
Equipped with the past experience, and inspired by ambitious goals, my teammate Alexander Chueshev and I went back to the drawing board, and rethought the system. We set out to introduce a solution that is highly collaborative, composable, and optimized for AI development efficiency.
We wanted this new iteration to hide low-level LangGraph implementation details, and to stop bothering developers with nodes or edges. The system ought to speak their language — the language of AI engineering — with agents being a central component.
It was clear to us that AI development reasons in terms of agents, rather than nodes that invoke models, execute tools, etc. Drawing lessons from the past iteration, we decided to base the new framework on three pillars:
We were convinced that if we were able to design them well, the new framework would be flexible enough to support any AI flow that users might want to build.
Components are the central and most important pillar of Flow Registry. They model common primitives such as agents, human-in-the-loop checkpoints, and fixed-logic steps. This pillar lifts the abstraction level to match terminology used within the AI engineering domain. Thanks to components agent builders no longer need to reimplement those primitives from scratch, but can declare them with YAML snippets that look like this example:
- type: AgentComponent
name: "developer_agent"
prompt_id: "developer_agent_prompt"
inputs:
- from: "context:goal"
as: "goal"
- from: "context:project_id"
as: "project_id"
toolset:
- "read_file"
- "find_files"
- "edit_file"
- "run_command"
- "create_merge_request"
Under the hood, the AgentComponent is still a piece of a LangGraph’s graph, whose simplified structure is shown in the diagram below. However, now its implementation complexity is hidden from agent builders, who operate with a more familiar primitive. The same architectural boundaries also benefit framework maintainers, giving them more freedom to modify and extend the underlying implementation, with changes propagating to flows transparently.
flowchart LR
%% External input/output
input((inputs<br>from<br>shared state)) --> LLMCall
End --> output((outputs<br>to shared state))
%% Prompts
Prompt["You are expert<br>software<br>engineer ..."] --> LLMCall
subgraph Prompts
direction TB
style Prompts stroke-dasharray: 4 4, stroke:#3CB371
Prompt
end
%% LLM and internal component
LLMCall --> End
LLMCall --> RunTools
RunTools --> LLMCall
subgraph Component
direction LR
LLMCall[LLM Call]
RunTools[Run Tools]
End[END]
end
%% Tools
EditFile --> RunTools
ReadFile --> RunTools
subgraph Tools
direction LR
style Tools stroke-dasharray: 4 4, stroke:#1E90FF
EditFile[Edit file]
ReadFile[Read file]
end
The agent as a component, with the ability to delegate work to subagents, is already a powerful base delivered by Pillar 1 alone. Many contemporary agent platforms consider it a complete and sufficient offering. However, GitLab has larger ambitions for Duo Agent Platform, which Flow Registry realizes with the next two pillars.
Flow Registry Routers enable agent builders to orchestrate multiple specialized agents, or even agentic teams, into a flow to model highly complex business, or software development processes. Even though the largest contemporary models are powerful enough to drive complex assignments on their own, a recent rise in popularity of subagent architecture shows that there are many benefits of assembling multiple agents to collaborate over a single task.
To demonstrate a practical example, let’s take a look at GitLab’s foundational flow: Fix pipeline. This flow is configured with an automated trigger to triage, and fix failing CI pipelines. Because CI pipelines can be very complex, not every failure requires any code change to be resolved, for example sometimes a dependency service might be not responsive, and a plain retry is enough to fix a failure. To acknowledge that dual approach, the flow branches early based on an agent that acts as a judge’s decision. The judge agent's ruling on whether a failure is actionable is then used by Flow Registry Routers to navigate flow execution into the correct branch.
routers:
- from: "fix_pipeline_context"
condition:
input: "context:fix_pipeline_context.final_answer.decision"
routes:
"add_comment": "fix_pipeline_add_comment"
"create_plan": "fix_pipeline_checkout_existing_branch"
"direct_code_suggestions": "fix_pipeline_code_suggestions"
"no_action": "end"
"default_route": "end"
It is true that state-of-the-art models should be able to make similar decisions and act on them simultaneously. However, thanks to multi-agent architecture, agent builders can capitalize on the following benefits:
Pillar 2 gives agent builders a choice: Use a simple flow architecture with powerful models, or offload complexity from models prompts into explicit flow structure — catering to a broad range of possible use cases, cost targets, and risk profiles.
The third and final pillar of Flow Registry is a shared state structure that acts as a communication protocol between components. Without it, the previous two pillars could not function, because components would lack a reliable way to communicate. Referring back to the Fix pipeline example: The judge agent's ruling would be of little value if it could not be reliably forwarded to a Flow Registry Router. More broadly, data produced by one agent is often required by subsequent ones, making a well-defined communication contract essential.
Flow Registry state structure includes a special catchall attribute called context, which behaves like a nested key-value store (or a JSON object) granting components a versatile storage space. To further complement context attribute flexibility, Flow Registry introduced a dot-notation declarative access to context, a convention familiar from other domains (such as GitLab CI Functions), where access to shared key-value storage must be expressed within static configurations.
To complete the third pillar, convention is required: Flow Registry supports flexible read operations from shared state via said dot-notation, however all writes follow strict rules, providing a set of stable, predictable outputs on which agent builders can rely. To see that in practice, let’s take a look again at a piece of Flow Registry config for another foundational flow: Code review.
# …
# Step 5: Fetch lightweight MR metadata (file paths + custom instructions)
- name: "fetch_mr_metadata"
type: DeterministicStepComponent
tool_name: "build_review_merge_request_context"
# ….
# Step 6: Transform prescan results into structured JSON for review consumption
- name: "analyze_prescan_results"
type: AgentComponent
prompt_id: "analyze_prescan_codebase_results"
prompt_version: "^1.0.0"
inputs:
- from: "context:fetch_mr_metadata.tool_responses"
as: "mr_context"
- from: "context:prescan_codebase.tool_responses"
as: "prescan_tool_responses"
optional: True
toolset: []
ui_log_events:
- "on_agent_final_answer"
# …..
routers:
# ….
- from: "fetch_mr_metadata"
to: "analyze_prescan_results"
Code Review’s agent analyze_prescan_results requires data pulled by a preceding fixed step action fetch_mr_metadata, that dependency is expressed via inputs declared for analyze_prescan_results agent
inputs:
- from: "context:fetch_mr_metadata.tool_responses"
as: "mr_context"
Here, dot-notation and strict output conventions work in tandem, giving agent builders a stable and predictable protocol for moving data between components within a flow.
Even though Flow Registry uses declarative YAML configurations, we started the design and rearchitecture in Python, and deferred any declarative configuration API to future iterations. However, once all three Flow Registry pillars came together within a single Python block, it became obvious to us that converting those declarations into a YAML config was just a step away, so we took it.
That change completely decoupled the Flow Registry framework from Python and LangGraph, offering a high-level abstraction syntax for declarative AI flow creation. It established a clean boundary between the platform still implemented on LangGraph foundations, and the external framework's declarative interface, which enabled AI engineers to operate with concepts more familiar to them.
Introduction of Flow Registry framework as a basis for GitLab Duo Agent Platform propelled the whole system from vanilla LangGraph per-use-case implementations into declarative YAML configs like this one behind GitLab Duo Developer Flow, which is currently operating in production.
With the platform decoupled from the framework API designed for AI engineers, the underlying Python codebase becomes shareable across all flows, and any improvements introduced to the engine itself are brought to all flows, further emphasizing the efficiency gains from the clear separation.
In addition, the per-flow code cost drops with every new flow added. At the time of writing, the ratio of Python source code per flow has been reduced by 45% in favor of Flow Registry — and it will keep improving with every new flow being built.
Finally, AI engineers and domain experts are no longer required to understand any of the underlying platform implementation details, nor do they need to implement any repetitive boilerplate Python code that would require its own test suite and maintenance. This was proven by almost 7,000 developers who signed up for the GitLab AI Hackathon earlier this year and submitted 600+ agents and flows.
A key observation we made is that modern AI engineering is still a very young branch of software development, in which common architectural patterns and paradigms haven’t fully been formed yet. However, it does not mean that already established good software engineering practices can’t be applied to AI engineering. In fact, as Flow Registry's story shows, reaching back to existing software engineering paradigms and practices, such as identifying repeated code, extracting it into named entities with clear roles within a system, and forming abstraction layers from them, can yield powerful results.
Beyond that, we would also like to share a few other takeaways that apply broadly to any team building agentic systems, while others are practical starting points for teams working with low-level frameworks like LangGraph.
To get a feel for how it all works in practice, visit the AI Catalog where GitLab exposes the resulting Flow Registry orchestration framework for anyone to build custom flows.
You can also try GitLab Duo Agent Platform for free.
Databases use indexes to make queries fast. Rather than check every row in a table for a matching value, look up that value in an index and seek directly to the right rows.
Most database indexes are b-trees or b+trees, which sort a column's values in a total order and can quickly find matching values by exact value, a range of values, or a prefix. So, for example, a b-tree can quickly find all users with the name Rick, all products with a price below $5, or all repositories with a creation date in November of a given year. But a b-tree is useless for matching content in the middle of a string. That's where full-text search indexes come in.
Do not do this, which checks every name in your users table:
SELECT * FROM users
WHERE name LIKE '% Royal';
And especially do not do this, which checks every description in your vendors table three times:
SELECT * FROM vendors
WHERE description LIKE '%postgres%'
AND description LIKE '%reliable%'
AND description LIKE '%fast%';
The right tool for that job is an inverted index, which is how essentially all full-text search engines locate documents quickly by the words they contain.
Note
PlanetScale TIN is a comprehensive full-text search index for Postgres. We have a deep dive on TIN's features, performance, and implementation in another article. Anyone who needs a general refresher on full-text search indexes should continue here.
An inverted index is a data structure, usually on disk, that maps terms to locations. It works like the index at the back of a book. It even works a lot like a b-tree, except that instead of being keyed by a text column's entire value, an inverted index is keyed by each individual word within a text field.
Quick aside: then why is it "inverted?" It's inverted relative to the text itself, not relative to other indexes. The text is a series of implicit locations, each with a word. An inverted index is a list of words, each with one or more locations where it can be found.
Back to the structure of the thing. At a minimum, an inverted index has a term dictionary and postings lists. Depending on its feature set, it may also have positional data and frequency data.
Try some searches here, and see how your queries compute either the union or the intersection of the document IDs in postings lists in the index.
The term dictionary maps all the terms found across all your documents to postings lists. On disk, this is often a b-tree or some other lexicographically sorted structure. When a search query arrives, it looks up all the query's terms in the term dictionary.
Queries with wildcards and fuzzy matches may scan part or all of the term dictionary looking for appropriate exact terms. For example, g* would look up both gonna and give, while u~2 would look up all terms within two character edits of u: the terms you and up. Try it in the figure above!
Each term in the term dictionary points to a list of locations called a postings list. These locations are document identifiers of some kind, enough for the database or search engine to find the document in a table or file storage. In most inverted indexes, these are sequential numbers; an index with ten documents uses the numbers zero through nine (or one through ten). However, any unambiguous identifier will do. Since a database needs to map values back to a row rather than to the nth document added to an inverted index, the inverted index either needs to store an additional map of document IDs to rows, or it needs to store the row identifiers directly in the postings list.
Postings lists are almost always sorted, then compressed in some way. If all the document IDs are under 256, they would be stored with at most eight bits each. If they are dense (i.e., many documents contain a given term), then the postings list may store only the differences between the numbers: id1, id2-id1, id3-id2, and so forth. These differences are always smaller than the IDs themselves, so they can be stored with fewer bits. Consider, for example, an index with 300,000 documents, of which 100,000 contain the word "who." Storing each ID literally would take lg(n) = 19 bits per posting. But the average gap between any two successive IDs (remember, the postings list is sorted) is just three, which can be stored in two bits.
If the postings are very dense, the postings list may store a bitmap. If document n contains a word, then the nth bit in the bitmap is one; otherwise, it's zero. Such an encoding takes exactly n bits for n documents and is optimal once around half of all documents contain a given term.
Taking the union or intersection of two postings lists that are sorted is fast, because it can be done in a single, O(n) pass. Taking the union or intersection of two postings lists stored as bitmaps is extremely fast, because recent CPUs with vector instructions can OR or AND 128, 256, or even 512 bits in a single instruction.
Some search engines support span queries and phrase queries. A span query is a query that requires terms to be in a specific part of the document or requires them to be within a specific maximum distance of each other. For example, rules IN FIRST 5% or gotta NEAR/3 understand. A phrase query is a special case of a span query that requires words to appear in exact sequence.
To support span and phrase queries, an inverted index can store positional data. The postings list identifies which documents each term appears in; positional data identifies where in the documents the term appears.
Because most queries aren't span or positional queries, positional data is usually stored separately from the postings lists so it can be loaded only when needed.
Some search engines support scoring and ranking documents. One common scoring method is BM25 (more on that below), which needs to know some statistics about the indexed documents: the length of each document, the length of the average document, how often each term t appears in each document d, and how many total documents contain the term t. The index precomputes all of this data so it can be fetched quickly when scoring results for each query.
The set of all indexed documents is called the corpus. The size of the index is usually some factor of the size of the corpus, with that factor depending on what features the index supports. An index with no positional or score data might be roughly 20% of the size of the corpus. A code-search index with overlapping tokens (to support exact-match and regular-expression search) and positional data could be as much as 300% of the size of the corpus. English-language text indexes with positional data and frequency statistics often are 30-50% the size of the corpus. Those numbers can vary depending on factors like document length and vocabulary size, but they give you an idea of the relative cost of implementing different features.
Each indexed document is a long string of text. Breaking that text into terms is called tokenizing it. Different use cases, especially different languages, require different tokenizers. Is you're one token or two? Are there any tokens at all in {[] => []}?
Unicode defines a good default set of rules to identify word boundaries.
After tokenization, a search engine may modify or omit terms before adding them to the inverted index. Eliminating linguistic suffixes is called stemming. For example, mapping strangers to stranger or mapping thinking to think allows a user to search for a word but match documents containing any form of that word.
Words so common they're not (usually) useful for searches are called stop words, and many search engines do not index them at all. However, an index that considers new a stop word can't distinguish between documents containing York or New York. An index that drops both the and who can't search for The Who at all. Stop words are a trade-off: the index is smaller and faster, but it's less precise in cases where those words matter.
Some search use cases require retrieving a few "best" results, rather than all matching results. This is in contrast to SQL, where SELECT <something> LIMIT 10 is allowed to return any ten rows available. One of many ways to determine the best results is to score them with BM25. BM25 is a formula that scores how good a given document is as an answer for all the terms in a given query. It multiplies a function capturing term frequency (how many times a term appears in the given document, relative to that document's length) with a function capturing the inverse document frequency (what fraction of documents contain the term at least once), then sums up that subscore across all the terms in the query. BM25 captures the intuition that rarer terms are more significant, and a document that mentions a given term lots of times is a better result for that term.
Scoring is almost always used in conjunction with a desired number of results, like SQL's LIMIT 10. That allows an important optimization.
Postings lists can be broken up into blocks, with frequency statistics for each block. When an index looks for the k documents with the highest score, known as a top-k query, it may be able to skip whole blocks of postings if the statistics for those blocks indicate that none of the documents in the block would produce a higher score than the best k documents the index has already found. This significantly speeds up top-k queries, especially for small values of k.
The simplest way to build an inverted index is all at once, in one shot. A small enough index can be built entirely in memory; after all, an inverted index is little more than a Map<String, Vector<ID>>. Indexes larger than memory are written piece by piece. The index creation process runs until memory is full, then dumps a self-contained inverted index to disk representing the first n documents. Then it repeats until memory is full again, dumps that self-contained inverted index to disk for the next n documents, and so on. Each self-contained inverted index is called a segment. When processing a query, the overall index must check for matches in all segments and then merge the results.
Of course, many data sets change over time. A corpus can grow as more documents are added. New documents can be batched in memory until there are enough to form a segment, or each can be added immediately to a mutable data structure on disk. That mutable storage has a less efficient layout than an immutable segment, but it has the advantage of being, well, mutable: documents can be added efficiently without knowing all of them in advance. Then the documents in mutable storage are searched alongside all the immutable segments in each query.
TIN uses a mutable segment containing postings lists, like a less efficient version of the immutable segments. Because the mutable segment is slow, it must eventually be sealed and converted to an immutable segment.
Try some inserts, updates, and deletes in the example segments below. For simplicity, the example shows at most two immutable segments. When the mutable segment reaches its size limit, the example immediately merges it into whichever immutable segment is smaller. In practice, a mutable segment that gets sealed would exist for a while as a small, standalone immutable segment.
Eventually, there will be too many segments. An index that searches hundreds or thousands of small segments will spend some amount of CPU and memory just tracking all the segments and merging the query results. So, inverted indexes normally need to merge segments. In a merge, two or more segments become a single, larger one. If the document IDs are sequential numbers local to each segment, they all must be reassigned; document i from one segment must not be confused with document i from another.
Merging is algorithmically straightforward; it's just a linear pass through the term dictionary and each postings list. But a merge requires a lot of disk space and a lot of I/O. Merging takes two or more immutable segments as input and produces a new output segment equal to the size of all the input segments, minus the postings for any documents that have been deleted. Segments can easily be many gigabytes each, so a merge process might read and write tens of gigabytes and consume, temporarily, that much extra disk space. Segment merging is always a trade-off between the I/O cost of merging and the efficiency penalty of keeping a larger number of segments around.
But that's just how we insert documents. What about deleting them? It's impractical to delete document IDs from the middle of a postings list, which would require recompressing part or all of the list. So deletion just creates a tombstone. A tombstone is an entry in a table indicating that a given document ID is no longer valid. The inverted index will still produce that document ID as a query result, but it checks each result against the tombstones and will remove that ID before returning it to the caller.
That creates another trade-off. Every deleted document still takes up disk space in the index and wastes CPU time retrieving it from a postings list and filtering it out of the list of results. But the only way to get rid of tombstones is to merge segments and filter the document IDs written to the output segment against the tombstone list. Sometimes, when a large fraction (nearing half) of the documents in a segment have been deleted, it may be worth rewriting that segment by itself, just to get rid of the deleted documents.
Updating a document is nothing more than deleting (tombstoning) its old version and inserting its new one. Immutable segments as described here can't do in-place updates.
Finally, we come to how inverted indexes work to provide full-text search in Postgres. How can we make this work?
CREATE INDEX ON songs USING tin(lyrics);
SELECT title, performer
FROM songs
WHERE lyrics ==> 'make you cry'
ORDER BY tin.score(ctid) DESC
LIMIT 10;
Each document is a row's value for a single text column. So if an index is on the column lyrics in the table songs, then each song's lyrics would be a single document. The inverted index has to provide a row identifier, the ctid, to tell the Postgres executor which ten rows to fetch title and performer from.
The whole inverted index needs to be stored somewhere, preferably as a WAL-logged index relation so Postgres replication and backups include it. When segments get deleted, their old storage is freed, but space in the middle of a relation can't be returned to the OS. Instead, the index must maintain a list of freed pages so it can reuse them later.
Merges are often triggered when a segment crosses some size threshold or its tombstone list crosses a threshold fraction of the total documents. But we'd really prefer not to perform a merge (remember: tens of gigabytes read and written, possibly several minutes to execute) inline in an INSERT, UPDATE, or DELETE. So we need a job queue and background maintenance workers.
Because it's Postgres, VACUUM needs to work. VACUUM removes dead row versions from the heap and asks each index to remove entries that point to them. VACUUM FULL rewrites all the rows in a table, so the ctids change. If the inverted index stores ctids, it needs to update them.
Document insertions and deletions must be associated with a transaction number so they aren't visible to other transactions until committed, and so that they can be rolled back. The index must return all the results that are visible and none that aren't. Even in the face of that requirement, a top-k query must actually return k rows. To make COUNT(*) and top-k queries work efficiently, the inverted index needs access to row-level visibility and page-level visibility maps.
Queries often combine inverted full-text constraints like lyrics ==> 'make you cry' with traditional SQL constraints like year = 1987. The query planner needs to know when to use a full-text index, when to use a b-tree, and when to use both and combine the results with a bitmap intersection. A query with full-text constraints on multiple columns, like title and lyrics, should use a CustomScan to filter both entirely within the index implementation, because that's much faster than returning a result set for each column and letting Postgres take the intersection.
For details on how we solved all these challenges and how well it worked, go read the deep dive on TIN, PlanetScale's new full-text search extension for Postgres.
Observability investigations rarely follow a straight line. A latency question might cause an AI agent to start with a metric, pivot into traces, compare a deployment window, and finish by reducing thousands of logs to a few patterns. Each individual query is easy, but propagating context throughout an entire investigation can be tricky and expensive.
With conventional MCP tools, each step becomes another exchange with the model: choose a tool, inspect its response, decide what to call next, and pull the new result into the conversation. That process works well for a focused lookup, but it can be inefficient in a multisignal investigation. The model ends up spending context on tool schemas, raw responses, and the intermediate steps between calls rather than focusing on outcomes.
Datadog Code Execution, generally available, gives AI agents a programmable way to investigate observability data through the Datadog MCP Server. From a sandboxed JavaScript environment, an agent can query several Datadog APIs, run independent work in parallel, branch on results, join data, and return only the evidence needed for the answer. By returning a more focused set of evidence to the model, Code Execution can improve answer accuracy while reducing the cost of running AI agents.
In this post, we’ll show how you can:
The Datadog MCP Server gives AI agents access to tools for querying logs, metrics, traces, monitors, dashboards, and other Datadog data. Traditional MCP tools provide the agent’s underlying model with clear, bounded actions and remain the shortest path for a focused question. For an investigation that crosses several data sources, however, an agent might need to call multiple tools and pass each result back through the model before deciding what to do next. Code Execution moves that intermediate work into code.
Code Execution combines a small MCP interface with a programmable execution environment. The agent interacts with Code Execution through two MCP tools, execute_code and search_datadog_sdk, to explore and query Datadog across its entire API surface. Within the execution environment, both control flow and the intermediate data remain in code instead of passing through the conversation one tool call at a time.
Inside the sandbox, the agent can run independent queries together, use one result to shape the next query, normalize responses from different APIs, and join them on a shared field. It can also filter or aggregate large responses before returning any results to the conversation. The available API operations are based on the Datadog TypeScript client SDK, so generated code uses the same clients and request shapes as other Datadog integrations. Each API operation stays individually typed and subject to its required permissions. The agent composes them by using ordinary control flow.
For example, an agent can generate and run the following script to query logs and spans for errors over the same 1-hour window. The code runs the two queries in parallel, groups the results by service, joins them, and returns only the services that appear in both result sets:
import { client, v2 } from "@datadog/datadog-api-client";
// Create clients for the two Datadog APIs we want to compare.
// Authentication stays outside the sandbox; the MCP service applies the caller's permissions.
const config = client.createConfiguration();
const logs = new v2.LogsApi(config);
const spans = new v2.SpansApi(config);
// Give both queries the same one-hour window so their counts are comparable.
const from = dd.time.hoursAgo(1);
const to = dd.time.now();
// Find the services producing the most errors in logs and traces. The queries
// run together, so neither result needs a separate round-trip through the model.
const [logErrors, spanErrors] = await Promise.all([
logs.aggregateLogs({ body: {
filter: { query: "status:error", from, to },
compute: [{ aggregation: "count" }],
groupBy: [{
facet: "service",
limit: 25,
sort: { aggregation: "count", order: "desc", type: "measure" },
}]
}}),
spans.aggregateSpans({ body: {
data: { type: "aggregate_request", attributes: {
filter: { query: "status:error", from, to },
compute: [{ aggregation: "count", metric: "*" }],
group_by: [{
facet: "service",
limit: 25,
sort: {
aggregation: "count",
metric: "*",
order: "desc",
type: "measure",
},
}]
}}
}}),
]);
// Index the log counts by service, then join them to the span results.
const logsByService = new Map(
(logErrors.data?.buckets ?? []).map(
bucket => [bucket.by?.service, bucket.computes?.c0]
)
);
// Return only the overlap instead of sending both raw responses to the model.
return (spanErrors.data ?? [])
.map(bucket => ({
service: bucket.attributes?.by?.service,
errorSpans: bucket.attributes?.compute?.c0,
errorLogs: logsByService.get(bucket.attributes?.by?.service) ?? 0,
}))
.filter(service => service.errorLogs > 0)
.slice(0, 5);
Each API in this example can return its top 25 services, but only the five services that appear in both result sets cross back into the model’s context. Without Code Execution, the model would have to receive both result sets and perform that join in the conversation.
To measure how Code Execution affects investigation quality and cost, we compared it with Datadog’s Core toolset. The comparison covered 25 observability tasks across metrics, logs, traces, Datadog Error Tracking, and investigations that crossed more than one data source. We ran each task three times with GPT-5.6 Terra, GPT-5.6 Sol, Claude Sonnet 5, and Claude Opus 4.8, and then we scored the final answers for correctness.
Code Execution improved answer correctness with every model we tested, with gains ranging from 7.7 percentage points (pp) to 21.8 pp:
| Model | Core toolset | Code Execution toolset | Change |
|---|---|---|---|
| GPT-5.6 Terra | 77.6% | 85.3% | +7.7 pp |
| GPT-5.6 Sol | 74.4% | 94.0% | +19.6 pp |
| Claude Sonnet 5 | 66.7% | 88.5% | +21.8 pp |
| Claude Opus 4.8 | 77.5% | 90.6% | +13.1 pp |
Because model costs depend on token usage, reducing the amount of context sent to a model can lower the cost of running an investigation. The averaged results across the four models showed that Code Execution used 73.2% fewer input tokens and 39.6% fewer tool calls:
| Category | Core toolset | Code Execution toolset | Change |
|---|---|---|---|
| Answer correctness | 74.1% | 89.6% | +15.6 pp |
| Input tokens | 159.4k | 42.8k | -73.2% |
| Tool calls | 4.08 | 2.47 | -39.6% |
Note: Values in the Core toolset and Code Execution toolset columns are rounded. Values in the Change column are calculated from the unrounded values.
Letting a model generate code against production observability data requires a clear security boundary. Code Execution keeps execution and authentication on opposite sides of that boundary.
Generated JavaScript code runs in an isolated sandbox without access to the caller’s credentials. When the code calls a dd.* method, the trusted MCP service makes the request on the caller’s behalf, enforces their existing Datadog permissions and Code Execution policies, sanitizes the response, and returns it to the sandbox. The code can work with the resulting data, but it never handles the credentials that are used to retrieve it.
To get started with Code Execution, connect the Datadog MCP Server to your AI client and enable the code-exec toolset.
When Code Execution is enabled, ask your agent a question that requires it to correlate multiple kinds of observability data. For example: “Find the services whose error rate changed after last night’s deployments, then show me the trace patterns that changed with them.” The agent can use Code Execution to gather the relevant data, correlate it, and return the evidence behind its answer.
Code Execution helps AI agents use the Datadog MCP Server to carry out multistep observability investigations while keeping intermediate logic and data inside a sandbox. By reducing the amount of intermediate context and the number of tool calls that pass through the model, Code Execution can lower the cost of running AI agents while helping them produce more accurate answers. To learn more, see the Code Execution documentation, the toolset configuration guide, and the MCP Server documentation.
If you’re new to Datadog, you can sign up for a 14-day free trial to try Code Execution for your investigations.
We’re moving our mobile apps from React Native back to native Swift and Kotlin. We’ve already done it with Shop, rebuilding and publishing the app in just 12 weeks. Now we’re applying what we’ve learned to the Shopify App, our largest app with more than 300 screens.
We’re using LLMs to rebuild it because they are really capable now; but getting consistent, high-quality, and maintainable results out of the box is difficult. They need tooling and guardrails. That’s why we built Helix.
Helix is a set of tools and skills that help LLMs migrate features and screens from the React Native app while following a highly opinionated architecture we designed for the new native apps.
Most tools try to gather as much information as possible, turn it into specs and task files, implement the whole thing, and hope the first result works. The engineer gets a huge chunk of code with everything left to test.
Helix takes a different approach. It doesn't expect the first output to be correct. It breaks the work down, learns from the engineer as it goes, and automates more of the task with every step it gets right. The goal is to accelerate the engineer to speeds that were previously impossible while maintaining high-quality results.
So, why does this work? Helix builds a loop where an imperfect attempt cannot move forward until it becomes a good result.
A migration works like this:
Helix rebuilding a screen in native as four checkpoints
There are two ideas that make all of this work: checkpoints that are small enough to review at a glance and gates that are strict enough to stop anything unproven. This is backed by an opinionated architecture that we have thoroughly documented so reviewers have a standard to enforce. Let's look at both in detail.
A Helix migration starts with the existing React Native code and the running app. The engineer picks the target—a whole screen or a single subscreen—and Helix breaks it into checkpoints of increasing complexity. The first checkpoint is usually the screen skeleton; the second is one deliberately small section. Later checkpoints grow only after the early decisions have passed review.
Each checkpoint is described in a few words, and this is deliberate. At this stage, the engineer only needs to check whether the sequence makes sense. Nobody can effectively review a wall of generated text. We'd rather give someone one decision they can make as opposed to ten pages they will skim.
Small checkpoints also fit in a small context window, allowing the agent to read the relevant part of the reference directly instead of relying on a huge spec file or task list to represent the code. The reference is the spec.
Behind the scenes, a subagent reads the reference code and generates test cases for each checkpoint. The test cases operate as integration tests, describing and testing the feature from the user’s perspective. These tests are an opportunity to direct the agent to dig deeper into the feature, finding edge cases outside the happy path. This ensures that each checkpoint is thoroughly reviewed.
Each checkpoint has to meet our quality standards before the agent can move on. Helix enforces those standards by making sure each checkpoint goes through four gates, in order.
If a gate fails, the agent uses the feedback to fix the implementation, then runs the check again. It can retry as many times as it needs to, but it can’t override a failed check just because it thinks the result is good enough.
Our CLI exposes the same screen state and actions as the app. For example, the home screen might expose analytics information and actions for navigating to other parts of the app. The agent analyzes how the reference app works, replicates it, and validates the functionality through CLI behavior tests. The test cases generated for the checkpoint define what "proven" means, and every relevant case has to pass.
The CLI doesn't need a simulator, which makes this loop fast. The agent can iterate on behavior dozens of times before taking a single screenshot.
Helix testing behavior using CLI tests
This is the most interesting part of Helix and the main reason the output lands so close to 1:1.
UI equivalence is almost impossible to specify. A human immediately notices when a title is too small, a divider is too dark, or an icon is slightly off, but these details almost never make it into a prompt. Pixel diffing doesn't work either because two UI frameworks don't produce byte-identical output.
While we were teaching agents to drive simulators, we found that current Gemini models have very good spatial awareness for this exact problem. They can catch multiple UI nuances like margin / padding issues and estimate the difference. So we built a gate around it. The orchestrator (GPT) captures the implementation and reference screenshots in matching states and asks Gemini to act as a perfectionist design reviewer, checking details like structure, spacing, and alignment. It judges sizes proportionally against each screenshot's dimensions, so undersized / oversized text can also be detected.
Gemini must list every difference it finds, each with a severity and an on-screen location. If a visual difference can be fixed in code, the gate treats it as a blocker by default.
Gemini can also mark a comparison as INVALID if the orchestrator sends screenshots of different sections or states. For example, one screenshot might show an unfulfilled order and the other a fulfilled order. The orchestrator then captures both apps in the same state and runs the comparison again.
The orchestrator limits each comparison to what the checkpoint has built. For a skeleton checkpoint, it might ask the reviewer to check only the navigation bar and title because the reference has a full screen and the new app doesn't yet. The scope grows with every checkpoint.
The first rendering doesn't need to be perfect. The system can see what is wrong, describe it, locate it, and require another attempt. This is much more reliable than trying to specify every visual detail before implementation begins.
Let's assume the agent gets everything working and looking almost pixel-perfect. The code underneath could still be poor, and this gate exists to catch that.
We invested in an architecture that is easy for agents to implement, and we documented it thoroughly. This documentation makes adversarial review enforceable. Two independent, context-isolated reviewer agents check the new code against it, including the UI code, which has its own guidelines. Every finding has to be fixed. The affected tests run again after the fixes, and the UI review gate also runs again if anything visibly changed.
The reviewers then examine the changed code again. The loop repeats until both reviewers approve.
By the time a checkpoint reaches an engineer, it’s already in good shape: the UI is nearly 1:1 with the reference, the code follows the guidelines, and the behavior is proven by tests. The loop forces it to reach this point.
The engineer looks at the code and running app and decides whether the result matches their expectations. Their feedback goes to two places: the agent addresses it and re-runs the gates, and Helix records it in memory to improve every checkpoint that follows.
This memory allows autonomy to grow during a migration. Early checkpoints get more engineer attention because uncertainty is high and there’s little accepted work to learn from. As approved code and feedback accumulate, later checkpoints can run with less oversight, and some can skip approval entirely if an engineer chooses autonomous mode.
Every checkpoint ends in a commit, and most engineers start creating branches and raising PRs from there.
This makes Helix better for engineers. One-shot tools put all the work at the end: an engineer has to review one large, uncertain diff across product behavior, visual fidelity, two platforms, and architecture. Helix moves feedback to the earliest useful point. The agent handles repeated implementation, runs the checks, and responds to reviewers. The engineer can focus on scope, product judgment, and taste while steering small changes that have been validated before becoming the foundation for the next one.
While engineer approval is mandatory by default, this can be changed. Helix can be asked to complete the next three checkpoints in one go, or skip approvals entirely, and it will keep working for hours or overnight. And the gates don't become more lax when nobody is watching. Every checkpoint still has to prove its behavior, pass the UI review, and satisfy both adversarial reviewers before the next one begins.
Migrations can also run in parallel. Helix isn't limited to one screen at a time, so several screens can be in progress and converge independently through their own gates.
After an autonomous run, Helix provides a series of committed checkpoints for review instead of one huge diff. Each checkpoint comes with evidence: archived UI reviews, passing tests, and reviewer verdicts.
Nothing in this loop is specific to migrations. For a new feature, Helix can use designs and product docs as its reference and follow the same process. It can also handle architecture migrations and refactors using the same checkpoint-and-gate strategy. A logic-only change simply skips the UI review gate.
This is the real lesson from Helix. We stopped optimizing for a perfect first attempt and started working towards reliable convergence. An attempt is allowed to be wrong. It is not allowed to ship until it isn't.
We’ll continue to share what we learn as we move our apps from React Native to native. If you want to help build the next generation of Shopify’s mobile apps, we’re hiring mobile engineers, infrastructure engineers, and developers working at the intersection of AI and software engineering.
When you ship an AI agent, the key question is whether it can execute a chain of work across dozens of sequential tool calls against a live environment, and recover when a step fails. Scoring whether the model sounds right tells you almost nothing about whether the work finished.
That gap is why agent evaluation has had to evolve from scoring a single function call to scoring an entire task, with tool calling as the connective tissue underneath. This post traces that arc and explains why nearly every serious agent benchmark now rests on tool use.
The original harnesses were built for static tasks. The first model-agnostic, open-source harness decoupled the model from the evaluation protocol.
Agents broke this assumption. Operating across multi-step tasks, an agent calls tools, handles errors, and observes results over many steps, making a single output string insufficient. The Berkeley Function-Calling Leaderboard (BFCL) emerged to evaluate function selection and argument accuracy across single- and multi-turn scenarios. However, BFCL only evaluates individual calls—a valid issue_refund call still fails if underlying checks or updates were skipped. Call accuracy is necessary, but not sufficient.
Full agentic evaluation now requires a full execution environment: one that executes each tool call, tracks state across steps, and reads the world afterward to decide whether the work got done.
Two scoring layers sit on top of it:
Step-level tells you where the chain breaks, which is what you want when debugging or targeting fine-tuning effort; E2E collapses a failure on step one and a failure on step nine into the same “task failed.” E2E is what your users actually experience, which is why most production evals gate the release on it and keep step-level tracing underneath for debugging.
Those two scores are two readings of one object: the trace. A trace is the ordered log of a single attempt: the user message, each step, and the environment state when the attempt stops. Process scoring grades the rows. E2E scoring grades the final state.
A tool-calling benchmark scores three things in order: deciding to use a tool, selecting the right one, and populating its arguments. A model that reaches for a tool when a direct answer would fail as surely as one that skips a tool it needed. Cost and latency ride on top, set by the call’s verbosity and runtime.
Every run rolls up through a fixed hierarchy: Benchmark → Trial → Task → Turn → Step:
A step is usually a tool call, and every score above it rolls up from those steps. The metrics worth tracking collapse onto three axes: accuracy, verbosity, cost (see Table 1, below).
| Metric | Formula | Axis | Why it exists |
|---|---|---|---|
| Task success rate | successful_tasks / tasks | Accuracy | The release gate. Did the environment reach the goal state? |
| Consistency | range of success rate across 3–5 trials | Accuracy | A 90% / 74% split isn’t 84%. Report 82–88%, not a point estimate. |
| Tool-call precision | correct_calls / calls_issued | Accuracy | Hallucinated names and extra calls surface here, not in success rate. |
| Argument accuracy | correct_args / calls_with_right_tool | Accuracy | Separates “wrong API” from “right API, filled wrong.” |
| Steps per success | steps / successful_tasks | Verbosity | How long the trajectory runs when the task actually finishes. |
| Cost per success | spend / successful_tasks | Cost | The economic unit. Tokens and GPU-seconds only matter per successful task. |
The pairings matter: success rate without consistency is a point estimate on a stochastic system (a model that hits 90% then 74% is a worse bet than one holding 84%); tool-call precision without argument accuracy hides slot-filling failures.
Step count is often the axis that varies most across models on the same task — four steps versus fifteen — though on suites like Terminal-Bench 2.0 steps-per-turn varies too, so which axis moves most is benchmark-dependent. Parallel tool calling cuts step count and latency, but not call count: a one-step turn firing four tools still issued four calls. Roll up in order; don’t average steps and call it a benchmark score.
Two benchmarks can both claim to test tool calling and produce numbers that aren’t comparable. Three dimensions explain most of the gap:
Contamination now extends beyond training data leaks to live variants: web-searching agents retrieving answer keys during evaluation, and datasets on Hugging Face quickly re-scraped into pretraining corpora. Private domain evals solve this by being unable to scrape.
Table 2, below, is a public trace from a real benchmark run using step-level and E2E scoring, where the suite rather than an artificial ticket provides the tools, user, and completion criteria.
pytest-dev__pytest-5262 (trial .2, turns 0–4)/testbed) so that the requirements specified in the issue are satisfied” — the issue: _pytest.capture.EncodedFile reports mode rb+ (binary) from its underlying buffer, but its write() only accepts str, so external code that checks .mode (e.g. youtube-dl) crashes when it writes bytes.terminal, file_editor, task_tracker, finish; parallel tool-calling off (one tool call per turn); repo state persists turn to turn (real filesystem + git, not a mock).| Step | Turn | Call | Environment observation | Verdict | Why (valid / useful / redundant / recovered / policy) |
|---|---|---|---|---|---|
| 1 | 0 | terminal(find /testbed -name "capture.py") | Returns /testbed/src/_pytest/capture.py | valid | Locates the file named in the issue before editing anything |
| 2 | 1 | file_editor(view, capture.py) | Dumps the full file (400+ lines) | redundant | File is large; grepping for the class first would have been more targeted |
| 3 | 2 | terminal(grep -n "EncodedFile" capture.py) | Returns 422: return EncodedFile(...) / 425: class EncodedFile(object): | recovered | Corrects step 2’s inefficiency by narrowing straight to the relevant lines |
| 4 | 3–4 | file_editor(view, view_range=[420,450]/[450,470]) | Shows EncodedFile.__init__/__getattr__, revealing it delegates .mode straight from the binary-mode buffer | valid | Pinpoints the exact root cause (unfiltered __getattr__ delegation) that step 5+ fixes |
Looking at the results that were outputted, it is important to look at the last 5 bullets: E2E check, E2E score, step level score, tool-call precision, and argument accuracy. E2E check tells us that the related tests for the bug issues it sought to fix passed, meaning E2E score in this case is 1 (is_resolved: true). The next metric is the step level score that tells you how many of the steps the model took were actually needed. Looking at the score for the table above, step level is 3/4 due to one of the steps in this case being redundant – in particular step 2. In this case, it directly plays into the tool-call precision which also received a 3/4 due to the minor misstep. For this trace, our final metric argument accuracy saw that all arguments filled in correctly with no malformed arguments.
The line between “calling a tool” and “completing a task” no longer holds: most benchmarks measuring general capability now also measure tool use because models aren’t run without tools in any viable deployment. A benchmark that withholds tool access scores a capability nobody ships.
Not every benchmark makes the case. HumanEval runs generated Python against unit tests, providing executable verification, but no tool call and no environment to act on. SWE-bench is where the shift becomes clear: resolving a real GitHub issue means navigating a codebase, writing a patch, and passing the suite — file-read, search, and edit calls in sequence. The score measures the outcome, but the trajectory underneath consists entirely of tool calls. In many cases, then, the benchmarks you already run for general capability are already exercising tool use. That reframes the question that matters:
Academic benchmarks measure a model’s capability ceiling in the abstract. Enterprise benchmarks answer the narrower, more useful question: can it do my job — your tasks, against your APIs, under your policies? The closer a benchmark sits to production, the more its score should weigh in your decision.
Read NVIDIA Nemotron 3.5 Lightning’s published suite as task completion and time-to-done, not isolated call accuracy. Banking scores completion across a multi-turn banking conversation — the refund trace at scale, not a single call. GDPval-AA v2 scores real agentic work from actual job outputs, judged pairwise by a panel of LLM judges with Elo anchored to a 1,000 human-expert baseline — the kind of human validation that keeps a judge score trustworthy. On PinchBench, Nemotron 3.5 Lightning hits 86% accuracy while finishing 10,000 tasks 30% faster than Qwen3.6 35B at comparable accuracy — a model that finishes efficiently beats one scoring higher on isolated accuracy while burning more steps and tokens.
Public scores are a great signal, but they shouldn’t be considered as a release gate. Adapting the model to your task and use-cases is as important as ever.
Verify consequences in the environment, use judges for language, and use tool-call precision and argument accuracy to find where the chain breaks.
To reproduce the published numbers, Nemotron ships reproducibility docs covering the configs behind the model card scores. Try it on build.nvidia.com, grab weights from Hugging Face, or follow the NIM guide.
Tool calling in the realm of LLM benchmarking is the foundation that evaluations today rest on. Being able to build, read, and understand these evaluations is pertinent in making an informed decision for your use case.
Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn, X, YouTube, and the Nemotron channel on Discord.
Access open Nemotron Models on Hugging Face and a collection of NIM microservices and Developer Examples on build.nvidia.com.
Data science teams often move among separate tools for governed data access, R analysis, Python model development, deployment, application development, and reporting. Positron, Posit’s integrated development environment (IDE) for data science, now runs on Amazon SageMaker AI.
For a data scientist, running Positron on SageMaker AI means:
Posit publishes a container image definition for Positron, built on the Amazon SageMaker Distribution image. Platform administrators build that image, push it to their own Amazon Elastic Container Registry (Amazon ECR) repository, register it with SageMaker AI, and attach it to a Studio domain. Data scientists then choose Positron when they create a Space and open the IDE directly in Studio.
This post shows how a data scientist experiences Positron in SageMaker AI, from exploring an Amazon Athena table to deploying a real-time endpoint.
This walkthrough uses a synthetic 50,000-loan portfolio. Amazon S3 stores the source data, and the AWS Glue Data Catalog registers it. Amazon Athena queries the data, R validates features, and Python trains an XGBoost classifier. Shiny for Python invokes the endpoint, and Quarto records the workflow. The screenshots and metrics come from the captured run. The data does not represent a production lending system.
To follow this walkthrough, an organization needs:
ml.t3.xlarge instance or larger for the demonstrated environment.Figure 2. One project connects governed AWS data, R and Python analysis, managed serving, an application, and a reproducible report.
The run began with Positron in a SageMaker Studio Space. The project explorer, editor, R and Python sessions, Variables pane, plots, terminal, and application preview were available in one browser-based environment on SageMaker compute under the Space execution role.
From the same Space, Posit Assistant identified credit_risk_blog.loan_tape_source in the AWS Glue Data Catalog and prepared a read-only Amazon Athena query. The query returned five sample rows across six fields, scanned 2.18 MiB, and completed in under one second.
Posit Assistant can use Amazon Bedrock as a model provider with AWS credentials and a configured AWS Region. No separate model-provider API key is required when Amazon Bedrock authentication resolves through the environment’s AWS credentials. Customer content is encrypted, isn’t used to improve base models, and isn’t shared with model providers (see Amazon Bedrock data protection). Private connectivity can be configured with AWS PrivateLink.
The captured Session information view recorded 6,657,942 tokens, including 6,118,411 cache-read and 462,905 cache-write tokens, with an estimated cost of $6.319 and 92.5 percent cache efficiency, as shown in the following figure. Those values describe this session and the Assistant’s estimate. They aren’t an AWS invoice or a general cost benchmark. Cache behavior and pricing depend on the selected model and provider.
The workflow used an aggregate Athena query to examine row counts, identifier uniqueness, missing values, numeric ranges, and target validity. The results identified 50,000 loans, including 1,500 records with missing income and 1,015 defaults, for an overall default rate of 2.03 percent.
The workflow loaded the 50,000-row table into the active R session and opened it in Data Explorer. R created debt-to-income and log-income features and displayed the debt-to-income distribution in the Plots pane. Excluding the 1,500 incomplete records left 48,500 loans for modeling and scoring.
The validated feature definitions then moved into Python. An XGBoost classifier trained on a matrix containing 40,000 rows and three model features. The held-out evaluation produced an AUC of 0.834 and showed a 12.3 percent observed default rate in the highest-risk decile.
The workflow wrote predicted probabilities and risk deciles for 48,500 loans to Parquet and registered the results as credit_risk_blog.scored_loans in Athena. It then created a SageMaker AI model, endpoint configuration, and real-time endpoint. The endpoint reached InService, and an invocation using a synthetic applicant payload succeeded.
The project used a Shiny for Python application to invoke the deployed endpoint. The application accepted synthetic applicant information, applied the feature definitions used during training, and displayed the returned probability of default. The source code and running application remained in the same Positron project which runs behind the Amazon SageMaker Studio application proxy and is reachable only by users authenticated to the Space. It invokes the endpoint under the Space execution role rather than any stored key, and the role is limited to sagemaker:InvokeEndpoint on the endpoint ARN.
The run concluded with a Quarto report that connected the Athena source, data-quality findings, R validation, Python model, scored output, SageMaker AI endpoint, and Shiny application. The report was generated directly from the project, preserving the workflow’s evidence and results in one reproducible document.
The deployment involves two paths: an administrator path that builds and registers the custom Positron image, and a data science path that uses it to analyze data and deploy models.
Positron runs as a custom image built on the Amazon SageMaker Distribution image in SageMaker AI. An administrator builds the Posit-published image definition, pushes it to a private Amazon Elastic Container Registry (Amazon ECR) repository in the Studio domain’s AWS Region, registers a SageMaker AI image and version, creates a JupyterLab app image configuration, verifies licensing, grants the execution role the required permissions, and attaches the image to the domain. Posit publishes the image definition, for example the Positron SageMaker Containerfile, which builds on the SageMaker Distribution base image. Amazon Bedrock is optional and is involved only when it’s selected as the Posit Assistant provider.
Responsibilities remain separate. Posit provides the software image and product support. The customer manages identity, permissions, licensing, networking, logging, image updates, and approved AWS services. AWS operates the managed cloud services.
The data scientist launches JupyterLab in a SageMaker Studio Space, opens Positron, and uses R, Python, Quarto, Posit Database Drivers, and optionally Posit Assistant to query and analyze data and deploy models.
The recorded workflow demonstrated the following capabilities within a single Positron Space:
The dataset and applicant payloads were synthetic. The workflow didn’t establish model fairness, calibration, lending suitability, production latency, load behavior, monitoring, or regulatory compliance. The AUC and decile results came from one held-out split, and the lower deciles weren’t strictly monotonic. The screenshots document one recorded run and shouldn’t be presented as a general performance or cost benchmark.
Production adoption also requires validating the Posit preview terms, license grant, and image version alongside supported AWS Regions and model availability. Teams must also confirm network design, least-privilege permissions, secrets handling, logging, image patching, and operational ownership.
To avoid ongoing charges, delete the resources this walkthrough created. Delete them in the following order, because the real-time endpoint depends on both its endpoint configuration and its model: delete the endpoint first, then the endpoint configuration, then the model.
aws sagemaker delete-endpoint --endpoint-name <name>aws sagemaker delete-endpoint-config --endpoint-config-name <name>aws sagemaker delete-model --model-name <name>aws s3 rm s3://amzn-s3-demo-bucket/<prefix>/ --recursiveThen, in the Amazon Athena console query editor, run:
DROP TABLE IF EXISTS <database>.<table>;aws ecr batch-delete-image --repository-name <repo> --image-ids imageTag=<tag>aws sagemaker delete-image --image-name <name>The recorded workflow shows how a custom Positron image can keep governed AWS data access, R and Python analysis, model deployment, application development, and reproducible reporting in one SageMaker Studio Space. The continuity is useful because the evidence, code, deployment result, and communication artifact remain connected. Production use still depends on the customer’s security, governance, validation, and operating controls.
For related background, see:
Abhishek is a Partners Solutions Architect at AWS, specializing in building Generative AI applications. With a deep passion for using agentic AI frameworks to solve complex business challenges, he brings nearly a decade of expertise in developing data and AI solutions that deliver tangible value for enterprises. Beyond his professional endeavors, Abhishek is an artist who finds joy in creating portraits of family and friends, expressing his creativity through various artistic mediums.
SriAakash is a Software Engineer on the Amazon SageMaker AI team, where he builds products and developer experiences across Amazon SageMaker Studio. He focuses on developing solutions that simplify and enhance the machine learning development experience for data scientists and developers. Outside of work, SriAakash enjoys staying active through hiking, biking, and long walks.
Arkaprava is a Software Development Manager at AWS on the SageMaker AI team. He has been at Amazon for over 10 years and works on improving the Amazon SageMaker Studio IDE experience for machine learning developers.
Arantza is a Senior Technical Product Manager for Amazon SageMaker AI. She is passionate about building scalable products that solve real customer problems. At AWS, she focuses on the developer experience of SageMaker AI Studio, helping data scientists across industries build, train, and deploy AI/ML models. Outside of work, Arantza enjoys traveling, playing soccer, and cooking.
Sam is a Senior Partner Development Manager working with GenAI ISVs, building strategic partnerships and innovative solutions for AWS customers. With over 12 years of experience across cloud technology and the partner ecosystem, Sam brings deep expertise in AWS Marketplace and collaboration with leading system integrators and GenAI partners.
During a recent Agent Hackweek, an internal Sentry event that gives us a week to build any AI or agent project we want, a colleague pitched me on writing the Laravel AI integration. The goal was to give agents built with Laravel AI the same Agent Tracing support we already have for other frameworks.
I liked the idea, he built Sentry's Agent Tracing for Python based agents before which meant he already had domain knowledge. We were also supposed to use AI for that, so the language barrier wasn't a real issue.
I was pretty confident that Claude would handle the coding reasonably well. Laravel exposes Events with typed data, so the shape should be fairly known.
After the first review, I was a bit shocked to see that Claude did in fact fail to figure out the correct shape of data and created a helper to access fields in the most generic way possible:
/**
* Access a property from a value that may be an object, array, or null.
*/
private function flexGet(object|array|null $source, string $key): mixed
{
if ($source === null) {
return null;
}
if (is_object($source)) {
return $source->{$key} ?? null;
}
return $source[$key] ?? null;
}
For anyone else who hasn't looked at PHP in a minute, this snippet is a generic helper that tries to retrieve values from arrays or objects regardless of their structure. $source->{$key} will resolve $key to its string value and will access the field. Needless to say, this is not good PHP code and is rarely ever useful since most of the time the data shape is more narrow than that.
Instrumentation in Python or JavaScript is often relatively easy: we can just wrap or patch a function. In PHP, not so much. We have to rely on hooks from a framework or library or ask users to replace classes with their own. We try to avoid the latter, since an integration that requires users to rewrite their code is not much of an integration. Luckily, Laravel AI emits events for most of the important parts, just not quite all of them.
At first glance, Laravel AI provided good ways to hook into its lifecycle. The PromptingAgent and AgentPrompted events cover an entire agent interaction, while InvokingTool and ToolInvoked cover individual tool calls.
Agent Tracing needs one more level of detail: every LLM invocation should appear as its own Chat span. Laravel AI does not expose an event for these invocations, so relying on its lifecycle events alone would leave a significant gap in the trace.
Most LLM invocations ultimately result in HTTP requests, and Laravel provides events for those: RequestSending and ResponseReceived. We could create a Chat span for every HTTP request made during an agent interaction, but that would also capture unrelated requests, such as HTTP calls made by tools.
Laravel's HTTP events do not include the AI invocation ID, so we cannot associate the requests directly. Instead, we store the configured provider URL prefix for each active invocation and compare it with the URL of every outgoing request. If more than one active invocation matches, we associate the request with the most recently started one. Only matching requests become Chat spans, which filters out unrelated HTTP traffic without losing individual LLM calls.
The Laravel AI integration shipped with sentry-laravel 4.27. For an application that already has Sentry tracing enabled, updating the SDK is all it takes. There is no integration to register and no Sentry-specific code to add. A regular Laravel AI agent is traced automatically, and implementing Conversational adds Conversations support.
// ...
class DemoOpsAgent implements Agent, Conversational, HasTools
{
use Promptable, RemembersConversations;
public function instructions(): Stringable|string
{
return 'You are DemoOps, an AI launch director. Be concise and practical.';
}
public function messages(): iterable
{
return [];
}
public function tools(): iterable
{
return [
new GetTime,
// ...
];
}
}
No Sentry code in sight. Agent invocations, LLM requests, and tool calls from this class show up in Agent Tracing, while its conversation context appears in the Agents section in Explore.
Once the first traces arrive in Sentry, the Agents section in Explore lists conversations with their duration, message and error counts, estimated cost, and the tools used.
Opening a conversation shows the transcript, with tool calls alongside the user's messages and the agent's replies.
Selecting a tool call shows its inputs and outputs. This makes it possible to compare what the tool returned with the agent's response.
It's super easy to get started. If you already have Sentry tracing set up in your Laravel app, all you need to do is update to version 4.27 of the SDK: Laravel Agent Tracing is enabled by default when laravel/ai is installed and tracing is active. For the full instructions on getting up and running, head over to our Laravel Agent Tracing docs and start exploring your agent traces and conversations in Explore > Agents.
The tokenizer has not historically been the bottleneck within ML workflows. Compute-wise, tokenization is light compared to the heavy modeling happening in the rest of the pipeline. Yet, in some cases, it has rapidly become key to accelerating (or slowing down) your machine learning work.
As models become faster and workloads scale, that balance begins to shift. Training on massive datasets, serving many concurrent requests, or repeatedly processing long inputs can put enough pressure on the tokenizer that it starves the model of data.
This is why we have chosen to heavily focus on performance for the upcoming version 1 of tokenizers. Tokenization should be light and should scale with your workflow. Your GPUs should never sit idle waiting for the CPU to complete its tokenization.
In this article, we look at what makes v1 faster than v0.23, often by tens of times.
This work was entirely possible thanks to the rest of the ecosystem. Tokenization is a very active area of open source work, and libraries such as gigatoken, tiktoken, kitoken, tokie, fastokens, wordchipper and ai-tokenizer, as well as many others, have each pushed on what a fast tokenizer can be. We read that work, and several of the ideas below reached us because another project showed they were worth trying.
Before this refactor, tokenizers was nowhere near the performance it could have had, so contributing to it may not have seemed worth it. With this refactor, we hope to make clear that we intend tokenizers to be a library worth contributing to.
We also thank IBM, NVIDIA, and the ExecuTorch team for contributing patches and helping us test across a wide range of hardware to broaden platform support.
We showcase results for the release candidate of tokenizers v1 against other widely used alternatives. We go over single-threaded, multi-threaded, scaling across threads, per-model comparison, per-language comparison, latency, decoding throughput, memory heap, as well as crate size.
We run this from the tokbench repository, and add a command to rerun the benchmarks on your hardware if you would like to do so.
v1 will produce the same token IDs as v0.23. The goal was to preserve the output, the API, the vocabulary and the merge ranks, and improve everything that can be improved. That includes breadth. The library stays general across tokenizer families rather than specialising on BPE, so v1 loads everything v0.23 loaded.
A tokenizer converts text into the list of integers a model reads. tokenizers runs that conversion in four stages. Normalization applies operations such as lowercasing or Unicode normalization to the raw text. Pre-tokenization splits the text into smaller pieces called pre-tokens. The model turns each pre-token into tokens and maps them to IDs in its vocabulary. Post-processing adds any special tokens the model expects.
The model stage is where most of the work described here happens. Eight of the ten model families measured in this article use byte pair encoding, or BPE. BPE starts from the bytes of a pre-token and repeatedly joins the highest ranked adjacent pair until no ranked pair remains. The ranking is learned when the tokenizer is trained and ships with it, so the same text always produces the same IDs. A merge never crosses a pre-token boundary. The other two families use WordPiece and Unigram, the two other model types the library supports.
The tokenization pipeline page documents the four stages. Tokenization algorithms documents BPE, WordPiece and Unigram.
Each stage was worked on. These are the changes that mattered:
| change | what it does |
|---|---|
| workspace split | one crate became a workspace: tk-encode is the required runtime, and tk-serialize, tk-convert and tk-train are linked only when an application needs them |
| no-alloc model | the merge working set lives in a caller-owned scratch buffer; the loop never touches the allocator |
| bitcannon | the split pattern becomes Boolean operations over bitstreams, using SIMD instructions to find splits instead of a regex engine |
| merge-loop rewrite | the pieces being merged form an intrusive doubly-linked list inside one preallocated buffer, so a merge updates two indices instead of moving data |
| word cache | a thread-local memo from pre-token bytes to finished ids, so a repeated word is merged once |
| native parallelism | one shared tokenizer encodes from many threads at once; each thread draws its scratch buffer and word cache from its own sub-pool, so threads no longer queue on a single lock (#2365) |
BPE models use a regular expression to split the input text into smaller, easier to process chunks called pre-tokens. Merges happen inside a pre-token and never across the boundary between two of them, so this split decides what the rest of the pipeline sees.
That regular expression is a fixed parameter of the model. It ships with the tokenizer and never changes at runtime, so there is no need for a general-purpose regex engine to interpret it on every encode. An equivalent splitting function can be written by hand, once, for the pattern a given model actually uses.
A hand-written function can then use the SIMD instructions (single instruction, multiple data) of a modern CPU, which apply one operation to many bytes at once and suit UTF-8 text well. bitcannon views the input's bytes as parallel streams of bits, so boundaries fall out of boolean operations across whole registers instead of a scan that advances one character at a time. It decides 64 bytes per register operation. The same idea drives Parabix for text processing and simdjson for JSON.
This depends on recognising the pattern. A handful of grammars cover most byte-level BPE models, and a tokenizer whose pattern is not among them keeps the regex path and none of this speed-up. That is why the gains above vary as much as they do.
Real text contains many repeated words. Because BPE always produces the same token IDs for a given pre-token, v1 can save the result after processing it once. A thread-local cache maps each pre-token's bytes to its token IDs, allowing later occurrences to skip the merge process.
Naturally, as the input grows, the number of unique words can grow more slowly than the total number of words. Repeated words then account for an increasing share of the input. New words still appear, which accounts for the occasional misses in the animation below.
Reproduce the shared-prefix result with:
tokbench measure prefix-sharing \
--engine pipeline \
--engine hf-tokenizers \
--compare-to pipeline-no-cache \
--corpus agentic_swe
Caching works best when the input contains repeated pre-tokens. Input with few repeated pre-tokens can pay for lookups without receiving many hits.
The next major cost comes from the BPE merge loop. For each pre-token, the loop repeatedly finds the highest-priority adjacent pair and merges it. The previous implementation allocated new memory for every call and built a new priority queue for every pre-token.
v1 reuses a scratch buffer owned by the caller, removing those repeated allocations. It stores symbols in a flat array and links adjacent symbols by their positions in that array, which makes updates during merging cheaper. It also processes a batch of pre-tokens in a single model call.
Each candidate pair is also packed into a single 64-bit value, with the merge rank in the high bits. Comparing two candidates is then just comparing two integers, and "no merge here" is the largest possible value, so the loop finds its next merge without a branch.
Small differences in benchmark design can produce large differences in tokenizer performance. We used the following rules to keep the comparison consistent across engines.
| rule | why |
|---|---|
| one timing loop | every engine runs the identical loop; no per-engine fast path |
| load excluded | vocabulary load is timed separately, never inside encode |
| id-hash verified | FNV-1a over the output ids must match the baseline exactly |
| common cells only | medians are over cells every engine ran and verified |
| complete sweep per process | each repeat starts in a new process and retains every cell |
| physical-core pinning | workers are pinned to eight distinct physical cores, never sibling SMT threads |
| independent Jobs | separate Jobs measure host-to-host variation |
Repeatedly encoding one document can be faster than encoding a stream of distinct documents on the same build. The first approach measures performance when the entire document is already represented in the cache. The second measures performance on new input while allowing previously seen pre-tokens to remain cached.
Both conditions are sometimes described as "warm," even though they measure different workloads. Our headline results use distinct documents, and the complete corpus is too large to fit in the cache. Tokenizer benchmarks should identify which workload they use because the choice can dominate the result.
Across the ten model families v1's encode path covers, it encodes text 3 to 30 times faster than v0.23 with one thread on an Apple M4 Max. The low end is t5-base, the high end gpt2. It scales at 76% of linear across eight workers. Throughout these changes, v1 produces exactly the same token IDs as the released library.
The overall improvement comes from several changes working together: a hand-written splitter in place of a regex engine, a cache that answers a repeated word without merging it again, a merge loop that never touches the allocator, and one model call per batch of pre-tokens instead of one per pre-token. Each reduces the work done at a different point in the pipeline.
The next priority is support for more model families. We will move additional models onto the new merge loop before 1.0.0. Once the release candidates stabilize, the next step will be bringing about the improvements within the transformers library and the rest of the ecosystem which depend on the tokenizers library.
This post is generated from tokbench results and will be updated as support expands.
A release candidate for v1 is on crates.io. The API you call is the one you already call, so the only thing that changes is which build you install.
It is the ordinary install:
cargo add tokenizers --pre
Training is behind a default-on feature that pulls a C++ dependency with it. If you only need to encode, turn it off to exclude the training implementation:
cargo add tokenizers --pre --no-default-features --features http
Encoding is unchanged: same call, same ids.
use tokenizers::tokenizer::{Result, Tokenizer};
fn main() -> Result<()> {
let tokenizer = Tokenizer::from_pretrained("deepseek-ai/DeepSeek-V4-Flash", None)?;
let encoding = tokenizer.encode("The tokenizer is no longer the bottleneck.", false)?; println!("{:?}", encoding.get_ids()); // [671, 17840, 9160, 344, 1119, 5827, 270, 111127, 16] println!("{:?}", encoding.get_tokens()); // ["The", "Ġtoken", "izer", "Ġis", "Ġno", "Ġlonger", "Ġthe", "Ġbottleneck", "."]
Ok(()) } ```
For a batch, `encode_batch` is what scales across cores. It is the call the scaling view above measures.
```rust
let encodings = tokenizer.encode_batch(documents, false)?;
Every figure in this post was measured against this crate. The Python bindings wrap the same code and are built from bindings/python, but they add per-call overhead that none of these measurements include.
The benchmarks in this post cover the completed release-candidate work listed first. The remaining sections show what is still required for 1.0.0 and what we plan to explore afterward.
This work is in the Rust pre-release on crates.io:
cargo add tokenizers --pre
tk-encode, tk-serialize, tk-convert and tk-train, so an application links only what it usesaf5a3e3STAGE_POST pipeline stage #2182role_to_token support #2343tk-encode during training validation so training and inference cannot produce different tokenization results
Just-in-time (JIT) compilation within the JVM is essential to Java's performance and plays a central role in ongoing OpenJDK projects such as Valhalla, Leyden, and Panama. JIT compilation is constantly evolving to support the evolution of the Java language, its main application areas, deployment models, and underlying hardware.
This talk provides an overview of recent performance improvements delivered by the JVM's JIT compilers, both as part of and in addition to OpenJDK projects, and highlights some of the most significant ongoing developments in this area.
Make sure to check the JavaOne 2026 playlist.
You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast?
Your instincts might lead you to send curl commands, hand-roll an asyncio script, or vibe code yet another one-off load generator. All of these paths have the same problem: single-process performance limits, Python’s GIL capping concurrency, or numbers measured against a reference you built yourself. Either way, you end up with results you can’t fully trust, attached to tooling you’ll have to rewrite the moment requirements change.
What you need is a load client that can saturate a real server without becoming the bottleneck, produce output you can act on, and take five minutes to configure, not five hours. That’s NVIDIA AIPerf.
AIPerf is the designated successor to GenAI-Perf and is a ground-up rewrite. The design choices reflect hard lessons from running LLM benchmarks at scale:
For this walkthrough we’ll use Qwen3-0.6B served through vLLM. The model choice is deliberate; it’s small enough to run on a single GPU and fast enough to iterate on without waiting. The point isn’t to benchmark Qwen3-0.6B specifically; it’s to establish the measurement loop. Once you have that, swapping in a different model or endpoint is a one-flag change.
Start the Server
Pull and start vLLM with the reasoning parser enabled:
docker pull vllm/vllm-openai:latest docker run --gpus all -p 8000:8000 -e HF_TOKEN vllm/vllm-openai:latest \ --model Qwen/Qwen3-0.6B \ --reasoning-parser qwen3 \ --host 0.0.0.0 --port 8000
We can use uv to install a central copy:
uv tool install aiperf
Or for a virtual environment:
uv venv venv source venv/bin/activate uv pip install aiperf
One platform note: on aarch64, the crick dependency ships as source-only and requires a C toolchain (build-essential on Debian/Ubuntu, Development Tools on RHEL). If the install stalls on that package, that’s why.
With the server up and AIPerf installed, we can now run our first profile:
aiperf profile \ --model Qwen/Qwen3-0.6B \ --endpoint-type chat \ --streaming \ --url localhost:8000 \ --synthetic-input-tokens-mean 128 \ --synthetic-input-tokens-stddev 0 \ --output-tokens-mean 128 \ --output-tokens-stddev 0 \ --extra-inputs min_tokens:128 \ --extra-inputs ignore_eos:true
A few flags here are doing more work than they look like:
--synthetic-input-tokens-stddev 0 and --output-tokens-stddev 0 pin the workload to exactly 128 input and 128 output tokens per request. This reproduces a commonly used static benchmark that holds request and output lengths constant.
--extra-inputs min_tokens:128 and --extra-inputs ignore_eos:true tell the model to actually emit 128 tokens rather than stopping early. Without these, the output token count is a suggestion. The model stops whenever it naturally finishes, which can be well short of your target OSL. Throughput numbers end up lower than they should be, and they’re not reproducible across runs.
--streaming is not optional if you want to measure TTFT and ITL. Without streaming, the server batches the full response before sending it, and there are no first- or decode-token events to measure.
We’ll walk through how to read these numbers in the next section. For now, notice the shape of the output in Figure 2, below: latency broken down by percentile, throughput in tokens per second, and request-level statistics all in one place. That’s the baseline you’ll be comparing everything else against.
Once a run completes, AIPerf prints a metrics table to the console and writes the full results to CSV and JSON. Here’s what you’re looking at.
The core four:
For full definitions of these and every other metric AIPerf reports, see the Metrics Reference.
Getting the full picture. Each of the above is reported in percentile breakdowns (p25, p50, p75, p90, p95, p99) alongside their minimums, maximums, averages, and standard deviations. These breakdowns matter because they can highlight long tail distributions; a server with a healthy mean TTFT and an outlier p99 looks fine in aggregate and fails in production.
Beyond the core four. With DCGM or pynvml available, AIPerf also pulls GPU power draw, utilization, and memory consumption into the same run output. Correlating a latency spike with a memory pressure event doesn’t require a separate profiling session, the telemetry is already there.
Now that our feet are wet with a static benchmark, we can start exploring something more dynamic. The section above provided an extremely fixed traffic pattern, but real inference traffic doesn’t follow a static pattern. To benchmark with a scenario that’s less rigid, we can use some of AIPerf’s synthetic workload knobs to introduce variability to our requests.
aiperf profile \ --model Qwen/Qwen3-0.6B \ --endpoint-type chat \ --streaming \ --url localhost:8000 \ --request-rate 10 \ --arrival-pattern poisson \ --synthetic-input-tokens-mean 512 \ --synthetic-input-tokens-stddev 128 \ --output-tokens-mean 128 \ --output-tokens-stddev 32 \ --random-seed 42 \ --request-count 200
A few things changed from the static benchmark above.
--arrival-pattern poisson with --request-rate 10 means requests arrive at an average of 10 per second, with inter-arrival times drawn from an exponential distribution. The server now experiences bursts and gaps rather than a single user stream, which is what queuing actually looks like under real traffic.
--synthetic-input-tokens-stddev 128 introduces variance around the 512-token mean, producing a mix of short and long prompts. The server has to handle variable prompt lengths during prefill rather than identical ones.
--output-tokens-stddev 32 adds variance on the output side. Notice that min_tokens and ignore_eos are gone from this command. In the static benchmark those flags pinned outputs to exactly 128 tokens to keep the baseline clean; we’re deliberately releasing that constraint so the output distribution can vary.
--random-seed 42 makes the Poisson timing and synthetic length draws reproducible. Rerunning this command produces the same sequence of requests.
--streaming is not optional. Without streaming, the server batches the full response before sending it, and there’s no first- or decode-token events to measure.
Looking at the LLM metrics from this run, the distributions are noticeably wider than the static baseline — which is expected when more requests are simultaneously competing for GPU access and prefill lengths vary per request.
Looking at the graphs in Figure 4, below, you can see that the Poisson command line introduced a request rate centered, but not exactly matching, around 10 requests/second. This arrival rate emulates jitter around when requests arrive compared to the constant mode which guarantees a fixed 10 requests/second.
You can see in Figure 5, below, that there is a variation in the request length centered around the mean of 512 tokens, with input sequence lengths ranging 154 to 818 tokens.
Comparing TTFT between the two runs, you can see that the Poisson run shows a much wider spread. More requests are simultaneously competing for GPU access, prefill lengths vary, and prefill and decode operations overlap. The single-concurrency case is an idealized scenario which runs one request at a time presenting the lowest possible TTFT, at the cost of throughput.
In Figure 6, above, you can see that the single user run experiences less TTFT variability than the much more varied workload in the Poisson experiment.
The tutorials in the AIPerf repo are the fastest way in. The AIPerf repo and docs are the canonical reference for new features and contributions.
AIPerf is a collaborative effort between NVIDIA and external contributors. Thank you to the following: Loki Ravi, Dan Ferguson, and Sheng Moua (AWS) for the continual collaboration, cross-company validation, and efforts to standardize on AIPerf; Aaron Batilo (Coreweave) for the Weights & Biases exporter, acceptance-length spec-decode datasets, and hardening sweep/credit-dispatch reliability under concurrency; Shounak Ray (Baseten) for faithful Baseten trace replay support; Michael Feil (Baseten) for faster trace loading, and session affinity headers. Cristian Lopez (Pinterest) for his close collaboration on the DAG benchmarking methodology. We’re grateful to Ben Hamm for his product guidance while we designed, planned, and implemented AIPerf.
Cloudflare operates at a scale so big that even after working here for years, it doesn’t seem real. We have thousands of servers all over the world with petabytes of RAM and millions of CPU cores, and all of it is pushed to the max. As vast as those resources feel, they are still finite, and when you need every service to run on every node, it doesn’t leave room for wasted space.
At this scale, small improvements are greatly magnified, so even 1%-at-a-time improvements are worth celebrating. And some tweaks add up to a lot more: in this post, we’ll look at how small changes to a single algorithm reduced the memory footprint of one of our Pingora-based services significantly. That allowed us to reclaim more than 100TB of RAM globally, on top of the 100TB of memory the DNS team was able to shed last month.
Maintaining equitable resource sharing between teams is not easy, especially in large organizations. One of the ways Cloudflare ensures the balance is kept is through the tireless efforts of the wonderful Performance team.
This story starts with a ticket filed by Ivan who found: Excessive memory usage from pingora-ketama in Pingora Backend Router. The finding was that our internal load-balancing service, Pingora Backend Router (yes, PBR), was using significantly more memory than expected — specifically in structures associated with pingora-ketama, which is our open-source library for handling consistent hashing.
In order to talk about how we addressed this seeming overuse of memory, we need to talk about what consistent hashing even is, why we are using it in PBR, and how it became so memory hungry. Along the way, we’ll learn some Rust and even a little math.
Consistent hashing is a widely used method for distributing tasks across multiple servers in a way that does not require large changes when servers are added or removed. Internally we use it to route cacheable requests to servers by URL. This allows us to keep only one copy of a file stored per data center and gives a stable way to find the location of each file. We have mentioned this system before, but let’s take the time to walk through how and why this algorithm is used and how it works.
The key concept of consistent hashing is that while hash functions can accept any kind of input, their output is limited to a single unsigned integer (32, 64, or 128-bit integers depending on which hash function). This allows us to relate tasks and servers to each other in a consistent way. Most discussions of consistent hashing have you think of that output space as a continuous, circular ring that wraps around from its max value to zero. This depiction makes for some nice visualizations, but it can also make the simple concept of integer ranges seem more complicated than it needs to be. For our discussion, we’ll represent the 32-bit output of our hash function as a number line.
Now, let’s say we have a set of servers, A, B, & C, and a set of tasks t-z. We can map each onto the number line based on the hash of their representative values, so something like IP addresses for servers and cache keys for tasks.
Assigning tasks to servers is now just a matter of finding the first server to the left of each task. We can represent this visually by coloring in the region of hashes that will be associated with each server. Notice that the range covered by server C wraps around to the beginning, hence the idea that hashes exist in a ring.
And that’s it. At a base level, consistent hashing is this simple — but it doesn’t take long to see that there is room for improvement. Notice that the range covered by server A in our example is significantly larger than that of either B or C. This is a problem because the fraction of the requests a server handles is going to be proportional to the size of its range on the number line. Ideally we would like to guarantee each server will have an equal size, but because hashes are essentially random numbers, we have to talk about the size of the regions in terms of statistics. 😨
First: don’t panic. I promise I'm not about to lie to you and that we will stay safely within the bounds of a day-one probability lesson. When we talk about statistical distributions, there are two big factors that help us quantify uncertainty in helpful ways: expected value and standard deviation. In (over-)simplified terms, expected value gives us a point where measurements based on a distribution will be centered, and standard deviation tells how close to that central point most measurements are likely to be.
For consistent hashing, we can calculate these factors for the fractional size of the range associated with one of N servers. (Details on where this formula comes from later).
In terms of concrete numbers, let’s say we have 100 servers. The formulas above give:
That tells us that we can expect that the range each server handles will be centered around 0.99% of the total and most of the lengths to fall within 1% of what's expected. This sounds good until we realize that that’s 0.99% of the total length. We need to scale the standard deviation by the expected value to see how big the error is as a fraction of the target size. This value is called the coefficient of variation.
The simplicity of consistent hashing is a double-edged sword. It’s easy to understand and implement because everything is turned into easily-relatable hashes on the same numberline, but any improvements to the system will also need to be relatable to that numberline. That means the solution to any consistent hashing problem can only be more hashes. It’s less like a golden hammer (a tool with which all problems look like nails) and more like a golden nail in that it turns all tools into hammers.
To solve the problem of imbalanced workloads, we can add multiple hashes to represent each server instead of just one. We’ll get to the math behind this momentarily, but it should make some intuitive sense that while each individual range has a large standard deviation, adding a bunch together should make their total size even out. If we take our three-server example from the above diagrams and add two more hashes at random for each server, we see that it helps even out each server’s workload.
This is an admittedly contrived example. The random nature of the system means there’s no guarantee how much improvement you will get from adding 2 additional hashes per server, but it should make some intuitive sense that combining more of these hash segments together produces a more even distribution. Each segment in the sum has a chance of balancing another. Maybe one is too short; maybe one is too long. This is essentially what the law of large numbers tells us should happen… The obvious problem is it only works for large numbers. In NGINX, the baseline number of hashes per server is hardcoded to 160, and Pingora uses the same value as the default. I’ll spare you the math for now, but if we go back to our 100-server example, if we use 160 points per server instead of just one, the coefficient of variation (which we can think of like an error margin) drops from about 99% to about 8%, a significant improvement.
We saw above that increasing the number of hashes per server by a constant amount allows us to improve how evenly workloads are distributed per server, but what if we don’t want to distribute the work evenly? In Cloudflare’s case, we have some servers that have more storage space than others, so it would be better to have the number of requests allotted to a server be proportional to its disk space. One way to accomplish this is with the ketama algorithm. The naming is a little funny because the algorithm is named after the library where it was first implemented, and the library was named … well you can google it 😶🌫️.
For us, since we want workload to be scaled based on storage, we can use the disk space as the weight, which is exactly what the Pingora team has been doing for years. Elsewhere in the company where workloads are more compute-intensive, weights might be based on CPU or GPU count.
The last problem we need to address is that so far we are working under the assumption that any server can handle any request, but in practice that is not the case. Things like compliance requirements or enabled caching features mean only a subset of servers can handle any particular request. Unfortunately, unlike before, we can’t solve this problem by adding more hashes to the same ring. We have to add completely new rings, and not only that — every combination of features potentially needs its own specific ring!
One big improvement came from Zaidoon, who had an insight about our struct for storing hashes in PBR. That struct looks like this:
Unfortunately, Rust doesn’t make it that easy. Changing the size of the index as we did above does nothing to reduce the memory footprint. This is because Rust has alignment rules that require the size of a structure in memory to be a multiple of its largest (or “most aligned”) field. In this case, the hash is the largest with four bytes, so when stored in memory, a Point is required to have size $mN \times 4m$, so the minimum size is eight bytes.
Luckily there are well-known ways around this. You (meaning me) might be tempted to use #[repr(packed)], but that is controversial for good reasons. A safer but less readable solution is to store the hash and index as raw byte array and access them with getters. Both methods compile to the same thing.
This simple (if wordy) change reduces the amount of memory used for consistent hashing by a whopping 25%! In order to do better than that, we’ll need to jump back into the math, so everybody hang on to something; this is the home stretch.
You may have noticed that we gave the formula for the standard deviation for the case where there is only one hash per server. Deriving the formula for the case where there are $m k m$ hashes per server is not easy, and most sources only give you an approximation or an asymptotic limit, but not us. I might not be a statistician, but I grew up with a calculus teacher (Hi, Mom!), and I wanted to know the actual value. The full derivation is in a supplemental post, but here is the payoff.
To see how increasing the hash count improves the accuracy, we need to look again at the coefficient of variation.
The predictions from my beautiful math only work if we think about hashes in a continuous ring, but in practice we use 32-bit numbers for the hashes that have the potential for collisions, and the probability of collisions goes up surprisingly quickly as the number of hashes increases (see the birthday paradox). Collisions matter because in the ideal case, every hash contributes to the volume and distribution of requests handled by the associated server, but a collision means some contributions are randomly dropped, introducing unpredictable error. If we compare some simulated results with 32-bit hashes with the predicted error rate, we can see that for data centers with 2048 servers, the error rate increases: between 10,000 and 100,000 hashes per server.
Ultimately, even though this realization feels kind of bad, it’s great news for our plan to reclaim some RAM! Now that we have some math to back it up, we determined that we could decrease the number of hashes we were generating for each server by 90% without incurring any appreciable error, so that is what we set out to do.
There was one more problem: changing the hash ring changes where some cacheable requests go. Even if the new ring is better, switching the whole network at once would effectively invalidate almost all cached content. It would turn a memory optimization into an apocalyptic increase in origin traffic.
So we did not make this a single global flip. For a while, PBR carried both versions of the cacheable load balancer in memory: the old ketama ring and the new smaller one. Each request used our normal migration framework to decide which ring should select the backend. That meant the rollout decision was stable per request hash, and it also gave us a clean rollback path. If anything looked wrong, we could send new requests back through the old ring without redeploying PBR.
We then rolled the migration out in layers. We started with small validation locations, moved through progressively larger groups of data centers, and only then continued toward the rest of the world.
The important part was that we controlled two dimensions independently: how much traffic used the new ring, and where that traffic was allowed to move. A plain global percentage rollout would have spread cache churn everywhere at once. Data-center-scoped rollout kept the blast radius small and made it much easier to tell whether a change was actually safe.
During the migration, we watched backend-selection traces, ring-version counters, PBR connection errors, process memory, startup time, cache behavior, and origin traffic. Once the migration reached 100%, we removed the temporary old-ring path, and voila!
The chart above shows the comparison of the memory used by PBR the week of the change compared with data from a few weeks before, as well as the result of subtracting one from the other. The sharp drop is the day where the version of PBR with the large (now unused) hash rings was decommissioned forever. Looking at the difference, we get the satisfying result that our changes dropped the used memory by 100TB!
All the changes we talked about in this post are available now in the pingora-ketama crate in the form of a (for now) unadvertised cargo feature. The v2 ring has the compacted storage format, a faster sorting method, and the ability to scale the base number of hashes per node. Our focus in making these changes had to be on stability and control, so the v1 ring is identical to what pingora ketama has always used, and the library makes it possible to run both simultaneously and decide on a request-by-request basis which to use and when.
Beyond trying our literal consistent hashing changes, I would like you to take away from this some inspiration to dig into your own systems to see what “simple” or “obvious” decisions are hiding potential wins, if you’re willing to get into the numbers. You might not be able to solve all your problems with Rust, but math is universal.
Organizations building multi-model agentic AI applications face growing infrastructure complexity. Managing container orchestration, scaling policies, identity, and observability for multiple model types adds operational overhead. Teams often spend more time on infrastructure than on agent logic development.
Developers running agentic frameworks on self-managed infrastructure such as Amazon Elastic Container Service (Amazon ECS) with AWS Fargate have full control over their deployment configuration. As agentic workloads evolve and scale, teams might choose to adopt managed runtimes that provide built-in session management, identity, and observability.
Amazon Bedrock AgentCore is a platform to build, connect, and optimize agents at scale, with any framework or model. AgentCore runtime, its managed deployment capability, handles container lifecycle, scaling, identity, and observability, so you can focus on your agent code.
In a previous post, Agentic AI with multi-model framework using Hugging Face smolagents on AWS, we showed how to build a healthcare AI agent with multi-model orchestration on self-managed infrastructure. In this post, we show you how to migrate that multi-model agent to Amazon Bedrock AgentCore runtime. The migration reduces infrastructure management while preserving agent capabilities, including triple-model orchestration and vector-enhanced knowledge retrieval.
This solution migrates a multi-model healthcare AI agent to Amazon Bedrock AgentCore runtime while preserving the existing agent logic. The agent processes medical queries across three model backends with vector-enhanced knowledge retrieval, all running inside a single AgentCore-managed container. You can direct each query to the model backend suited to the task. A domain-specific model such as BioM-ELECTRA-Large-SQuAD2 on Amazon SageMaker AI handles specialized biomedical queries, and a foundation model (FM) such as Llama 3.1 70B Instruct by Meta on Amazon Bedrock handles broader medical reasoning. This approach helps healthcare teams address a range of query types while reducing the operational overhead of managing the underlying infrastructure.
The standalone version from the previous post deployed on Amazon ECS with AWS Fargate includes container orchestration, scaling, identity, and observability configured by the user. The AgentCore version wraps the same agent logic with the AgentCore runtime decorator pattern, and AgentCore runtime handles these operational concerns automatically.
Hugging Face smolagents is an open source Python library designed to build and run agents using a few lines of code. This solution uses Hugging Face smolagents framework as a reference implementation, demonstrating that AgentCore runtime supports any agentic framework. With the bring-your-own (BYO) agent approach, you can deploy existing agent code to AgentCore runtime without rewriting or adapting to a specific framework.
Note: This solution is a sample implementation for demonstration purposes. Production deployments handling medical or other sensitive queries use Amazon Bedrock Guardrails for content filtering and grounding validation as a standard control.
The solution consists of the following services and features:
Note: The previous post (standalone version) uses Claude 3.5 Sonnet V2 by Anthropic. This post uses Llama 3.1 70B Instruct by Meta, demonstrating that AgentCore runtime is model-agnostic. The model choice is an implementation decision, not a requirement.
The following diagram illustrates the solution architecture and how the agent orchestrates across three model backends.
A client web interface connects to Amazon Bedrock AgentCore runtime, which hosts the healthcare agent container. The container uses the Hugging Face smolagents framework with the AgentCore runtime decorator. AgentCore runtime provides built-in identity and observability. The agent orchestrates across three model backends: Amazon SageMaker AI with BioM-ELECTRA, Amazon Bedrock with Llama 3.1 70B Instruct by Meta, and a containerized model server with BioM-ELECTRA. The solution includes Amazon OpenSearch Service for vector-enhanced knowledge retrieval.
This solution supports deployment options with each backend optimized for different scenarios:
The three backends implement Hugging Face Messages API compatibility, providing consistent request and response formats regardless of the selected model service.
The complete implementation is available in the sample-healthcare-agent-with-agentcore-on-aws GitHub repository.
This section walks through migrating the existing healthcare AI agent to Amazon Bedrock AgentCore runtime using the AgentCore CLI.
Before you deploy the solution, you need the following:
Amazon Bedrock AgentCore runtime uses a decorator pattern to wrap your agent logic. The key components are:
@app.entrypoint – decorates the function that AgentCore runtime calls when a request arrives.app.run() – starts the AgentCore runtime server.The following code shows the AgentCore integration pattern:
from bedrock_agentcore.runtime import BedrockAgentCoreApp
app = BedrockAgentCoreApp()
@app.entrypoint
def healthcare_agent_entrypoint(payload):
user_input = payload.get("prompt", "")
model_type = payload.get("model_type", "sagemaker")
# Your existing agent logic here
agent = TripleHealthcareAgent(vector_store=vector_store)
response = agent.run(user_input, model_type=model_type)
return str(response)
if __name__ == "__main__":
app.run()The agent code between the decorator and return statement remains unchanged from the standalone version. AgentCore runtime handles container lifecycle, scaling, identity, and observability automatically.
Create an AgentCore project and add your existing agent using the AgentCore CLI.
Install the AgentCore CLI:
npm install -g @aws/agentcoreCreate a new AgentCore project:
agentcore create --project-name healthcareagent --no-agent --build Container --language Python --protocol HTTP --model-provider Bedrock --memory noneAdd your existing agent as a bring-your-own (BYO) agent:
agentcore add agent --name healthcare_agentcore --type byo --build Container --language Python --protocol HTTP --network-mode PUBLIC --code-location ./agent-code --entrypoint healthcare_agentcore.py --framework Strands --model-provider BedrockNote: The --framework flag specifies the CLI template. The actual agent code uses Hugging Face smolagents, which is compatible with AgentCore runtime regardless of the template selection.
pyproject.toml in your agent code directory to define dependencies:
[project]
name = "healthcare-agentcore"
version = "1.0.0"
requires-python = ">=3.10"
dependencies = [
"smolagents>=1.24.0",
"transformers>=4.55.0",
"boto3>=1.37.0",
"opensearch-py>=3.1.0",
"requests-aws4auth>=1.3.1",
"bedrock-agentcore>=0.1.0",
"numpy>=1.26.0",
"requests>=2.32.0",
"docker>=7.1.0",
]Dockerfile:
FROM public.ecr.aws/docker/library/python:3.12-slim
RUN pip install --no-cache-dir uv
WORKDIR /app
COPY pyproject.toml ./
RUN uv pip install --system -r pyproject.toml
COPY . .
EXPOSE 8080
CMD ["python", "healthcare_agentcore.py"].dockerignore to keep the image size within the 2 GB limit:
venv/
.venv/
__pycache__/
.git/
*.pycWith the project configured, you can deploy the agent using a single CLI command.
Deploy the agent:
agentcore deploy -yThe CLI builds the container, pushes it to Amazon Elastic Container Registry (Amazon ECR), and creates the AgentCore runtime agent. Deployment takes approximately 10–15 minutes.
You can test the deployed agent in two ways: using the AgentCore CLI or programmatically with boto3.
Invoke the agent using the AgentCore CLI:
agentcore invoke --prompt '{"prompt": "What are the side effects of metformin?", "model_type": "llama"}'Or, invoke programmatically using boto3:
This path invokes the same deployed agent as the CLI, using the boto3 SDK directly. The agentRuntimeArn identifies your deployed agent, contentType specifies the request format, and payload carries the prompt and model selection.
import boto3, json
client = boto3.client('bedrock-agentcore', region_name='us-west-2')
payload = json.dumps({
"prompt": "What are the side effects of metformin?",
"model_type": "llama"
})
response = client.invoke_agent_runtime(
agentRuntimeArn='<your-agent-runtime-arn>',
contentType='application/json',
accept='application/json',
payload=payload.encode('utf-8')
)
result = response['response'].read().decode('utf-8')
print(result)The standalone version and the AgentCore runtime version deploy the same agent in different ways. The following sections describe what each path provides.
The standalone version runs on Amazon ECS with AWS Fargate. You define ECS task definitions and service configuration, set auto scaling policies, configure IAM roles per service, and set up observability through Amazon CloudWatch. Deployment uses a Docker build, an Amazon ECR push, and an ECS service update. This path gives you full control over container configuration, networking, and scaling behavior. The agent code lives in healthcare_agentcore.py, integrates with Amazon Bedrock, Amazon SageMaker AI, and the containerized backend, and uses Amazon OpenSearch Service for vector search.
The AgentCore runtime version runs the same healthcare_agentcore.py agent code with the AgentCore decorator pattern. AgentCore runtime provides container orchestration, session-based scaling, identity management through IAM integration, and observability through built-in tracing and logging. Deployment uses a single command (agentcore deploy). The model integration (Amazon Bedrock, Amazon SageMaker AI, containerized backend) and vector search (Amazon OpenSearch Service) remain the same as the standalone version.
Both deployment approaches have distinct advantages. Amazon ECS with AWS Fargate provides full control over container configuration, networking, and scaling policies, suitable for teams with existing container operations expertise or specific infrastructure requirements. Amazon Bedrock AgentCore runtime is suited for teams that prefer managed infrastructure and want to focus primarily on agent logic development.
Regardless of the deployment path, the following elements remain unchanged when migrating from the standalone version to AgentCore runtime:
BedrockAgentCoreApp decorator + existing code).To avoid incurring future charges, delete the resources you created when you no longer need them. If you plan to continue using the deployed agent, no action is required.
agentcore remove allThen deploy again to tear down the AWS resources:
agentcore deployaws sagemaker delete-endpoint --endpoint-name healthcare-agentcore-endpoint-1 --region us-west-2aws opensearch delete-domain --domain-name healthcare-vector-store --region us-west-2In this post, we showed how to migrate a multi-model healthcare AI agent from self-managed Amazon ECS with AWS Fargate infrastructure to Amazon Bedrock AgentCore runtime. The migration required no changes to the core agent logic. The same healthcare_agentcore.py file orchestrates across Amazon Bedrock, Amazon SageMaker AI, and a containerized model server. It runs on AgentCore runtime with the addition of the AgentCore decorator pattern (BedrockAgentCoreApp, @app.entrypoint, and app.run()). For healthcare teams, this pattern directs specialized biomedical queries to a domain-specific model such as BioM-ELECTRA-Large-SQuAD2 on Amazon SageMaker AI. It routes broader medical reasoning to a foundation model such as Llama 3.1 70B Instruct by Meta on Amazon Bedrock. Together, these backends support a range of query types.
For teams that choose managed infrastructure, AgentCore runtime handles container orchestration, scaling, identity management, and observability. You can focus on agent logic development instead. The framework-agnostic design supports a wide combination of models and agentic frameworks, making this migration pattern applicable across industries including healthcare, financial services, and manufacturing.
To get started, clone the sample-healthcare-agent-with-agentcore-on-aws GitHub repository and follow the deployment steps in this post. To understand the standalone implementation that this post migrates from, see Agentic AI with multi-model framework using Hugging Face smolagents on AWS. If there are questions about getting started with Amazon Bedrock AgentCore, speak with an AWS generative AI Specialist.
Sanhita Sarkar, PhD, drives global AI/ML and generative AI partner solutions at AWS. She brings extensive leadership experience across edge, cloud, and data center environments, holds several patents, has published research papers, and serves as chair for technical conferences.
Many applications export their metrics directly to Prometheus. If you’re unfamiliar with Prometheus, in a nutshell it’s a time-series database for storing metrics, like counters and histograms. Applications that store their metrics in Prometheus typically use a popular Prometheus client as part of the integration.
Now that OpenTelemetry is a graduated CNCF project, many companies are now increasingly looking to move to OpenTelemetry to add more signals beyond metrics to their observability architecture. Logs and traces are popular additions for getting further insight into how applications behave. Profiles are also starting to become a popular fourth telemetry signal for even deeper understanding.
This can create a migration hurdle - how can we migrate our applications from one system to another for metrics without having a single cut-over event? To de-risk any migration an incremental approach would be preferred, where metrics are exported to both systems for a period of time so that “before” and “after” states can be compared and checked to ensure there is no loss of production visibility in either system for observing metrics or driving alerting.
The
latest release
of the OpenTelemetry Prometheus exporter for .NET allows you to take this exact
approach with your production metrics. You can use the
.NET Meter class
from your application and framework code to collect metrics and export them to
both Prometheus and another exporter, such as the
OTLP exporter, provided by the
OpenTelemetry.Exporter.OpenTelemetryProtocol
NuGet package.
The Prometheus exporter is effectively a Prometheus client library written on top of the OpenTelemetry SDK, exposing an HTTP scrape endpoint in your application to allow a Prometheus server to collect metrics from your application at regular intervals. The exporter implements the OpenTelemetry Prometheus specification (matrix) and implements all of the documented text exposition formats (except OpenMetrics 2.0, which is still experimental) to allow for compatibility with OpenMetrics.
flowchart LR
subgraph APP["Application"]
AC["Application code"]
SDK["OpenTelemetry SDK"]
PE["Prometheus exporter"]
OE["OTLP exporter (Client)"]
EP["GET /metrics HTTP endpoint (Server)"]
AC -->|"Generates metrics"| SDK
SDK -->|"Feeds metrics"| PE
PE -->|"Serves metrics as text/plain"| EP
SDK -->|"Feeds metrics"| OE
end
P["Prometheus (Client)"]
OTB["OpenTelemetry Backend (Server)"]
P -->|"HTTP GET /metrics (scrape request)"| EP
EP -->|"Metrics response (text format)"| P
OE -->|"OTLP export request"| OTB
OTB -->|"OTLP response/ack"| OEBy using only the Meter class alongside the Counter<T>, Gauge<T> and
Histogram<T> instruments in your .NET application code metrics can be
collected without needing to use both the .NET OpenTelemetry SDK and a dedicated
Prometheus client.
public class BlogPostComments
{
private readonly Meter _meter;
private readonly Counter<long> _likes;
public BlogPostComments(IMeterFactory meterFactory)
{
_meter = meterFactory.Create("OpenTelemetry.Blog");
_likes = _meter.CreateCounter<long>("blog_post_likes");
}
public void BlogPostLiked(long id) =>
_likes.Add(1, new KeyValuePair<string, object?>("post_id", id));
}
It’s then a small amount of code to configure the OpenTelemetry SDK to export
your metrics to both Prometheus and over OTLP to a backend that supports
OpenTelemetry by adding the
OpenTelemetry.Exporter.Prometheus.AspNetCore
NuGet package to your project.
using OpenTelemetry;
using OpenTelemetry.Exporter;
using OpenTelemetry.Metrics;
using var meterProvider = Sdk.CreateMeterProviderBuilder()
.SetResourceBuilder(CreateResourceBuilder())
.AddMeter("OpenTelemetry.Blog")
.AddOtlpExporter()
.AddPrometheusExporter()
.Build();
Your application will also need to expose the HTTP scrape endpoint that
Prometheus will use to collect metrics from your application. This can be done
by adding the UseOpenTelemetryPrometheusScrapingEndpoint extension method to
your IApplicationBuilder in the Configure method of your Startup class.
For example:
var builder = WebApplication.CreateBuilder(args);
// Configure services here
var app = builder.Build();
// Configure other middleware here
app.MapPrometheusScrapingEndpoint();
app.Run();
Using the Meter APIs to export metrics makes your application code more
portable and uncoupled from Prometheus specific APIs. This allows you to remove
any Prometheus client library dependencies from your application code. As well
as making your code ready for use with the OpenTelemetry ecosystem, it also
opens up the ability for you to use other .NET ecosystem tooling such as the
dotnet-counters
tool to view metrics.
If your application only uses a native Prometheus client such as
prometheus-net today then
you will need to gradually migrate to using the Meter APIs first. How long
this migration will take will depend on the complexity of your existing
Prometheus instrumentation and the resources available to you to make the
appropriate changes.
Some challenges you may encounter during this migration may include the
following Prometheus features which do not have direct equivalents in the
Meter APIs, and are therefore not supported:
Alternatively if you only have a Prometheus server and no OTLP compatible backend and only want to export metrics, Prometheus itself has opt-in support for ingesting metrics pushed to it over OTLP.
First ensure that you run Prometheus with the --web.enable-otlp-receiver
command line flag.
Then configure the OTLP exporter similarly to the code snippet above, but in this case you wouldn’t need to use the Prometheus exporter as well. Also note that the OTLP exporter specifies a base path for the metrics OTLP endpoint and uses HTTP/protobuf as the protocol for the OTLP exporter.
using OpenTelemetry;
using OpenTelemetry.Exporter;
using OpenTelemetry.Metrics;
using var meterProvider = Sdk.CreateMeterProviderBuilder()
.SetResourceBuilder(CreateResourceBuilder())
.AddMeter("OpenTelemetry.Blog")
.AddOtlpExporter((options, _) =>
{
options.Endpoint = new Uri("http://prometheus:9090/api/v1/otlp/v1/metrics");
options.Protocol = OtlpExportProtocol.HttpProtobuf;
})
.Build();
This approach allows you to push metrics to Prometheus with the OpenTelemetry .NET SDK over OTLP without depending on a Prometheus client library in your application code.
You can find a complete example for this approach in the Getting Started with Prometheus and Grafana sample in the OpenTelemetry .NET repository.
With minimal runtime overhead, the application can both push OTLP metrics and have Prometheus metrics pulled, allowing for both systems to be used in parallel until such time that you decide to go all-in with an OpenTelemetry-compatible backend for your metrics.
This customer ships financial products to millions of users across dozens of markets, and growth shows no sign of slowing. Sustaining that pace is an engineering problem before anything else, and the company's engineers lean on AI coding agents to do it.
That puts inference on the critical path of how fast the company ships, rather than inside any single customer-facing feature. The workload runs on GLM-5.2, the mixture-of-experts model built for long-horizon coding and agentic work, served on Together. Traffic follows the working day: spiky, concentrated in engineering hours, and it climbs every time another team adopts agents into its workflow.
The customer came to Together after running coding workloads with other inference providers, and first consolidated onto our earlier dedicated offering. That offering worked, but wasn't built for how this workload actually behaves. The coding-assistant traffic isn't steady; it's peak-load and relatively low-TPS, concentrated in engineering hours, with sharp bursts in concurrency and prompt size as more teams put agents into their daily workflow. That shape is precisely why concurrency, not raw throughput, was the design priority when the workload moved to GLM-5.2.
Under the earlier model, absorbing that kind of burst meant someone had to see it coming. Teams ready to move agents into their daily workflow often waited on capacity rather than provisioning it, and the customer's platform team absorbed the coordination for every one of them, filing requests and sizing clusters. The team worked to plan ahead, but planning stopped working once adoption became unpredictable in both timing and size. You can't forecast a burst that's driven by a hundred different engineering teams independently deciding to lean on their coding agent harder this week.
When capacity is provisioned to yesterday's forecast and traffic is genuinely spiky, prefill capacity and KV cache headroom get exhausted exactly during the burst, which is when it matters most, producing exactly the kind of multi-minute request queuing that agentic coding workflows can't tolerate. Fixing that after the fact, versus giving the customer's own teams the ability to see load and scale ahead of it, is the difference between a coordination problem and an infrastructure one.
The customer set requirements for the Together team around autonomy, in addition to raw performance, and the workload's own shape makes clear why. The coding-assistant traffic runs at ISL p50/p90/p95 of roughly 81K/163K/178K tokens and RPS p50/p90/p95 of 3/6/7, a peak-load, relatively-low-TPS pattern where bursts in concurrency matter far more than raw tokens-per-second. That's the backdrop for what the customer asked of Together:
Dedicated Model Inference exposes the full endpoint lifecycle: creation, sizing, scaling policy, and configuration changes. The customer's infrastructure team used exactly this when a migration reshaped their GLM 5.2 endpoint, shifting toward fewer, larger replicas, same total footprint, different ratio of replica count to chips per replica. When that re-shape hit near-100% prefill capacity a few days later, with requests queuing one to three minutes and decode throughput collapsing to roughly 5 tokens per second, the fix wasn't a new deployment or a ticket back to Together. It was a live configuration change: restoring the tuned cache-session-aware routing policy in place of DMI's default cache-aware-by-hash policy, and widening the max-inflight-per-worker threshold. All of it was pushed same day with zero downtime.
Programmatic access to endpoint usage and performance data is how the root cause was found. Together API Support traced a single 192-second slow request end-to-end through the metrics data and found it wasn't compute-bound, and had spent almost the entire span queued behind a 2.3M-token pending-prefill backlog from other requests, not its own 250K-token prompt. That's the specific value of self-serve observability: the customer's own team diagnosed a queuing problem, not a capacity problem, without waiting on Together to pull logs.
Dedicated Model Inference gives users self-serve access to frontier open-source models, as well as performance-aware configurations to help customers opt for any combination of TTFT, TPS, TPM, and other metrics. The customer's team works closely with Together's forward-deployed engineers to continuously optimize these configurations as its coding agent use evolves.
This showed up as the GLM 5.1 to GLM 5.2 and context-length iterations transitioning on the same account and endpoint pattern, with no renegotiation involved. It also showed up as a deliberate configuration trade-off the customer's team made themselves: given their traffic profile, they evaluated a 1M-context configuration and turned it down, because doubling context to 1M would have cut the concurrency headroom their peak-load, low-TPS workload actually depends on. The team chose to stay at 256K/512K instead.
The coding-assistant relationship predates the GLM 5.2 production endpoint by several months. Together's Solutions Architecture team had already built dedicated load-testing infrastructure modeling the customer's actual usage pattern, initially validated against an earlier GLM release.
That groundwork carried straight into GLM 5.1. Early on, the customer's project lead asked over a weekend for a checkbox-style concurrency test of GLM 5.1 across 8 to 16 B200s, explicitly for the coding use case and distinct from earlier tests that had been consumer-facing and latency-focused. Together turned the test endpoint around the same day, and GLM 5.1 passed the bar and moved to production: two dedicated endpoints, split by accessibility, running as the customer's internal developer-facing coding assistant.
Before DMI, changing a config or adding capacity meant routing through Together: filing a request, sizing a cluster, waiting for a redeploy. Every engineering team that wanted to adopt agents added to that same queue, so the customer's platform team ended up coordinating on behalf of the whole organization.
On DMI, that entire flow moved in-house:
The deployment stopped being a single endpoint and started being a surface for innovation. That pattern is now repeating as a pipeline, not a one-off. The customer's team is already scoping a dedicated GLM 5.1 node in a new region, sized against a real production workload. It's a different shape of workload than the original coding assistant: a chat-style customer-support NLP workload rather than long-horizon agentic coding, landing on the same infrastructure and provisioning pattern.
In August 2026, Hacktron reported what looked like a remote code execution (RCE) vulnerability in Next.js image optimization. Their investigation found that the vulnerable code was not in Next.js itself, but upstream in libheif, an AVIF image decoder used by Next.js, ImageMagick, WordPress, sharp, and much of the web.
Shortly after Hacktron notified us, we worked with them to reproduce the RCE against a current Next.js build and disclose it to the maintainers of sharp, libvips, and libheif. We then deployed a platform-wide mitigation on Vercel and started working with the maintainers on a fix.
Next.js image optimization lets applications resize and optimize images through the <Image> component (next/image). For AVIF images, the image-processing dependency chain is as follows:
<Image> invokes /_next/image,
/_next/image calls sharp
sharp calls libvips
libvips uses libheif to decode the image
That meant the vulnerable code was not in Next.js, but it was still reachable through Next.js image optimization. A malicious AVIF image sent to the image optimization endpoint would invoke libheif through sharp and libvips.
As such, one obvious mitigation was to disable AVIF optimization in Next.js. Malicious AVIF images would then stop at the image optimization endpoint instead of being passed through sharp and libvips to libheif. The exploit would not propagate upstream.
However, only mitigating Next.js, without an upstream fix, posed a disclosure problem.
After we worked with Hacktron to successfully reproduce the issue, we rolled out a platform-wide mitigation on Vercel and reached out to the maintainers of sharp, libvips, and libheif to disclose the vulnerability and begin working on a fix.
Here is the timeline:
August 11-12: Hacktron reported the issue to Vercel; Hacktron and Vercel reproduced the RCE with a working proof of concept.
August 13: Vercel applied a platform mitigation through its Image Optimization Service.
August 19: The Next.js team met with the libvips maintainer and began coordination across sharp, libvips, and libheif.
August 24: Next.js informed its security partners.
August 25: Next.js published a security release that disabled AVIF optimization.
The Vercel security team contacted the maintainers of sharp and libvips by email, and opened coordination with libheif through a GitHub Security Advisory. Hacktron had also submitted vulnerability and exploit details to libheif. On August 19, the Next.js team met with the libvips maintainer and aligned on the path forward across sharp, libvips, and libheif. The libheif maintainer continued remediation through Hacktron’s GitHub Security Advisory.
On August 24, Next.js informed its security partners of the libheif vulnerability and its impact on Next.js (partner notifications are a routine part of Next.js’ security release process).
On August 25, six days after the August 19 meeting, the libheif maintainer released v1.23.2, which remediated the RCE.
Securing Vercel and its customers was straightforward: all Next.js image optimization requests on Vercel go through a central Image Optimization Service. Therefore, we disabled AVIF optimization and resizing in that central service. Any incoming AVIF images were not passed to libheif for decoding and RCE was not possible on Vercel.
Protecting self-hosted applications required a Next.js release. On August 25, Next.js published a security release that had originally been planned to address a separate issue. After coordinating an upstream fix, we bundled the AVIF mitigation into that release and shipped it a day earlier than planned. The release disabled AVIF optimization and resizing in Next.js; given that the patched libheif release was still propagating downstream, this was the most timely option. We also published a security advisory to communicate the issue’s severity.
The volume of OSS vulnerabilities discovered continues to increase, and the numbers are overwhelming:
In 2026, the CVE program has published more than 35,000 CVEs.
Private vulnerability reports on GitHub grew from 500 per week in January to 3,000 per week in May.
GitHub also reported 1,560 reviewed advisories in May 2026, the highest monthly volume in the advisory database’s history.
As LLMs accelerate vulnerability research, we expect to see more upstream vulnerabilities like the libheif RCE surface across the OSS ecosystem. There have been a higher number of Next.js security releases in recent months, and we expect that trend to continue as we mitigate new vulnerabilities that both we and the research community uncover.
We are committed to proactively finding vulnerabilities before attackers, responsibly disclosing everything we find, and collaborating with researchers and maintainers on fixes.
Thanks to Hacktron for responsibly disclosing the AVIF vulnerability, working with us to reproduce the issue, and coordinating with the upstream maintainers through remediation.
We also want to thank the maintainers of sharp, libvips, and libheif. Their work on the upstream fix made coordinated remediation possible across the image processing dependency chain.
We work with a talented set of researchers to secure Next.js and other open source frameworks through Vercel's Open Source Bug Bounty. Anyone interested in contributing to the security of eligible frameworks is encouraged to participate there.
If you want to get the most out of coding agents in your organization, you need to stop guessing how well your agents are performing, and start measuring.
There are a couple of ways to do this. The typical approach is DORA metrics: PR merge rate, cycle time, defect rate, time to fix errors, etc. If those metrics are all going in the right direction as you increase agent usage, it's a good signal you are getting value from coding agents.
There’s a second approach though, that you should also consider: directly scoring coding agents using LLM-as-a-judge. Since agents provide a complete digital record of their work, you can examine and grade past sessions, see where they are deficient, and adjust going forward.
Scoring forms the basis of agentic self-improvement, where observer agents automatically suggest changes to improve agent ROI based on past scores of how the factory is performing.
There are a few prerequisites for setting up an effective scoring system. I’ll illustrate the primitives using the built-in scoring infrastructure in Warp Factories, but you can also create something similar on your own.
Here is the tl;dr:
Let’s take a closer look:
First, you need a record of prior agent traces that your scorers can grade. These traces should include not just the agent conversation, but the agent’s entire “input and output;” and they should be stored in the cloud and be accessible via API so agents can analyze them.
“Inputs” are prompts, tool calls and MCP results, input images, etc. “Outputs” should include all artifacts created by the agent like PRs, specs, screenshots, etc; anything that would be helpful in judging whether the agent did its job. In Warp Factories we automatically store all this info and make it API accessible (potentially in a company’s own storage). Depending on your factory approach, you may have to do some infrastructure work to set this up.
Second, you need a way of defining and triggering “scoring agents.” A scoring agent takes a prior agent trace as an input and returns a grade. Each scorer typically focuses on a single dimension like cost or quality, and is defined by a prompt, classification instructions, and a judge model to use. You’ll also need a place to store and view the aggregate scores. Again, this is built into our factory infra; if you are building your own you’ll want to use some sort of cron-based cloud agent to score prior runs.
You can define scoring agents along different dimensions:
For example, here’s the definition for a custom scorer that checks for redundant test creation, a common failure mode we were seeing in our internal factory.
It contains a judging prompt (our full redundant tests scoring rubric for reference):
Along with a set of output classifications – what counts as a “pass” –
and a sampling rate, indicating what percent of runs to score.
When a scoring agent runs, it loads an agent trace, brings all its inputs and outputs into context, and then prompts an LLM to judge the run. The output is a classification like in the above example.
You won’t necessarily want to score every run, since scoring itself costs money. Instead, you’ll want to (third) decide on a sampling strategy. It could be percent-based, it could be classifier based (e.g. “score all my front-end tasks”), etc. For our internal factory, scoring currently accounts for about 3% of total token costs – that’s a reasonable amount to get visibility into agent performance.
Over time, (fourth) you’ll build up a corpus of your scored runs. At the simplest level, you can use these just like DORA as another measurement of the efficacy of your factory. You can graph how the metrics are changing over time, catch regressions when they get worse, etc. Depending on how your factory is set up, you can try to correlate changes to models, skills and context with improvements (and regressions).
In the above graph you can see that our scorer thinks we are mostly avoiding redundant tests, but there are a few failing runs every day. To investigate, you can click into the failures and examine what the coding agent did and also examine the scorer run itself, since it’s just another agent, to understand why it thinks these coding agent runs produced redundant tests. You may notice patterns, and then adjust the skills which drive your agents, so that they write tests more sparingly.
Once you get a feel for checking your scorers by hand, you’ll probably want to (fifth) automate how they are used, and create an actual learning loop. In Warp Factories we call this “self-improvement,” and you can learn more about it here. The tl;dr is that scorers can be input into another agent loop that synthesizes their output in batch and creates updates to the factory definition automatically.
Scorers also (sixth) form the basis of more advanced optimizations like benchmarking, where you test different model configurations against your factory to optimize its cost and performance. If you want to learn about benchmarking, check out this post.
In sum, if you aren’t currently scoring your coding agents, you are missing a crucial layer of visibility into how they are performing and how you might improve them. It’s a bit of work to set up, but in an age where more and more of your company’s software production depends on how efficiently your agents work, it’s well worth the effort to gain that visibility.
If you are interested in learning more about Warp Factories and how they are helping companies scale development on open, observable infrastructure, you can request early access here. We are offering up to $10k in usage to qualified companies.
Meet Neki: sharding for Postgres. Neki allows applications to connect to massive, sharded databases over a single connection string. This post takes apart the architecture from the bottom up, one piece at a time, starting with what's underneath all of it.
Neki is built as a sharding and scaling solution for real Postgres. It's not a fork, nor a wire-compatible reimplementation, nor a MySQL sharding idea wearing a Postgres label. Neki uses ordinary PostgreSQL instances that store rows in Postgres data pages using MVCC, carry out transactions, and work as you would expect with psql and other Postgres drivers. Neki builds around those instances to let you shard them, scale them, and manage them as one database.
Let's take a look:
Using vanilla Postgres means Neki needs a way to run and manage each instance. That includes starting and stopping Postgres, owning its data directory, and configuring replication so a new instance can join a shard. PostgresManager handles this coordination, running as the first process in the Postgres container and managing the postgres process directly.
Postgres uses a separate backend process for each connection and limits how many can be open at once. Neki’s Sidecar sits in front of each instance and pools connections, letting many client connections share fewer Postgres backends.
The Router, which is the component that accepts external client connections, communicates with the Postgres nodes via these Sidecars.
It also reports each Postgres instance's health and whether it is a primary or replica, so the rest of the cluster knows whether it can receive write queries.
The pool doesn't treat every connection the same way. The length of time a connection is checked out for use varies depending on what it's being used for. A multi-statement transaction holds on to its connection until commit or rollback. A session-scoped advisory lock needs a connection of its own, because the lock has to outlive whatever transaction is open at the time and can't share that connection. Everything else checks a connection out and hands it back the moment the statement finishes.
The Sidecar knows which of the three to use because the Router sends the necessary information with the query: autocommit, an open transaction, or a session that has to stay on one backend.
Each Postgres instance gets its own Sidecar and PostgresManager pair. Real deployments need more than one instance: a primary and its replicas. Neki calls that group a shard, the unit it splits data across. It's always advised to run a shard with a primary and 2+ replicas for high availability, as well as for additional read query capacity.
A shard is considered one Postgres cluster. Its replicas are physical copies of the primary, so they share a catalog and the same object identifiers.
Object Identifiers (OIDs) are how Postgres tracks objects internally, rather than by name. A client reads a column’s type OID off the wire to interpret its bytes and may cache that OID for later re-use. A custom type therefore needs to carry the same OID no matter which shard answers the query. Independent shards can assign that type different OIDs, so Neki designates one shard in the entire Neki cluster as the authoritative shard. This shard is the source of truth for translating custom type OIDs in responses from other shards to match. It ensures OIDs are consistent across the many shards of the Neki cluster.
The authoritative shard's Sidecar also watches for schema changes and reports them to the Routers. This keeps the Routers' view of the schema current when a table is renamed or a column is dropped.
In a distributed system, instances can fail independently while the rest of the system lives on. Neki is no different. A primary or replica can go down at any moment while its fellow instances on the shard are healthy. The Admin's job is to detect failures, promote a replica, and maintain each shard’s durability policy.
It health-checks every Sidecar, tracks replication lag for each replica, and decides when a shard needs a new primary. When a primary goes down, it coordinates an emergency failover, promoting a replica to take its place. It can also coordinate a planned switchover, which are needed for intentional node resizes and version upgrades. In both situations, Admin uses pg_rewind to bring diverged instances onto the new primary’s timeline, copying only the data that changed since the timelines diverged.
Each shard has a durability policy that determines when a commit is acknowledged:
Postgres enforces whichever one is configured, using its own synchronous replication machinery. The Admin keeps that configuration correct as replicas join or leave shards, or a failover moves the primary to a different zone.
Much of Admin’s work, however, doesn’t involve changing the primary. It repoints replicas to the correct replication source and corrects roles when Postgres and the topology disagree.
Neki’s components need to be deployed, updated, and replaced when their machines fail. Neki is built Kubernetes-first, and the Operator manages this full lifecycle.
The Operator models a cluster as a hierarchy. A cluster owns routers and shards, and each shard owns the pods running its Postgres instances and Sidecars. When the Neki cluster configuration changes, the Operator works out which pods need to be created, updated, or removed.
How it replaces an instance depends on whether that instance is still running. For a live instance, the Operator builds a replacement and confirms it has caught up before deleting the old one. If a node fails and loses its ephemeral storage, the Operator rebuilds the lost instance from scratch once its safety checks pass.
Admin and the Router handle the database side of those disruptions. Admin coordinates a switchover for planned primary replacements or a failover when a primary goes down. The Router can buffer queries that are safe to retry while a healthy primary becomes available.
We've talked a lot about how the Neki cluster operates and handles failure internally. What we've yet to dive into is how applications use the thing!
The Router is the entry point for clients connecting to a Neki cluster, presenting a single Postgres wire-protocol endpoint to connect to a (potentially) massive sharded database. Applications use Postgres drivers to send SQL and open transactions without managing connections to individual shards.
Authentication and role checks are done as if it were the Postgres instance itself, and the protocol's own extended-query flow and prepared-statement lifecycle are all built into the Router.
Once a query arrives, the Router runs a Postgres-compatible parser against the authoritative shard's catalog, plans it against the current sharding layout, and sends it to whichever Sidecar needs to run it over gRPC.
Not every query can run on a single shard. A join may need data from several shards or an aggregate may need to read from all of them. The Router coordinates that work as a distributed query.
Whenever possible, it leaves the work to the Postgres instances. If both sides of a join are on the same shard, the Router sends the join to that shard. When a join needs to run across shards, the Router executes it itself, choosing between nested-loop, hash, and merge joins based on cost estimations.
Note
Read more about Routers, parsing, and sharded query planning in our other blog, The lifecycle of a sharded Postgres query.
Earlier, we covered how Admin promotes a new primary during a switchover or failover. If that happens, the Router can buffer queries, giving the Admin time to complete the handover. For queries that can safely be retried after failing against a primary, the Router buffers the query and waits, for a fixed time, for a healthy primary. Once a healthy primary is available, the Router releases queued queries gradually.
Router, Sidecars, and Admin all need a consistent picture of which shards exist, what key ranges they own, and which tables are sharded at all. If the Router's copy is wrong, a query can land on the wrong shard. This is all specified with a Data Topology, and etcd holds the single, authoritative copy of it. When the Data Topology changes, the Router, Sidecars, and Admin pick up the updated configuration without a restart or manual synchronization.
The Data Topology defines shard groups, named sets of physical shards, each owning a range of routing keys. Each table belongs to a shard group. Shard indexes specify the columns or expressions and the strategy used to turn row values into routing keys. Those keys determine which shard receives each row.
As a database grows, its layout may need to change. Tables need to be imported, shards need to be split, and schemas need to change all while applications keep using the database.
Neki's Replicator handles the data movement behind all such operations. It runs as a separate process colocated with a shard's Sidecar and Postgres. It is responsible for copying existing rows to new destinations, and also keeping the data current by decoding changes from a Postgres logical replication stream and applying them as SQL.
Three workflows use the Replicator:
Once the data has been copied and the destination is caught up, the workflow switches from the original tables or shards to their replacements. This is the cutover. The Router uses the same buffering mechanism that handles primary changes for this step. It buffers queries during that switch and releases them afterward.
Together, these components let Neki scale Postgres horizontally while presenting a single database to applications.
Neki is in Platform Preview right now.
Start a Neki cluster today: build on it from scratch, or import an existing Postgres database.
When DuckDB-Wasm was launched in 2021, databases could not be persisted: everything lived in the Wasm heap and vanished when the tab closed. Keeping data meant serializing tables to Parquet, storing the bytes in IndexedDB, and re-registering them on the next page load. This was doable, but had to be handled at the application layer and was not offered out of the box by DuckDB-Wasm.
Modern browsers (since March 2023) now ship the Origin Private File System (OPFS), a per-origin, sandboxed file system with random-access reads and writes. DuckDB-Wasm (tested with versions 1.32.0 and 1.33.1-dev64.0) can use it as a storage backend, as described in the DuckDB documentation: a database opened at an opfs:// path survives reloads and browser restarts.
The following call opens a database file in OPFS:
await db.open({
path: 'opfs://analytics.duckdb',
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
The result is a regular .duckdb file with a write-ahead log and checkpoints that survives page reloads and browser restarts.
At the time of writing, the build that npm serves as
latest(1.33.1-dev57.0) creates the OPFS files but never writes to them, so nothing persists. It canonicalizes the path toopfs:/analytics.duckdbwith a single slash, which no longer matches the OPFS handle. Pin 1.32.0 or use thenexttag (1.33.1-dev64.0 or later).
The setup is the same as for any DuckDB-Wasm application: pick a bundle, start a worker, instantiate the database. The only new part is the open call, marked below. The import resolves to whichever version is installed, and getJsDelivrBundles() fetches the matching worker and .wasm files, so install a version that persists correctly: npm install @duckdb/duckdb-wasm@1.32.0 or @next.
import * as duckdb from '@duckdb/duckdb-wasm';
const bundles = duckdb.getJsDelivrBundles();
const bundle = await duckdb.selectBundle(bundles);
// Worker scripts must be same-origin, so wrap the CDN worker URL in a Blob
const workerUrl = URL.createObjectURL(
new Blob([`importScripts("${bundle.mainWorker}");`], {
type: 'text/javascript'
})
);
const worker = new Worker(workerUrl);
const db = new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker);
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
URL.revokeObjectURL(workerUrl);
// NEW: open a persistent database in OPFS instead of the default :memory:
await db.open({
path: 'opfs://analytics.duckdb',
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
const conn = await db.connect();
await conn.query(`
CREATE TABLE IF NOT EXISTS transactions (
id BIGINT,
ts TIMESTAMP,
merchant VARCHAR,
category VARCHAR,
amount DECIMAL(10, 2)
);
`);
await conn.query(`INSERT INTO transactions VALUES (1, now(), 'Coolblue', 'electronics', 49.95)`);
await conn.query('CHECKPOINT');
const result = await conn.query('SELECT count(*) AS n FROM transactions');
console.log(result.toArray()[0].n);
Reload the page and run the same code. The CREATE TABLE IF NOT EXISTS statement finds the existing table and does nothing, the insert adds a second row, and the count prints 2. There is no sync step, no export, no localStorage key to remember. The opfs:// prefix tells DuckDB-Wasm's file system layer to resolve the path against the origin's private file system instead of the in-memory Emscripten file system.
Opening the database creates the database file and its .wal in OPFS. Builds from 1.33.1-dev64.0 onward also create two empty helper files, .wal.checkpoint and .wal.recovery, that DuckDB uses during checkpointing. The .duckdb file is a regular DuckDB database file. If you pull it out of OPFS (shown below) and open it with the CLI or the Python client, it works.
The same prefix works for data files. A common pattern is to load a remote dataset once, keep it in the persistent database, and cache derived results as Parquet files in OPFS. The example below uses the TPC-H orders table (scale factor 0.01, about 1,500 rows) that the DuckDB web shell serves:
await conn.query(`
CREATE TABLE IF NOT EXISTS orders AS
SELECT * FROM 'https://shell.duckdb.org/data/tpch/0_01/parquet/orders.parquet';
`);
await conn.query('CHECKPOINT');
DuckDB-Wasm reads the remote file with HTTP range requests. Because the table is created with IF NOT EXISTS, the file is fetched only on the first page load; on later loads the table comes from OPFS and no request goes to shell.duckdb.org. You can see this in the browser's Network tab, which lists the range requests on the first load and stays quiet afterwards, or in DuckDB-Wasm's own logs: the ConsoleLogger passed to AsyncDuckDB records each HTTP read, so the absence of those log lines on a reload confirms the data is served entirely from OPFS.
With the data local, an aggregation can be written to a Parquet file in OPFS and read back later:
COPY (
SELECT o_orderpriority AS priority,
date_trunc('month', o_orderdate) AS month,
sum(o_totalprice) AS total
FROM orders
GROUP BY ALL
) TO 'opfs://cache/monthly_totals.parquet';
SELECT * FROM 'opfs://cache/monthly_totals.parquet';
Nested directories such as cache/ are created on demand. OPFS files are ordinary DuckDB file paths, so globbing, read_csv and the other readers work as usual. Reading and writing opfs:// paths from SQL needs one extra option on open(), described next.
With opfs: { fileHandling: 'auto' }, DuckDB-Wasm scans each statement for single-quoted 'opfs://...' literals, registers those files before execution (creating them and any missing directories if needed) and drops the handles afterwards. The option only takes effect when the database itself was opened from an opfs:// path. Without it, every file other than the database has to be registered by hand:
// Option 1: automatic registration of opfs:// paths found in SQL
await db.open({
path: 'opfs://analytics.duckdb',
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
opfs: { fileHandling: 'auto' },
});
// Option 2: manual registration (the default)
await db.open({
path: 'opfs://analytics.duckdb',
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
});
await db.registerOPFSFileName('opfs://cache/monthly_totals.parquet');
// ... run queries against it ...
await db.dropFile('opfs://cache/monthly_totals.parquet');
Automatic mode is convenient for one-off reads. Manual mode requires more code but avoids re-acquiring an OPFS access handle on every statement, which adds up for applications that run many small queries. A file can be held by only one handle at a time, so the DuckDB documentation recommends dropping registered files with db.dropFile() before another connection or database instance opens them.
DuckDB-Wasm writes to OPFS the same way native DuckDB writes to a local disk: through a write-ahead log and periodic checkpoints. What differs is that a browser tab is rarely closed cleanly, so the defaults that work on a desktop can leave you with a slow reopen.
DuckDB uses a write-ahead log. Committed transactions are appended to analytics.duckdb.wal first. The main file is updated at checkpoint time. A checkpoint happens automatically when the WAL grows past checkpoint_threshold (16 MB by default), when the database is closed cleanly, or when you run CHECKPOINT yourself.
In a desktop process, "closed cleanly" is the common case. In a browser tab, it is not: the user closes the tab, the phone kills the background page, the laptop lid goes down. None of these run your shutdown code reliably. Two rules follow from that.
Call CHECKPOINT after writes you cannot afford to lose. The DuckDB documentation is explicit about this: writes are flushed to OPFS by CHECKPOINT. Committed transactions are appended to the WAL, and DuckDB replays the WAL on the next open, but a browser tab can be terminated at any point, so a checkpoint is the only way to be certain that the data is in the main file.
Checkpoint per batch, not per statement. A large WAL also makes the next open slower, because replay has to happen before the first query. For an interactive app, checkpointing after each batch of user edits keeps both the data safe and the reopen fast:
await conn.query('INSERT INTO transactions VALUES (...)');
await conn.query('CHECKPOINT');
If you would rather not track batches, set the checkpoint threshold to zero once after connecting. DuckDB then checkpoints after every statement, which costs some write throughput but removes the question entirely:
await conn.query(`SET checkpoint_threshold = '0KB'`);
A clean shutdown looks like this:
await conn.query('CHECKPOINT');
await conn.close();
await db.terminate();
What happens when a tab is killed mid-transaction, and how to share one database between tabs, are covered in a follow-up post.
There is a second kind of durability to keep in mind, one that sits below DuckDB. OPFS is browser storage, not a hard guarantee. The browser can evict it when disk space runs low or when the origin has not been visited for a long time, and the user can clear it from the site's settings. Treat OPFS as a fast local cache for accelerating startup and persisting working state, not as your only copy of data you cannot lose. For durable storage, keep the source of truth somewhere stable and sync back to it: a DuckLake catalog, or plain files on object storage through s3:// paths.
Users will want to move their data to another device, back it up, or open it with a different tool. DuckDB-Wasm itself cannot move files into or out of OPFS yet, but the database is a plain DuckDB file and the browser's OPFS API lets you read it back as bytes:
await conn.query('CHECKPOINT');
const root = await navigator.storage.getDirectory();
const handle = await root.getFileHandle('analytics.duckdb');
const file = await handle.getFile();
// Offer as a download, upload to your backend, etc.
const url = URL.createObjectURL(file);
Or export from SQL to Parquet:
COPY transactions
TO 'opfs://export/transactions.parquet'
(FORMAT parquet, COMPRESSION zstd);
Combined with DuckDB's Parquet support, this allows preparing and cleaning data in the browser before uploading it to a server. And because the on-disk format is standard, the reverse works too: ship a pre-built .duckdb file with your app, copy it into OPFS on first launch, and open it. Users get a local dataset without an import step.
Lack of persistence was the main limitation of DuckDB-Wasm for a long time. With OPFS, DuckDB-Wasm can open a database file in the browser, commit transactions to a WAL, checkpoint, and reopen the same database after a reload. Three things make it work well: run CHECKPOINT after each batch of writes rather than after every statement, give users a way to download the database file, and read the limitations listed in the DuckDB documentation before shipping: one handle per file, and renames from SQL only work between two already-registered OPFS files.
With this, a local-first application no longer needs a server, IndexedDB wrapper, or custom serialization to keep analytical data between sessions. Try it in your own application, and share what you build on GitHub or Discord.
Included Health is an all-in-one healthcare platform that partners with employers and health plans to provide their employees and members with healthcare navigation to services like virtual primary care, behavioral health, urgent care, specialty care, and more. The product experience centers answering medical, financial, or administrative questions via Dot—an AI-powered healthcare guide built on top of a federated multi-agent architecture using Deep Agents and LangGraph.
Healthcare is one of the few domains where what a person asks for and what they actually need can be entirely different. A member asking "is an artery plaque scan covered by my insurance?" might, with a few follow-up questions, reveal that they are managing elevated cholesterol and have a family history of heart disease. The right response includes the dollar figure—but it may also mean recognizing an opportunity to encourage a conversation with a primary care physician.
Historically, health systems handled this kind of routing with structured navigation trees. That approach made complex needs manageable for software, but only by flattening them into a series of predefined decisions. As Kartik Darapuneni, Engineering Manager, described it: “For the member, it feels really rigid, and it’s just not a good experience.” The limitations become even more consequential when a conversation begins with “I’m having chest pain.” The system needs to recognize the potential emergency in the first turn, not after seven clarifying questions.
This is the broader tradeoff that has shaped software for decades. To scale, technology has typically had to standardize complex human situations around the average case. In healthcare, where context is often the difference between a merely correct answer and a helpful one, that tradeoff is especially costly.
LLMs, combined with an agent harness, change what is possible. They can process dense individual health records, reason about ambiguous needs, and ask clarifying questions without forcing members through predetermined paths. “LLMs addressed all three of those blockers all at once,” said Kartik. Conversations not only become more natural, software no longer has to choose between personalization and scale. Included Health built a healthcare experience that adapts to each member’s context, responding with the urgency, guidance, and next step that their situation calls for.
Included Health's production architecture centers on a main LangGraph graph they call the Dot supergraph. Within it, Dot acts as the primary conversational router for transactional interactions (e.g. handling coverage questions, billing inquiries) and also navigation to the right care point. A set of sub-workflows handle domain-specific member journeys including urgent care intake, appointment scheduling, finding a specialist, behavioral health, and more.
Different product teams at Included Health own different parts of this graph. Scheduling alone, for example, has to account for which services a member is eligible for, their coverage details, whether they're a primary member or dependent, and a range of clinical nuances.
Included Health added Deep Agents for consistency across those services. "Originally, you would jump into a different agent and suddenly it was a lot more short and brusque. It didn't have the same voice and tone," said Rohan Bhandari, Staff Machine Learning Engineer. With Deep Agents, the team created a global platform prompt for voice and tone that could be passed across all agents without each team having to manage it independently. When routing from one workflow to another, Deep Agent’s filesystem and built-in context management allow the outgoing agent to summarize the conversation and pass both the summary and a file path to the full conversation history for the receiving agent. This setup ensures members never have to repeat themselves across different agents owned by different teams.
For example, the shared coverage question skill: coverage questions don't necessarily arrive at the start of a conversation. A member could be mid-way through finding a specialist and want to know what it will cost. Before Deep Agents, handling this required threading a coverage capability through every sub-workflow's routing logic. Now, "we decomposed it into a platform sub-agent that all the Deep Agents can inherit. Meaning every agent can answer coverage questions," said Rohan.
LangGraph enables consistent composition and distributed development so each product team can build and own their service independently. Deep Agents adds the shared filesystem that keeps tone and behavior consistent as customer conversations move across those services.
Included Health gives agents clinical capabilities and services using Deep Agent skills. Each skill describes what a service is, when it's appropriate, when it isn't, and how to handle edge cases. For example, what to do when a dependent wants to book a service that has eligibility nuances.
The model uses progressive disclosure as a way to drive the right conversation for navigating a member. For example, if a member says they want to see a doctor, there could be 3 or more appropriate ways to help (e.g. virtual urgent care, virtual primary care, find an in-person doctor). The agent has a skill registry managed through a virtual filesystem, and up front it gets a short description of each skill. Upon invocation, the model decides which skills are relevant and can then load full skill files. With those skill files, it learns about the nuances and what questions to ask to best navigate the member (e.g. do they want a virtual or in-person visit? Is the issue they are describing acute or better managed through a long term provider relationship?).
Included Health supports third-party employer benefits in addition to its own services, and is working toward encoding those as skills too, to include the 20 to 30 benefits per employer plan.
We have a clinical team who reviews chats and confirms whether they agree with which care spot we sent a member to, given their issue," added Rohan. That feedback loop has allowed Included Health to tune skill definitions over time and stay above their target level of clinical routing agreement of 95%.
A distinctive aspect of Included Health's agentic system is how they incorporated human-in-the-loop to improve the experience for patients. "We think about LangGraph as our entire messaging platform," said Kartik. LangGraph's durable execution allows the agent to maintain full context across the conversation, supporting indefinite pauses and context retention. When the agent reaches a point of uncertainty, it pauses the graph, routes to a human member care advocate for a multi-turn exchange, and then resumes—with the agent holding the full context of what the human did and said.
This design reflects the long-lived nature of healthcare relationships. A member can come back to the same thread days or weeks later with a follow-up question, and the agent can pick up where they left off, including the full context of any human-assisted portions. "From a human perspective, they’re helping the agent get unblocked, as opposed to doing all of the work," said Kartik.
The architecture also leaves room for the next evolution: running a parallel agent thread while a human is handling a conversation, so the agent can do background research and surface recommendations to the care advocate in real time.
LangSmith annotation queues are central to how Included Health runs clinical oversight. Right now, every conversation goes into a queue for clinical team review. Reviewers assess whether the agent's navigation recommendation was correct, whether emergency guardrails triggered appropriately (or correctly did not trigger), and flag anything that needs follow-up. Those labels are exported from LangSmith into Included Health's data warehouse, where the data science team builds the operational metrics dashboards.
Multi-turn user simulation evals using LangChain's user simulation package became the safety net for architectural changes. The migration from standard agents to Deep Agents across the supergraph affected four product teams, all wary of breaking changes. "We were able to run our whole eval suite, see that we got, for the most part, better performance, and then we had the confidence to share that out to the other teams," said Rohan. Thanks to these evals, the migration happened in under 2 weeks, with no significant regressions and no team resistance.
Dot launched to clients in August, in what Rohan described as “the smoothest launch the team has seen in the past few years.” Early metrics are tracking in the right direction across three areas:
Interested in building production-grade agent systems with Deep Agents? Learn more about Deep Agents.
Part 1: Parsing, chunking, and vectorization
Some time ago, we set out to build the best semantic code search platform we could: a RAG pipeline that gives LLM agents precise, citable evidence from real repositories instead of whatever grep happens to surface. The eventual solution was JetBrains Context. We got it working, we got it into production, and we collected a lot of scar tissue along the way. In this series of posts, we’ll share the parts we wish someone had told us on day one.
Coding agents are undoubtedly the biggest technology leap for software development of our decade. Agents and frontier models are proving their aptitude in the face of seemingly insurmountable code complexity to produce ostensibly reliable code.
However, as more and more development processes become agent-driven, the agent’s efficiency and the quality of the produced code become increasingly important. The question is not so much about whether an agent can complete the task, as given enough time and token resources, it surely will, but rather how much time, effort, and steering is required for it to generate production-grade results. For large-scale code bases specifically, the agent would spend a great deal of time searching for the relevant pieces of code relevant for the feature it’s working on and pulling them into the context.
Attempting to locate the right code snippets, the agent will resort to traditional tools for code search such as keyword search and grep. These tools, however, are limited in that they require the agent to know in advance which exact text to search for. For example, an agent looking for where session tokens get refreshed cannot rely on the code helpfully containing the word “refresh”. To reason through abstract domains, the agent needs the ability to search for code by meaning, also known as semantic search. This is where retrieval-augmented generation (RAG) comes into the picture. If we can index the source code in a way that captures its semantics and then allow the agent to retrieve the relevant pieces on demand using free text search, we create an interface that plays to the agent’s strengths.
Like many great ideas in the agentic era, a native, prototype implementation is extremely simple. A well-evaluated production grade solution most certainly is not. In this series of blog posts, we want to share what is involved in making an effective RAG system, as well as the wrong turns we took in our journey to create our own: JetBrains Context. We’ll tackle each stage, from pre-processing to storage and agent integration, providing some more technical context and advice.
This first part of the series will cover the initial stages of the pipeline: parsing and chunking, where raw source files are divided into properly scoped units, and vectorization, where those units are transformed into a representation that supports semantic search.
Parsing and chunking is a critical pre-processing step in a good RAG solution, but it is often overlooked. In order to allow the LLM to embed or otherwise index the source code, we must first feed it the raw lines of code. This may sound trivial, and probably would be for small-scale demo projects. However, production-grade systems contain thousands of files, which, in turn, span hundreds or even thousands of lines. If anything, agents have compounded the problem, as they tend to be prolific writers, further inflating the codebase. Each file may contain multitudes of classes, fields, and methods, with varying degrees of relatedness among them.
Even if it were possible to fit these huge code files into an embedding model in their entirety, that expensive feat would ultimately be self-defeating. Because the entire file was embedded in a single unit, the search would return the entire file. This is counterproductive to the goals of agentic code exploration and navigation, which are mostly concerned with finding a specific function, symbol, or code snippet.
On the other hand, if we were to take the other extreme and granularly embed each separate line of code, we would be facing a problem of a different sort. These individual lines can be semantically insignificant without the surrounding context. A generic function name or comment does not merit embedding and will produce the wrong retrieval result. In a sense, we would not be able to see the forest for the trees, and the agent would be overloaded with multiple, often insignificant micro-results.
It is therefore imperative to find the right method to chunk or divide the code into groups that are properly scoped. Each group should include enough of the necessary context and represent common semantic meaning.
Chunking is a generic name for the technique of taking content that will be fed to the agent and dividing it into a set of chunks. A naive approach to chunking could be simply splitting a large file into groups with a fixed number of lines. However, if we were to take that approach, we would find the resulting groupings semantically wrong. Unrelated code pieces would be grouped together, for example, an import statement and some function content, leading to mistakes during retrieval.
To solve the problem, we can leverage the fact that every source file has a pretty well-defined structure. Take Java as an example – imports tend to be at the top of the file, followed by a class definition with an optional doc-comment preceding the header. The class will contain fields and methods, which in turn may also have their own doc-comments. Knowing about the conventions and rules that define the class structure allows us to perform smarter chunking and achieve the right balance of surrounding information.
Over the last 26 years, we at JetBrains have developed parsers that are smart enough to adjust for the various quirks, irregularities, conventions, and nuances of specific languages. Alongside other tools, these parsers form our internal JetBrains Code Engine platform on which JetBrains Context is developed. At the moment of this article’s composition, JetBrains Context supports parsing and structure-aware chunking for nine major languages: Kotlin, Java, Python, JavaScript, TypeScript, C#, PHP, Go, and Rust. For all other languages, our implementation simply falls back to naive, line-based splitting to ensure that any language or document can be indexed and searched.
The parser allows us to break source files into streams of syntax nodes that carry information about what they represent – comments, whitespaces, lists of modifiers, and so on. The chunking algorithm then consumes that stream and applies logic that decides the scope of a given chunk. Based on the node’s type and size, as well as its descendants, the algorithm makes a decision. If a node exceeds the size threshold but has no children, it will fall back to more primitive splitting strategies.
Some language-specific constructs are kept as single slices even if they exceed the preferred size. Prefixes such as documentation, annotations, visibility modifiers, and keywords are kept together with the declaration; suffixes (usually closing syntax) remain associated with the construct they close. There is also some language-specific cleaning, where, for instance, common and semantically meaningless Java annotations such as @NotNull or @Override are removed.
The algorithm bears some similarities to cAST, authored by Zhang et al. in 2025. Both our implementation and cAST retain the largest syntax units that fit, subdividing only the units that are too large, and grouping smaller adjacent units to avoid tiny chunks that are not usually semantically meaningful. The biggest difference is that we coded more language semantics into our implementation, keeping Python decorators together with definitions, KDocs next to Kotlin declarations, and so on.
After grouping, chunk normalization is performed, which involves:
Following the normalization procedure, the chunk is then passed to the next step – embedding – along with metadata that consists of a relative path, which gets embedded alongside the normalized chunk content.
It is hard to give a concrete answer as to what the input to the embedding model should look like. Chunk size matters, but as discussed before, bigger is not always better. Additionally, some metadata embedded alongside the code may be useful, while some may introduce noise that ultimately decreases search quality.
We opted to use an LLM-as-a-judge strategy to inspect the chunks as a part of the evaluation. The judge, using a chunk and the source file, considers whether the boundary makes sense. It looks for unexpected artifacts, such as detached documentation, orphaned closing syntax, or fragments of code that are cut through a meaningful construct. In addition, any changes to the source code processing pipelines also go through the full, end-to-end retrieval evaluation. We’ll get back to that evaluation pipeline in the following part of this series.
Having pre-processed the source code, we finally have text chunks that are hopefully just the right size and correctly grouped for semantic retrieval. Our next task is to transform these fragments in a way that will later allow us to support semantic search, through a process called vectorization.
With vectorization, an embedding model reads a piece of text and emits a fixed-length list of numbers (a vector), which amounts to a point in a space of a few thousand dimensions. Significantly, the model is trained so that texts with similar meaning land close together. Traditional search might miss the connection, but here, a function that flushes buffered write operations and one that drains a pending queue can end up near each other despite sharing no common keywords. The distance between vectors hence becomes a measure of relatedness. A query is turned into a position in the same space, and the results are whatever lies nearest to it.
Any attempt to vectorize a large codebase must take into account both cost and performance. A single embedding is cheap, but a large repository produces millions of chunks, which become millions of vectors that must be stored, held in memory, and compared against each incoming query. A vector of a few thousand dimensions in 32-bit floats weighs around 16 kilobytes, so a few million chunks add up to tens of gigabytes of index before any bookkeeping. At such a scale, the allocation of bytes per vector becomes cost-limited, and the leading question quickly shifts from “how accurate can we be?” to “what do we get per byte?” In other words, we need to find a way to reduce the cost while retaining as much search quality as possible.
There are two ways to reduce vector cost. The first is to keep fewer dimensions. Modern embedding models are trained so that a leading slice of the vector works on its own. The dimension loss is applied across several nested prefix lengths simultaneously, pushing the coarsest structure into the earliest dimensions. This means you can cut a vector short and renormalize it, and it still retrieves. Alternatively, you can keep every dimension and spend less on each one by sacrificing on precision and thus keeping fewer bytes for each vector.
These two options are independent of each other and can be combined, which means any storage budget can be met through different mixes of dimension count and numeric precision. The real question is which mix retrieves best for the same number of bytes. The trade-off is far from even. Suppose the budget is 512 bytes per vector. You could spend it on 128 dimensions kept at full 32-bit precision, or on all 4,096 dimensions kept at a single bit each. Both fit the budget exactly, but in testing, you’ll find that the second option retrieves considerably better.
To see why, it helps to think of each dimension as one small question the model has learned to ask about the text: Is this about error handling? Does it touch the network? Is it test code? And there are a few thousand similar topics and questions that haven’t been named. (The real dimensions are blurrier than that, but this is a useful abstraction.)
No single answer means much on its own. We consider two chunks to be similar when their answers to many of these questions are the same. Therefore, we should assess the vectors by looking at the coverage of the questions rather than the exactness of the answers.
Keeping all 4,096 dimensions at one bit preserves a rough yes-or-no answer to every question. Truncating to 128 dimensions keeps very precise answers to three percent of the questions and throws the rest away, and no amount of precision on the surviving dimensions can recover the information the discarded ones carried. In a sense, a long questionnaire filled in with checkmarks beats a short one filled in to six decimal places. Dimensions are what you want to keep; precision is what you can afford to lose and is easier to compensate for later on.
So we chose to keep every dimension and take the precision reduction to its limit, dropping the vectors to one bit each, which is 32 times smaller than the same vector in 32-bit floats. The quantization itself turns out to be surprisingly simple. Every component at or above zero becomes a one, while every negative component becomes a zero, and the magnitudes are thrown away:
Changing the representation changes the metric with it. Cosine similarity needs the magnitudes we just threw away, so binary vectors are compared by Hamming distance instead, which is simply the number of positions where two bit patterns disagree. Compare, for example, 10110100 and 10010110. They differ in two positions, so the distance between them is two. At full length, the computation stays just as simple. A 4,096-bit vector is stored as 64 words of 64 bits, and comparing two of them means XORing each pair of words, which leaves a 1 wherever the two vectors disagree, and then counting the 1s. A CPU does each of those in a single instruction per word, so a full comparison costs in the order of a hundred instructions where cosine similarity on the original floats needed thousands of multiplications.
Note that the metric was never a separate decision. We chose one-bit precision for the storage savings, and once every component is a sign bit, Hamming is the only comparison left that makes sense. Choosing the precision chose the metric.
Binary quantization still costs a few points of recall against the unquantized vector. We accepted that cost after considering that a reasoning agent would be consuming the results. A code search feeding an agent needs the right neighborhood far more than a perfectly ordered top 10. When the agent asks where session tokens get refreshed, what matters is that the relevant handful of files shows up among the first dozen results. Whether the best chunk ranks second or fifth changes nothing, because the agent opens the candidates and reads them anyway. In that loop, a ranking degradation that would be plainly visible in a three-result UI built for humans is mostly invisible.
The trade-off we made had a subtler cost that took us a bit longer to understand. Binary quantization doesn’t only sacrifice accuracy; it compresses the *range* of similarity scores. With full-precision vectors, an unrelated pair can score near zero while near-duplicates score near one, a comfortably wide spread. Sign bits behave differently. Around half the bits of two entirely unrelated vectors still agree by pure chance, while a strongly related pair might have agreement for two-thirds. So every score in the index, relevant or not, lands in that thin band.
Ranking survives the compression, since relevant results still score above irrelevant ones, but thresholding does not. Picture a feature that volunteers related code without being asked, say a panel that suggests existing implementations while you type. Its most difficult requirement is knowing when to stay silent. To make that determination, it needs a usable gap between “related” and “unrelated” scores. Binary vectors don’t leave one. Any cutoff placed inside that narrow band either fires on everything or on nothing. So where an index needs an absolute relevance judgement rather than a relative ordering, we keep 16-bit floats and pay for the storage.
While indexing and searching use the same model, the two jobs could not be more different. Indexing is throughput-constrained, with millions of chunks asynchronously handled. The GPU will handle about 32 chunks per batch before becoming saturated. A search, on the other hand, needs to be fast and responsive. Users will give up if they are not provided with results within a couple of seconds at most. Therefore in deploying these models we optimize them accordingly: one to maximize chunks per second, the other for minimizing time to first result.
We chose an instruction-following model, trained with a deliberate asymmetry between the two sides of retrieval. Significantly, the two sides are represented by very different types of text. A query is a short question in natural language, while a document is a chunk of code. A document is embedded as is at indexing time. A query is wrapped with an instruction describing the retrieval task, something like “given this search query, find the code that answers it”, which tells the model what role the text is playing. We preserve that arrangement at inference because it is the shape the model learned.
To allow the two sides to align more easily, we embed each chunk together with its file path. The path supplies metadata that the chunk alone lacks: which module it lives in, and what the file is. In a monorepo, though, the path itself becomes a problem. The IntelliJ IDEA monorepo runs to over a million files. The median source file there sits nine directories deep behind a 91-character path, and close to 10,000 source files have paths longer than 150 characters, the longest of them 218. That is before any checkout root is prepended.
Most of those characters are used for structural nesting and offer no useful information about the file. A run of segments like `src/org/jetbrains/kotlin/idea/k2` restates the package hierarchy, which a compiler needs and a search does not. Meanwhile, the file at the end of that longest path is 24 lines long. If we simply embed the path text as is beside a chunk, we’ll find that the path will sometimes take up more space than the code itself. To compensate for that, a path is capped before it reaches the model, and the rule is that *both ends survive*. The leading segments tell you which module you’re in, while the last two, the immediate parent and the filename, tell you what the file is. The middle is the part that can go, and only as much of it as the cap requires. Keep the longest prefix that still fits, elide what falls between into `…`, and if even parent-plus-filename is too long, keep only the name itself.
The same discipline applies when a user scopes a search to a subdirectory. The obvious implementation is a metadata filter: run the search as usual and discard results that fall outside the directory. We do something different. The scope is rendered into the query text itself, in the same shape, with the same abbreviation function and the same separator the indexed chunks used. If a chunk went into the index under the abbreviated form of `community/plugins/kotlin`, a query scoped to that directory carries the same string in exactly the same form, so the query vector lands in the same region as the chunks it is supposed to match.
There was one last design consideration we took into account. It was important for us to be attentive to customer privacy and security concerns. The source code of a company is often the core of its IP. Exposing it to third-party cloud models, or even to another company, increases the risk of inadvertently exposing sensitive data or even training other models to use it.
To make sure we address these concerns, we made the decision to adhere to several practices early on:
These self-imposed design restrictions carry no cost in terms of retrieval quality. We evaluated the open-weight candidates against the hosted embedding APIs from the major providers on our own code-retrieval benchmarks, and ours came out on top. Open-weight embedders are now good enough that the interesting engineering has moved into what you feed them, how you serve them, and what you choose to keep.
In this blog post, we covered the first stages of the retrieval pipeline: the journey from raw source files to compact vectors that are ready to be searched.
At this point, we have millions of binary vectors and a way to produce more. The problems we haven’t solved yet are how to store them efficiently, how to create a system that can answer a query in milliseconds, how we can continuously evaluate our results to ensure we are making the right choices, and how we can get the agent to actually use our shiny RAG apparatus.
These topics and more will be the subjects of the next parts in this series, which we’ll be releasing over the next few weeks. As always, please feel free to ask any questions in the comments or share your own hard lessons from designing a RAG solution. We are eager to learn of different and creative ways you have found to be effective! In the meantime, feel free to check out JetBrains Context, currently in public preview, it is already included with your JetBrains license 😀
Until next time!
It is now possible for Bionic to reference and introspect past sessions, making it much more capable of handling long-term context and retrieving forgotten details. You can also reference sessions directly from the composer using an @ mention.
Reference another session directly from the composer with an @ mention.
Models are increasingly good at finding information in a large "haystack" using search tools. Often a vague mention is all that's needed for a capable model to find relevant parts of the codebase. This got us thinking: what if we gave the agent the ability to read/search through session transcripts from both the current session (which may be long and have undergone many compactions) and other sessions, even in other projects?
Bionic now has a set of tools and built-in skills we call "Introspection". These tools allow the agent to recover details that were previously forgotten or omitted during compaction. It essentially gives the agent a way to "look back" at its own history and fill in gaps in its knowledge.
This comes in handy over very long sessions. Design decisions, pitfalls, environment info, and many other things are now just one introspection away.
While we already have multiple tricks that improve the performance of the agent after compactions (which we will eventually write a blog post about), there is a noticeable leap in the ability to adhere to the plan in extremely long-horizon tasks, tasks that take multiple hours to complete. Usually, with compaction, as soon as a piece of information that is not classified as "always keep" is missed in a handoff, the information is lost forever.
Nevertheless, with introspection, the agent is able to just read the transcript and get the information back.
The following diagram illustrates how introspection allows a compacted Bionic session to recover details from its persisted transcript.
Introspection searches the persisted transcript and recovers details omitted during compaction.
As with anything we build in Bionic, we are extremely careful about the context, and we don't want to fill it with tool definitions. All tools used for introspection are implemented as progressively disclosed tools, documented in a built-in SKILL. The agent will only load the relevant tools and documentation when it sees the need to introspect.
In order to further save context, we also employ "tiered" tool designs for introspection, meaning we provide tools that read/search transcripts at a high level with heavy truncation. Once Bionic identifies a message of interest, the agent may retrieve the full content of the message with a separate tool call.
We also provide options to filter out things like tool call results, since those are normally not useful but can be accessed if needed.
One thing to be careful about with introspection is cross-session contamination. Sometimes, sessions reach a dead end or reflect some abandoned or undesired path. We don't want the agent to proactively read those sessions and contaminate the current session. To address this, we gate the ability to read other sessions behind a permission dialog. Reading the agent's own transcript does not require approval.
Reading other Bionic sessions requires explicit permission; reading the current session does not.
@If you have a past session you want the agent to reference, you can easily do so with the @ syntax. This works across projects, too.
Reference another session directly from the composer with an @ mention.
Quality and reliability have always been a point of pride for Spotify. We run an extraordinarily complex ecosystem of interconnected microservices and data pipelines that all come together in a super app for 777 million monthly active users across more than 2,000 supported devices. At any given moment, our platform serves around 100 million concurrent clients, processes 11-12 million backend requests per second, and runs nearly 3,000 production services. Quality at our scale has never been a solved problem. Before AI entered our workflow, a weakness anywhere in the system could reach listeners and creators quickly. Recently, though, four particular areas have tested us at once, and, what may surprise some, AI slop isn’t among those. It’s the pace of change inside Spotify and across the world that has forced us to adapt.
Spotify processes more than 500K new songs, videos, podcasts, and audiobooks every day, and it’s growing rapidly. It's critical to the artists and creators behind this that these become available quickly and reliably.
Pre-existing to this onslaught of content upload, we already had two existing weaknesses that have impacted this process and the people dependent on them. First, processing failures could be masked. For example, a media file we could not process could sometimes fail silently and its impact on publishing go unnoticed for hours because the failure did not page anyone. Second, the pipeline did not have enough capacity for spikes in our growing video catalog processing; valid video episodes could wait in a queue unalerted when transcoding capacity was exhausted.
Our report describes what happened on June 24, where small changes related to these factors combined: a scheduled batch job was competing with new episodes, a recent quality improvement had increased the compute each episode required, and a scheduling bug reduced throughput by about 10%. Episodes that normally published within minutes were delayed for hours. The root causes were garden variety ones, and ours: subtle misses on failure alerting, challenges in capacity planning, and small gaps in workload controls.
We have since added end to end monitoring so we know of failures before creators do, fixed the scheduler, moved batch jobs to run at a lower priority, and increased capacity. We also reworked service tiering and workload prioritization so critical services and new uploads take precedence when capacity is constrained. Episodes from bad actors are suppressed and lowered in priority greatly reducing overall load and doesn’t compete with higher-priority episodes. AI helped deliver these faster, but, as was the case before we started using agents, these are problems that require distinct judgement, an end to end mindset and skills from our engineers.
A separate challenge is managing the growing scale of automated change. Our custom Fleet Management framework has for years made large scale changes across our fleet every day, with the vast majority merged automatically after passing safety checks. For over a year now, we have expanded this to support more complex agentic-driven changes, including a recent Java migration across backend services completed in three days. The pace has accelerated even further, shielding engineers from even more mundane tasks.
But that increased automation also creates new failure modes. This year, an automated dependency upgrade passed our checks, but still failed in production, impacting end users. We are responding by strengthening safeguards, expanding rollback capacity, and scheduling automated changes during owning teams’ working hours.
Across the industry, the use of AI has triggered a huge spike in demand, without commensurate supply increases, on both CPUs and GPUs, reducing spare capacity. Spotify has long operated in an environment where compute capacity was generally available when we needed it. As industry demand has grown, that capacity is less predictable.When a single region fails, we shift traffic to another region to handle the load. While in one sense a regional failover is a significant event, we’ve designed this to minimize customer impact.
Earlier this year, when Spotify executed regional failovers, this lack of capacity exacerbated issues that were previously trivial and unnoticeable. In this new environment of capacity constraints, our end users noticed. This is an impact of AI to our quality of service, but an indirect one, not one that matches those commonly cited.
As a result, we have had to review our network edge and tiering approaches. Now, when we failover, we must accept that there may not be capacity for the lower tiers of services. We are also strengthening resilience across our production services. We doubled reserved edge capacity after a May incident. Today we can shift part of our internal service-mesh traffic manually; extending that control to edge traffic and testing regional spillover are still under way. The goal is to move traffic gradually while ensuring receiving regions can absorb the additional load while maintaining stability.
For over a decade we’ve seen an ebb and flow in our mobile app quality. We get intense about shipping amazing new features fast, and this can introduce trade-offs with the quality of the experience. As these new negative quality signals build, we then shift capacity and incentives toward quality. We broaden our guardrail metrics, push to recover, and get back to a good state with broader guardrails. Over time, new issues emerge outside those that existing metrics and guardrails capture, and the cycle repeats.
AI has increased the pace of change, which means this cycle moves at a higher frequency and gaps surface faster. The issue isn’t that AI-assisted code is inherently lower quality; it’s that our systems for measuring and maintaining quality need to keep pace with how quickly we can now build and ship.
Our release process already has deliberate checkpoints before production. What our recent work highlighted is that individual releases can look healthy while smaller regressions accumulate over time, affect particular phones, or sit outside the signals we are watching. We have now broadened those quality signals and added longer-term trends to those decisions so we can identify deterioration earlier.
Moving to AI-assisted development at this scale raised a fair question inside the company and outside it: what does this do to quality? Google Cloud's 2025 DORA research found that AI adoption was associated with higher delivery throughput and product performance, but lower software delivery stability. But every company is different, so we decided to answer the quality question by looking at our own data.
First and foremost, we looked at production incidents, as these are the ultimate measures of quality. Every month we run a retrospective of all major incidents. During our AI ramp-up, we began asking two additional questions: Did AI-authored code directly contribute to the incident? And did the increased volume of change put additional pressure on review, testing, rollout, or observability?
Across the incidents reviewed so far, we did not identify AI-authored code as a material direct contributor. We did, however, observe the second risk: the volume of change increased faster than some of our verification controls could adapt. In response, we are strengthening the entire delivery system, including review, testing, rollout, observability, and rollback.
Next, looked for evidence of a quality-for-velocity trade offs further up the pipeline. We classify every merged PR by the work it contains: features, code quality and optimization, maintenance, and documentation. Total merged changes more than doubled year over year in August, from roughly 8,100 to 17,000. Quality and optimization work rose from 27% of that mix to 31%, which means engineers put more than twice as much absolute work into code quality this August as last. Feature work grew as well. Maintenance and configuration fell from 31% of the mix to 25%. The increase in both the absolute and relative time spent on code quality and optimization is one reason we believe we are not seeing a simple quality-for-velocity trade-off.
We also rebuilt our rework rate metric to separate genuine rework from new work and legacy refactoring. Code churn measures how much code gets removed relative to what gets added. Rework rate weighs the age of the code being changed, which is a better proxy for whether recent work holds up. The FAROS 2026 report found a sharp industry-wide rise in code churn, but we see no corresponding rise in rework rate. That is a clear signal that we are not accumulating AI-induced quality debt.
There are two warning signals we are watching: code complexity and PR size are both creeping up. Pre-AI, those were unambiguous quality concerns. Now a larger PR may just mean a human and an agent reasoned together and delivered a bigger unit of work safely, and complexity thresholds calibrated for what one person could hold in their head may no longer apply. We don't have conviction in either hypothesis, so we are deliberately not rewriting the thresholds to make ourselves feel better. We'll continue to watch these metrics to see if they are truly leading indicators.
Earlier this year, we didn’t live up to our quality standards everywhere we’d have liked. So, we investigated and continue to remediate and improve. The causes were from the mundane to, in hindsight, the predictable, given the more rapid pace. What may surprise some, is that data does not show a distinct direct AI-authored failure signature.
AI increased the capacity to produce change. The next constraint became our ability to verify it. Keeping the delivery system aligned with that increased pace of change is now a continuous effort, automated safeguards, rollback, observability, failover, and quality measurement. The work now is to ensure those controls operate at the same pace as development.
As one of the fastest growing mobile apps in the world, the Shop app serves hundreds of millions of customers and millions of merchants. Since we introduced it in 2020, Shop app has been at the forefront of Shopify’s wider investment in React Native, adopting the framework from its inception. That decision has served Shop app well, but our increasing ability to develop with coding agents has allowed Shop to go fully native, building with Swift and Kotlin.
We wrote in depth about our decision to go native, but the TL;DR is that advances in coding agents changed the tradeoffs behind maintaining a shared mobile codebase. Building separately for iOS and Android still has costs, but agents made it reasonable to reconsider that decision.
For the Shop App, this coincided with our next major React Native investment: adopting the New Architecture. That work would have required us to revisit native module integrations, rendering, and the boundaries between shared and platform-specific code. Before committing to this investment, we tested whether coding agents could help us build directly in SwiftUI and Jetpack Compose while keeping product behavior aligned across platforms.
Before committing to a full migration, we ran a small proof of concept. One engineer spent a week working with coding agents to migrate as much of the existing React Native app as possible into a native iOS app built with SwiftUI.
The results were compelling. Using the existing React Native app as a reference, we were able to recreate screens, interactions, and application flows quickly. The result wasn’t production-ready after just one week, but it demonstrated that a close, feature-for-feature migration was achievable and gave us confidence to pursue a full migration.
Agents were particularly effective when they had an existing implementation to work from. They ported defined features, scaffolded screens, wired up data, implemented animations, and refined layouts based on visual feedback.
After we made the decision to migrate, a core group of six engineers built the native foundations and the app’s main user journeys. Feature teams joined midway through the migration to validate their areas and cover edge cases. Our priority was preserving both the behavior of the features we ported over and the analytics events that downstream systems depended on.
The migration needed to feel like a normal app update for existing users: they should remain signed in and continue receiving push notifications. Interactions also needed to emit the expected events, with the context required by downstream systems, like recommendations. While preserving that continuity, we also used the migration to simplify the app, deliberately retiring some screens and streamlining others.
We compared the native apps with their React Native predecessors across startup time, session stability, app size, build time, and rendering performance.
These recordings compare cold starts of the native iOS and Android apps with their React Native counterparts. Time is measured from tapping the app icon until the initial home feed content is visible.
| Platform | Native | React Native | Startup time reduction |
|---|---|---|---|
| iOS | 2466 ms | 3200 ms | 23% |
| Android | 2233 ms | 4433 ms | 50% |
The native releases recorded higher stability rates than React Native. Our historical stability was at 99.5%+ but with the release of the native version, our session stability has climbed to 99.95%+ — a 10x reduction in sessions that crash.
The native Android release build was substantially smaller than its React Native counterpart, shrinking by 109 MB. On iOS, the release build was similar in size, increasing by 1 MB.
| Platform | Native | React Native | Difference |
|---|---|---|---|
| iOS | 68 MB | 67 MB | +1 MB (+1.5%) |
| Android | 184 MB | 293 MB | −109 MB (−37.2%) |
Android release build time fell approximately 75%. On iOS, release builds are taking about the same amount of time. The benefit of build time improvements include being able to test out new builds faster as well as consuming less computational time.
In this recording, the native Android app reaches 120 FPS while scrolling the feed and navigating between screens on a Pixel device. This has been accomplished with very little optimizations so far, underscoring a critical improvement in performance.
Agentic development workflows differ from traditional ones. Rather than having one developer make a change, run the app, and iterate, we now often run multiple agent sessions across separate worktrees. For this migration, we focused on giving agents clear tasks, maintaining fast build and test loops, and reviewing changes frequently.
We built a reusable migration workflow as an extension for the Pi coding agent. Specialized subagents inspected the React Native source, documented its behavior, prepared platform plans, implemented features, and reviewed parity. The source review covered UI, state, navigation, analytics, accessibility, and data behavior. Engineers reviewed requirements and plans before proceeding with implementation. Plan acceptance was tied to a hash of its contents: changing a plan invalidated its previous acceptance. This kept approval attached to the implementation plan that had actually been reviewed.
Agents also needed feedback from the running app. We built Tardis, a debugging tool to give them structured access to live native app events, logs, and state, along with the ability to send commands to the app. Agents could use that feedback to investigate issues, check navigation and analytics, and validate fixes with less manual UI inspection.
To support parity reviews, we added a Tardis feature that captured screenshots and event windows from the React Native and native apps at named checkpoints. With raw event capture enabled, agents could compare event names, counts, and payload fields. The comparison instructions accounted for values that naturally differed between runs, like timestamps and page UUIDs, while checking the relationships between events and their page or entity context. This gave our engineers a repeatable way to investigate discrepancies in the flows they exercised.
Native expertise remained essential. Generated code could satisfy feature requirements while still introducing duplication, architectural drift, or performance problems. Repository guidance helped, alongside linting, tests, static analysis, performance checks, and code review.
The learning curve was nevertheless more manageable than we expected: familiar declarative UI concepts helped React Native engineers become productive in SwiftUI and Jetpack Compose, while platform knowledge guided our architectural decisions and reviews. A simplified version of our Avatar component illustrates that shared structure.
These simplified examples use our Gravity design system. Imports, image loading, and accessibility details are omitted.
One of the key principles we established when undergoing this migration was that Android and iOS must be at feature parity at all times. React Native enforced this through a shared codebase, and now that we’re building natively we’ll continue to enforce this through our development and release process.
We know there’s still more polish and improvements we can make to our performance. This first version raised the ceiling of where we can go as a mobile app, and we see it as the baseline that we’ll improve from. As we continue to evolve our tooling and development process, we hope to share more learnings here in the future.
Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction.
We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific AI workloads. Software is changing just as fast, and AI coding tools now generate in minutes what took months of effort a few years ago.
With so much of computing now centered on AI, GPU kernels are a crucial component of its success. These are the low-level programs that run inside the GPU, and writing efficient ones is far from obvious — it takes years of expertise to get right. Transferring a kernel from one vendor’s hardware to another is harder still, and often means rediscovering the same optimizations from scratch. The CUDA ecosystem, for example, has accumulated decades of hard-won kernel expertise: hand-tuned implementations of attention, state space models, and other critical operations representing thousands of engineering hours. Newer hardware ecosystems (Apple Silicon, custom AI accelerators, and others) are growing fast but lack this depth.
In this work we ask whether that expertise can be transferred automatically. We built on K-Search, an evolutionary kernel search framework introduced by Cao et al. at Berkeley Sky Lab that uses AI to optimize GPU kernels, and extended it with a backend for MLX — Apple’s machine-learning framework for its own Apple Silicon chips. We developed a novel structured CUDA-to-MLX translation layer that lets K-Search take existing CUDA kernels as a knowledge base and adapt them into high-quality GPU kernels for Apple Silicon, rather than rebuilding from scratch.
We show that our approach reaches near-expert level performance on Apple Silicon with 0.97x speedup compared to the native MLX Attention kernel, and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel; we report the numbers, and how much of the gain comes from the translation layer, in the sections below. Although we focus on MLX kernels for Apple Silicon, the method is not specific to MLX and applies to any ecosystem where CUDA expertise is transferable.
Apple’s MLX framework has seen remarkable adoption since late 2023. With Apple Silicon in hundreds of millions of MacBooks and Mac Studios, MLX enables local AI inference without cloud costs. The unified memory architecture makes it especially attractive for mid-sized models (7B–70B parameters on M series chips).
Yet beneath this momentum lies a significant gap: many performance-critical kernels that the NVIDIA ecosystem takes for granted: paged attention, optimized SSM scan kernels, fused MoE routing are either absent or naive without hardware-specific tuning. MLX runs models correctly but often leaves significant performance on the table.
This gap is what motivates the rest of this post.
K-Search is an evolutionary kernel optimization framework originally developed by our first author Shiyi Cao at UC Berkeley Sky Lab. Given a naive kernel and a hardware specification, it runs an iterative optimization loop: an LLM reasons about which optimizations to try next, a code-writing model generates candidate kernels, and those candidates are compiled and benchmarked on real hardware.
Measurements feed back into the search, which keeps refining, pursuing promising directions and dropping dead ends until performance converges.
Algorithm 1: K-Search via co-evolving world models. The search alternates between selecting the most promising action, instantiating and evaluating code until improvement stagnates, and evolving the world model through insert, update, and prune operations. Adapted from Cao et al. (2026).
Search is grounded by a Spec: a domain-specific document encoding hardware rules, optimization patterns, and mathematical constraints which keeps generated code from hallucinating invalid primitives and ensures candidates will actually compile and run efficiently.
In our runs, a single model (Gemini 3.5 Pro Preview) plays both roles: it maintains the reasoning state and writes the kernels. The reasoning half is prompted as a “GPU kernel performance engineer” and asked to work through a fixed analysis before proposing anything: classify the kernel (reduction, scan, attention/softmax, …), rewrite the reference computation in canonical form, map out data layout and access patterns, and hypothesize the likely bottleneck (bandwidth, latency, compute, or synchronization) in each runtime regime. Only then does it emit candidate optimizations, each as a single change implementable in one iteration.
We call the persistent reasoning state a world model. Rather than a flat list of things to try, it is a decision (prefix) tree: each root→leaf path composes a full optimization plan, and sibling branches are competing alternatives. Every node is scored — an overall_rating in [0, 10], a confidence in [0, 1], and per-node impacts on memory bandwidth, register pressure, and compute/hardware fit — so the search can rank partial plans and expand the most promising ones. The tree persists and grows across rounds: refining an idea adds a child node rather than overwriting its parent, and if the best score fails to improve for a few rounds (a stagnation window) the search backs off to explore an alternative branch. A single node, as it appears mid-run on the attention kernel, looks like this:
{
"action": "Replace the threadgroup-memory softmax reduction
with a register-only reduction: each SIMD group
owns 8 query rows and reduces across lanes with
simd_shuffle_xor, removing a threadgroup_barrier.",
"difficulty_1_to_5": 4,
"impacts": {
"memory_bandwidth": 8,
"register_pressure": 4, // risk: spill if Br > 8
"compute_hw_fit": 9 // SIMD width 32; keep tile 8x8
},
"overall_rating_0_to_10": 8,
"confidence_0_to_1": 0.7
}
Listing 1: Example K-Search world-model node. Each candidate optimization records a concrete action, estimated hardware impacts, an overall priority rating, and the model's confidence.
Figure 2: Overview of K-Search. The framework operates on a Search State $S_t$ structured as a search tree. The tree consists of Closed nodes (blue, visited states with attached program like $x_{12}$) and a Frontier of Open nodes (orange, pending hypotheses like $u_{13}$). The workflow iterates through three phases: (1) Action Selection, where the most promising action node is retrieved from the frontier based on world model estimated priority score $V$; (2) Local Refinement, where a stochastic policy $\pi_{\mathrm{code}}$ samples concrete implementations until stagnation; and (3) World Model Update, where the LLM reasons over the trajectory to update the search tree via Insert (adding new actions), Update (adjusting $V$, e.g., $u_{11}$ dropping from 0.9 to 0.6), and Prune (removing less promising nodes like $u_{10}$).
The original K-Search paper evaluated this search strategy on CUDA kernels from FlashInfer. Across GQA decode, MLA decode, MLA prefill, and MoE, K-Search improved more consistently than OpenEvolve and ShinkaEvolve over the same 120-iteration budget. These results establish the search framework we build on here; the remainder of this post asks whether its optimization knowledge can transfer beyond CUDA.
Figure 3: Main results from the original K-Search paper. Across three runs, K-Search achieves stronger best-so-far search scores, per-workload kernel performance, and speedup distributions than OpenEvolve and ShinkaEvolve on four FlashInfer CUDA kernels. Reproduced exactly from Cao et al. (2026).
To bring K-Search to Apple Silicon, we first built a native MLX backend. We implemented a full MLX-specific task adapter for K-Search, including:
k_search/tasks/ handling kernel compilation and execution on Apple Silicon via MLX’s Metal/C++ APIs.mlx.core measurement utilities.However, the more interesting challenge was not simply running K-Search on MLX. The key insight is that expert CUDA kernels encode decades of optimization knowledge that is transferable to Apple GPU if you can bridge the conceptual gap. Simply handing an LLM a CUDA kernel and asking it to port it is not enough: without deep hardware context, it produces code that is syntactically valid but architecturally wrong (wrong tile sizes, invalid primitives, mismatched memory assumptions).
Our translation layer consists of:
__shared__ maps to Metal threadgroup memory but with a hard 32 KB limit (vs. NVIDIA’s 48 KB)warp_reduce maps to MMA (preferred)__syncthreads() becomes threadgroup_barrier(mem_flags::mem_tg)simd_shuffle_xor in an 8×8 MMA tile layout, or the “exp2 trick” (replacing $exp(x)$ with $exp_2(x \log_2 e)$) for faster softmax on Apple’s fast $exp_2$ hardware instruction.We evaluate three configurations of an MLX attention kernel for Apple Silicon: (1) a naive baseline, (2) pure evolution with no additional provided context, and (3) a full context translation layer, which supplies the optimizer with architecture-specific implementation knowledge extracted from high-performance kernels (e.g., FlashAttention-2), letting the evolutionary search reason about implementation strategies rather than starting from a naive kernel. Together, these three configurations let us isolate the exact impact of the translation layer.
Figure 4: Performance scaling of the Attention Kernel through stacked optimizations. The "Full Context" configuration successfully discovers and implements advanced strategies like double buffering and loop unrolling, achieving near-expert performance.
The jump from 0.26× to 0.97× the speed of Apple’s state-of-the-art attention kernel — illustrates how much the translation layer matters. With full context, the evolved kernel independently discovers the key optimizations in FlashAttention 2: threadgroup memory tiling, online softmax, K-transposition for memory access, and the exp2 trick. The last of these replaces every softmax exponential with a base-2 exponential,
\[e^x = 2^{x \log_2 e},\]which is exact and lets the kernel use Apple’s fast fast::exp2() hardware instruction directly instead of paying for a base conversion at runtime.
To evaluate whether K-Search generalizes beyond attention kernels, we applied it to the state-space model (SSM) kernel used by Mamba. Unlike attention, the computational bottleneck is a recurrent state update rather than a softmax, providing a substantially different optimization challenge. We compare the evolved implementation against the community MLX implementation (mlx-lm) and the PyTorch reference implementation (mamba.py) on an M1 Max.
Evaluated on mamba-370m f16, M1 Max 64GB:
| Metric | mlx-mamba (ours) | mlx-lm (community) | mamba.py |
|---|---|---|---|
| Decode | 152 tok/s | 116 tok/s | 40 tok/s |
| Prefill L=512 | 5,751 tok/s | 329 tok/s | 1,089 tok/s |
| Prefill L=1024 | 6,010 tok/s | 327 tok/s | 1,127 tok/s |
| Prefill L=2048 | 6,612 tok/s | 326 tok/s | 1,092 tok/s |
| Prefill L=4096 | 6,743 tok/s | 339 tok/s | 1,042 tok/s |
Table 1: Prefill and decode throughput on mamba-370m (f16, M1 Max 64GB). mlx-mamba (ours) reaches ~20× higher prefill throughput than the community mlx-lm baseline, while decode remains comparable.
The ~20× prefill speedup over mlx-lm comes down to one difference: mlx-lm does not implement a parallel scan for the SSM. The state recurrence
\[h_t = \bar{a}_t h_{t-1} + \bar{b}_t\]looks inherently sequential, but each step can be written as a pair $(\bar{a}_t, \bar{b}_t)$ under the associative combine
\[(a_2, b_2) \circ (a_1, b_1) = \left(a_2 a_1,\ a_2 b_1 + b_2\right),\]which reproduces the recurrence exactly. Because the operator is associative, the whole sequence can be evaluated with a parallel (prefix) scan in $O(\log N)$ dependent steps instead of $O(N)$. mlx-lm skips this and processes tokens one at a time, leaving most of Apple Silicon’s compute idle; our evolved Metal kernel applies the scan and makes much fuller use of GPU throughput. The gain shows up in prefill, where the full sequence is available to scan in parallel, and not in single-token decode, where there is only one new token per step and no scan to parallelize — which is why the decode row is roughly flat while prefill is ~20×.
mamba.py is slow on both prefill and decode because it is a PyTorch reference implementation that falls back to CPU or MPS on Apple Silicon, forgoing the hardware-specific optimizations that MLX’s Metal backend makes possible.
On the two kernels we studied, AI-driven evolutionary kernel search grounded in structured cross-platform translation knowledge reached near-expert performance on Apple Silicon without a team of GPU experts starting from scratch. We do not yet know how far this generalizes, but the result is encouraging.
For us the main takeaway is that the bottleneck was not the LLM’s ability to write Metal code, but the quality of the context and constraints we gave it. Our CUDA translation layer converts existing NVIDIA kernel expertise into actionable guidance for Apple Silicon, and lets K-Search’s evolutionary search do the rest.
We are actively extending this work in several directions: supporting new architectures, with current efforts focused on developing new kernels for the IBM Spyre AIU and broader hardware targets; adding more kernels such as paged attention and fused MoE routing; and improving integration with the K-Search evolution loop to make translation context even more automatic.
This work was carried out by IBM Research and builds on K-Search from the UC Berkeley Sky Lab (Cao et al., 2026). We welcome collaboration and feedback from the MLX and broader AI systems communities. If you are working on kernel optimization for non-CUDA hardware, we would love to hear from you.
@article{cao2026k,
title={K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model},
author={Cao, Shiyi and Mao, Ziming and Gonzalez, Joseph E and Stoica, Ion},
journal={arXiv preprint arXiv:2602.19128},
year={2026}
}
The MLX backend is built on top of the open-source K-Search repo, so the results here can be reproduced directly. The steps are:
1. Clone and install
git clone https://github.com/caoshiyi/K-Search.git
cd K-Search
uv pip install openai wandb
uv pip install git+https://github.com/caoshiyi/flashinfer-bench-ksearch.git
2. Set your credentials
Open the relevant script under scripts/ and set three variables at the top:
KSEARCH_ROOT=/path/to/K-Search
API_KEY=your-llm-api-key
3. Run kernel search
# Optimize Flash Attention on Apple Silicon (world-model mode)
bash scripts/mac_flash_attention_wm.sh
# Or a Mamba SSM kernel, e.g. the selective scan
bash scripts/mamba_selective_scan_fwd_wm.sh
Full CLI reference and documentation are in the README.
opencl: add A8 Q6_K non-MoE dp4a binary kernel (#29057)
Website:
Attestations:
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
See kubernetes-announce@. Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.
fe2a6b2be8] - crypto: update root certificates to NSS 3.125 (Node.js GitHub Bot) #64746871167ddfd] - deps: update corepack to 0.36.0 (Node.js GitHub Bot) #65653b816fc8958] - deps: upgrade npm to 10.9.9 (npm team) #64884e306521444] - deps: update icu to 78.3 (Node.js GitHub Bot) #62324a9cb31129f] - deps: update OpenSSL 3.5.8 (Node.js GitHub Bot) #655422a3548e51c] - deps: update Undici to 6.28.1 (mcollina) #657903909ff2c4a] - node-api: support SharedArrayBuffer in napi_create_typedarray (Yilong Li) #6271066de6349ad] - node-api: add napi_create_external_sharedarraybuffer (Ben Noordhuis) #6262344cf27b8fa] - build: update binary-upload to use correct tarball name (Stewart X Addison) #65282fe2a6b2be8] - crypto: update root certificates to NSS 3.125 (Node.js GitHub Bot) #6474671feba6b69] - crypto: update root certificates to NSS 3.123.1 (Node.js GitHub Bot) #63527871167ddfd] - deps: update corepack to 0.36.0 (Node.js GitHub Bot) #656532a3548e51c] - deps: update undici to 6.28.1 (mcollina) #6579059d853a4df] - deps: update archs files for openssl-3.5.8 (Node.js GitHub Bot) #65542a9cb31129f] - deps: upgrade openssl sources to openssl-3.5.8 (Node.js GitHub Bot) #655423376e27de2] - deps: V8: cherry-pick a6eaf7574109 (Camillo Bruni) #65402b816fc8958] - deps: upgrade npm to 10.9.9 (npm team) #648844e4bd1b104] - deps: update timezone to 2026c (Node.js GitHub Bot) #645887d82841b4e] - deps: update c-ares to 1.34.8 (Node.js GitHub Bot) #6433001855a19d3] - deps: c-ares: cherry-pick 8ba37af8e3fb (René) #6411023fb398c3d] - deps: update corepack to 0.35.0 (Node.js GitHub Bot) #633755286330365] - deps: update corepack to 0.34.7 (Node.js GitHub Bot) #628106d6c3c98b1] - deps: update timezone to 2026b (Node.js GitHub Bot) #62962e306521444] - deps: update icu to 78.3 (Node.js GitHub Bot) #62324d9cb8468a3] - doc: clarify filter option of sqlite.database.applyChangeset (Antoine du Hamel) #63515c9c5662d91] - doc: add sxa GPG key (ed25519) (Stewart X Addison) #6419337f21068c4] - fs: restore fs patchability in ESM loader (Joyee Cheung) #628357c2df5dd96] - http2: avoid uaf while receiving and sending rst_stream (esgor) #641663909ff2c4a] - node-api: support SharedArrayBuffer in napi_create_typedarray (Yilong Li) #6271066de6349ad] - node-api: add napi_create_external_sharedarraybuffer (Ben Noordhuis) #62623ce9139107f] - src: escape Windows environment variables in task runner (Antoine du Hamel) #65217839480a471] - test: fix link-local dgram scope assertion (Filip Skokan) #6562940eac4a32f] - test: account for varied OpenSSL CCM final behaviours (Filip Skokan) #6554275098a9e8c] - tools: update gr2m/create-or-update-pull-request-action to v1.10.1 (Mike McCready) #6306508b6ac0416] - tools: revert OpenSSL update workflow to ubuntu-latest (Richard Lau) #6262702cafc479f] - tools: fix commit linter for semver-major release proposals (Antoine du Hamel) #629936f6cd3768d] - tools: sync mk-ca-bundle.pl with curl (Archkon) #64753bc5753d438] - tools: remove envinfo from our workflows (Antoine du Hamel) #64259d68ee9f8a5] - tools: validate version number in release proposal commit message lint (Antoine du Hamel) #6407038ee2e895f] - tools: avoid test/fixtures/wpt/README.md conflicts (Filip Skokan) #63938fdc65e489f] - tools: use different branch for tool updates on staging branches (Antoine du Hamel) #63110e5a6fde002] - tools: update gyp-next to 0.22.1 (Node.js GitHub Bot) #629615fbbad6e82] - url: handle unparsable serialized URLs in setters (Matteo Collina) #64651ed019e4854] - util: preserve function names without source map names (Hiroki Osame) #65108Changes since langchain-anthropic==1.7.3
chore(anthropic): fix integration test cassette (#40790)
release(anthropic): 1.7.4 (#40786)
fix(anthropic): add Opus 5.5 and GPT-6 profile augmentations (#40785)
feat(anthropic,openai): mid-conversation tool changes on SystemMessage (#40758)
This week's release includes faster rendering in syntax-highlighted files and Markdown code blocks, a language server command picker, a setting that keeps your system awake during long-running agent turns, and BYOK support for Claude Opus 5.5 (Anthropic) and GPT-6 Astra, Sol, and Luna (OpenAI).
agent.prevent_idle_sleep setting, enabled by default, to prevent idle system sleep while agent threads are running. (#53130; thanks cppcoffee)agent.threads_sidebar_default_width setting to configure the width of the Threads Sidebar. (#62883; thanks porada)agent: rename selected thread action for renaming the active Terminal Thread from the Agent Panel. (#63660; thanks mauriciord)Cut, Copy, and Paste context menu actions to the commit message editor. (#64142; thanks hooch)showDocument requests. (#63607)comment_empty_lines parameter to the editor::ToggleComments keybinding action for multiline selections. Set it to true to comment blank lines or false to skip them; Zed's default keymap now uses true, while the VS Code keymap uses false. (#63961; thanks UdeshyaDhungana)editor.code_lens.foreground for customizing CodeLens text independently. (#64084; thanks giorgiopogliani)"..." in read_only_files so project settings can extend inherited read-only patterns. (#64222; thanks porada)menu (Linux and Windows) and shift-f10 (all platforms) shortcuts to open the context menu for the selected Project Panel entry. (#46744; thanks CCXLV)terminal.background color. (#64163; thanks chrisdrackett)Changes since langchain-openai==1.6.4
release(openai): 1.6.5 (#40787)
fix(anthropic): add Opus 5.5 and GPT-6 profile augmentations (#40785)
feat(anthropic,openai): mid-conversation tool changes on SystemMessage (#40758)
This fixes this stupid bug and flicker that we would always select the
first entry when opening context menus for A11y reasons even when A11y
is disabled for the entire window. I call it a bug because it adds
nothing, flickers, and has annoyed me since this was introduced.
Release Notes:
Released on 2026-09-22.
This release addresses GHSA-2cv4-cqwr-gwf7, which is a path traversal weakness during wheel installation on Windows. No other platforms are affected by this advisory.
--output-format json to uv pip install and uv pip sync, including for --dry-run and --check (#21893)--check to uv pip install and uv pip sync to report planned changes without modifying the environment (#21844)get_requires_for_build_* hooks correctly in build errors (#21881)uv build --no-build-isolation with --preview-features build-dependency-check; use --skip-dependency-check to opt out (#21880)uv_build editable wheel creation by omitting compression from temporary wheels (#21918)uv add, uv remove, or uv version fails or is interrupted (#21860, #21856)dependency-metadata when checking whether installed requirements are satisfied (#21843)? (#21920)curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/uv/releases/download/0.12.18/uv-installer.sh | sh
powershell -ExecutionPolicy Bypass -c "irm https://releases.astral.sh/github/uv/releases/download/0.12.18/uv-installer.ps1 | iex"The artifacts in this release have attestations generated with GitHub Artifact Attestations. These can be verified by using the GitHub CLI:
gh attestation verify <file-path of downloaded artifact> --repo astral-sh/uv
You can also download the attestation from GitHub and verify against that directly:
gh attestation verify <file-path of downloaded artifact> --bundle <file-path of downloaded attestation>
BUG FIXES:
Fixed an issue where Terraform fails when rendering policy evaluation outcomes for older versions of Terraform Enterprise (#39095)
stacks: Fix invalid deferred error triggered by provider returning a deferral when a resource also has an unknown count/for_each. (#39237)
Full Changelog: rust-v0.156.0...rust-v0.156.1
Changes since langchain-openai==1.6.3
release(openai): 1.6.4 (#40775)
chore(model-profiles): refresh openai model profile data (#40774)
Changes since langchain-anthropic==1.7.2
release(anthropic): 1.7.3 (#40773)
chore(model-profiles): refresh anthropic model profile data (#40772)
fix(anthropic): auto-route with_structured_output to method="json_schema" for fable and opus 5.5 (#40766)
chore(anthropic): update docs for Opus 5.5 (#40765)
feat(anthropic): send mid-conversation SystemMessages in place (#40622)
chore(deps): bump anyio from 4.11.0 to 4.14.2 in /libs/partners/anthropic (#40643)
GET /api/show now advertises each model's thinking controls and default:
Available in the CLI with:
ollama show gemma4
thinking
levels false, true
default true
Available in the API with:
curl http://localhost:11434/api/show -d '{"model": "glm-5.3-flash:cloud"}'{
"thinking": {
"values": ["low", "high", "max"],
"default": "max"
}
}Also available on ollama.com directly for cloud models.
Full Changelog: v0.34.2...v0.34.3
This release contains additional security hardening for next/og. For more information, check out https://nextjs.org/blog/nextjs-security-update-september-22-2026
This release contains a security fix for GHSA-vcvr-r3jv-pc5j: Remote Code Execution in next/og ImageResponse
claude-opus-5-5), now the default Opus model — 1M context, $4/$20 per Mtok with $0.20/Mtok cache reads/skills list, and a skill's state options in /plugin can be clickedCLAUDE_CODE_MAX_MCP_DESCRIPTION_LENGTH to change the 2,048-character cap on MCP tool descriptions and server instructions for every MCP server in the sessionhook_execution_complete OpenTelemetry eventacceptEdits, allow rules and auto mode no longer approve one landing outsidepath, file_text, file_content or a stray description instead of file_path and content/model, /effort, /config, /status, /usage, /plugin, /sandbox, /permissions, /artifacts, /mobile, /login, /upgrade, /usage-credits, /install-github-app, /setup-bedrock, /setup-vertex) quitting Claude Code instead of closing the dialogn closing dialogs and a stray y confirming them; Enter and Esc accept and cancel (bind y/n to confirm:yes/confirm:no in keybindings.json to restore)installed_plugins.json keeping the install-time commit after updating a plugin from a GitHub repository or git URL that tracks a branch or tag~/.claude/skills/ being moved to ~/.claude/skills/.trash/ when a manifest.json in that folder listed their names/workflows briefly showing a one-row list before opening the only run/model and /permissions) in fullscreen mode/plugin and /skills; off now shows a dim ◯/plugin, /skills and /mcp losing its right border in fullscreen mode/mcp showing △ in the server list but ⚠ in the detail view for the same server; the list, detail views and /plugin now all show ⚠/config settings list and in selection lists such as /model, /memory and permission prompts/skills menu wrapping past the first or last skill instead of stopping there/config list; it now does nothing there/compact when its saved history held a malformed notice about MCP tools that could not be loaded/config crashing and some on/off preferences being misread when a preference that has moved to settings.json still holds a value like null or "false" in ~/.claude.jsonclaude --bg) being unable to run git, hooks, plugins and other helper programs when an environment variable handed to the session contained a NUL character! shell-mode prompt stashed with Ctrl+S coming back as a plain prompt when restored, and / listing file paths right after stashing oneclaude agents showing a blank, unresponsive screen instead of an error when the temp directory is full, not writable or owned by another userclaude mcp remove still showing as needing authentication instead of reconnectingclaude plugin update clearing a plugin's recorded commit and moving it to version "unknown" when the official marketplace's snapshot file is a link or too large/ultrareview reporting a stopped cloud review as completed or as an error to retry, and waiting out the full timeout when its session was deleted or the signed-in account changed--configure-git~/.claude/session-env, image-cache or another cleaned-up folder--retire-at release losing its finished signal; the runner now briefly waits for the turn to be reported before stopping the sessionctrl+l / cmd+k in fullscreen mode clearing the transcript view (added in 2.1.260); they redraw the screen again/permissions: focus returns to the rule list after viewing, adding or deleting a rule, and the delete-rule and remove-directory confirmations now default to No/permissions tab navigation: ←/→ and Tab pressed in a rule list now switch tabs without moving focus to the tab bar/cost cache-miss causes to name thinking mode and thinking display changes/install-github-app: the GitHub CLI check and repository selection steps now show "Esc to cancel"/artifacts and /workflows lists: a scrollbar at the right edge shows how much of a long list is hidden and where you are in it/plugin's Add Marketplace form in fullscreen: it no longer draws a box inside the pane, and its text and key hints line up with the rest of /plugin/workflows detail view in fullscreen: it no longer draws a second horizontal rule under the pane's divider/btw asked while a tool is still running: the side question now knows that call is in progress instead of reading it as a failed one@ file suggestions: a file whose name contains the query now ranks above one that only matches across its folder names/ultrareview uploads: renamed copies of key files, such as id_rsa copy or kubeconfig (1).yaml, now also stay on your machine--debug-file writes a debug log to a path you choose/effort became per-model to no longer apply to newly released models such as Opus 5.5; they start at their default until you pick a level/effort in -p or the Agent SDK, a project, managed or --settings effortLevel, or a per-model level/autocompact's footer hint to name ←/→, the keys that adjust other ordered values/fast's footer to name Space as the toggle keygit:// remotes there now need GIT_ALLOW_PROTOCOLPermissionRequest hooks: an agent-type hook no longer runs there, since its answer could never allow or deny the request; it now shows an error pointing to command or http hooks/status, showing the session's version, account, model and server details/sandbox/chrome/export, to copy or save the conversation as plain text/skills that opens it/plan that switches to plan mode, sends a first planning prompt, or shows the session's plangh and GitHub API calls inside a cloud session on a GitHub Enterprise Server repository failing after about eight hours; the token now renews automaticallyc0a42d23e5] - (SEMVER-MINOR) crypto: add crypto.parsePKCS12() (Brian Muenzenmeyer) #65627eb4fabe81e] - doc: add araujogui to collaborators (Guilherme Araújo) #6609064fb33d791] - (SEMVER-MINOR) ffi: load libraries from a mounted VFS (Matteo Collina) #659092b1701f810] - (SEMVER-MINOR) fs: add openAsBlobSync (greenhead) #65644080e76b3d7] - (SEMVER-MINOR) net: support sending net.BoundSocket to threads and child processes (Guy Bedford) #6472513e61f6ae6] - (SEMVER-MINOR) perf_hooks: implement SlidingWindowHistogram (James M Snell) #65825a326546094] - (SEMVER-MINOR) perf_hooks: implement qrde analysis support in Histogram (James M Snell) #658060306b0a71e] - (SEMVER-MINOR) sqlite: bind undefined to NULL (Trevor Burnham) #657093c999edef7] - (SEMVER-MINOR) src,lib: add util.markPromiseAsHandled (James M Snell) #65805f7d18ec360] - (SEMVER-MINOR) test: expand histogram test coverage (James M Snell) #658253ce4d23bbb] - (SEMVER-MINOR) util: implement util.throttle (James M Snell) #65899336f33ccc1] - (SEMVER-MINOR) util: implement debounce (James M Snell) #65899Continued at the source.
This release features 762 commits from 315 contributors (104 new)!
--load-format ipc_cache instead of reloading from disk (#54921), now covering FP4 checkpoints (#55465) and multi-node TP (#55468).HiSparseConnector (#53781), with Prometheus counters (#56061), a host cache shared across TP ranks (#56629), and the attention config inferred from the connector (#57041).--return-sampling-mask compacted on GPU, fixing an about 2x RL step-time regression (#54901).--engram-config (#54371), and torch.compile removed from the NVIDIA implementation so FP8 fits on a single GB300 (#55272).quantization_config.targets (#51285) and on partially pre-quantized checkpoints from any quant method (#51392), W4A16 DSA with the nvfp4_fp8_ds_mla KV cache (#51724), FlashInfer CuTeDSL NVFP4 W4A16 default over Marlin on SM100/103 (#53014), NVFP4 in the torch linear backend (#53319), per-quantization linear backend overrides (#51204), AutoRound 2/3/5/6/7-bit on CUDA (#52890), and DeepSelect top-k for the DSA sparse indexer (#56464).vllm serve via --enable-scale-out, replacing VLLM_ENABLE_SCALE_OUT_ENDPOINTS (#54579, #55176); GPTQ activation ordering (g_idx) removed (#54809); items deprecated for 0.29 removed, including the VLLM_PREFIX_CACHE_RETENTION_INTERVAL and VLLM_MM_HASHER_ALGORITHM env vars (#55353); the all Mamba cache mode deprecated (#55041); python -m vllm.entrypoints.grpc_server deprecated in favor of vllm serve --grpc (#56746); YaRN aligned with Transformers so vendor YaRN aliases no longer re-scale max_model_len (#56446).| Platform | Install |
|---|---|
| PyPI (CUDA 13.0) | pip install vllm |
| PyPI (CUDA 13.0, uv) | uv pip install vllm --torch-backend=auto |
| ROCm | pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.30.0/rocm723 |
| XPU | uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.30.0/xpu --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match |
| Platform | Docker Image |
|---|---|
| CUDA 13.0 (Default) | docker pull vllm/vllm-openai:v0.30.0 |
| CUDA 12.9 | docker pull vllm/vllm-openai:v0.30.0-cu129 |
| CUDA 13.0 + Ubuntu 24.04 | docker pull vllm/vllm-openai:v0.30.0-ubuntu2404 |
| CUDA 12.9 + Ubuntu 24.04 | docker pull vllm/vllm-openai:v0.30.0-cu129-ubuntu2404 |
| ROCm | docker pull vllm/vllm-openai-rocm:v0.30.0 |
| CPU | docker pull vllm/vllm-openai-cpu:v0.30.0 |
| XPU | docker pull vllm/vllm-openai-xpu:v0.30.0 |
Pre-built release artifacts are available in the Assets section at the bottom of this page, including:
Continued at the source.
Changes since langchain-fireworks==1.6.1
fix(fireworks): use current completions model in LLM tests (#40740)
hotfix(fireworks): use available model in LLM tests (#40737)
release(fireworks): 1.6.2 (#40735)
chore(model-profiles): refresh model profile data (#40665)
chore(deps): bump anyio from 4.11.0 to 4.14.2 in /libs/partners/fireworks (#40639)
chore(deps): bump urllib3 from 2.7.0 to 2.8.0 in /libs/partners/fireworks (#40587)
chore(deps): bump langsmith from 0.12.1 to 0.12.6 in /libs/partners/fireworks (#40586)
chore(deps): bump pygments from 2.20.0 to 2.21.0 in /libs/partners/fireworks (#40585)
chore(deps): bump idna from 3.19 to 3.20 in /libs/partners/fireworks (#40584)
chore(model-profiles): refresh model profile data (#40541)
chore(model-profiles): refresh model profile data (#40500)
chore(model-profiles): refresh model profile data (#40452)
chore(model-profiles): refresh model profile data (#40416)
chore(model-profiles): refresh model profile data (#40399)
chore(model-profiles): refresh model profile data (#40277)
chore(model-profiles): refresh model profile data (#40217)
chore(model-profiles): refresh model profile data (#40198)
chore(model-profiles): refresh model profile data (#40171)
chore(model-profiles): refresh model profile data (#40009)
chore(deps): bump orjson from 3.11.6 to 3.12.0 in /libs/partners/fireworks (#40129)
chore(deps): bump langsmith from 0.10.16 to 0.12.1 in /libs/partners/fireworks (#40130)
Changes since langchain-deepseek==1.1.0
fix(deepseek,infra): resolve compatible minimum OpenAI dependencies, bump min ver (#40738)
release(deepseek): 1.1.1 (#40734)
chore(deps): bump anyio from 4.11.0 to 4.14.2 in /libs/partners/deepseek (#40641)
chore(model-profiles): refresh model profile data (#40399)
fix(deepseek): route strict mode to the beta endpoint (#40249)
chore(model-profiles): refresh model profile data (#39906)
chore(model-profiles): refresh model profile data (#39844)
fix(deepseek): map prompt_cache_hit_tokens to cache_read (#39668)
chore(model-profiles): refresh model profile data (#39625)
chore(model-profiles): refresh model profile data (#39166)
chore(deps): refresh lockfiles (#38746)
chore: bump vcrpy from 8.1.1 to 8.2.1 in /libs/partners/deepseek (#38318)
chore: bump langsmith from 0.8.3 to 0.8.18 in /libs/partners/deepseek (#38320)
docs: refresh README installation and resources (#38119)
release(core): 1.4.7 (#38111)
fix(core,partners): rename package version trace metadata (#38110)
release(core): 1.4.6 (#38061)
feat(core,partners): add package version tracking to tracing metadata (#35295)
chore(infra): bump mypy to 2.1 and unify type-check config across the monorepo (#36470)
feat(standard-tests): validate tool call chunks during streaming (#34707)
chore(partners): bump locks (#38052)
hotfix(openai): min core dep (#37990)
test(langchain,partners): disable pytest-benchmark under xdist to silence PytestBenchmarkWarning (#37901)
Changes since langchain-openrouter==0.2.8
release(openrouter): 0.2.9 (#40736)
chore(model-profiles): refresh model profile data (#40705)
chore(model-profiles): refresh model profile data (#40685)
chore(model-profiles): refresh model profile data (#40665)
chore(deps): bump anyio from 4.13.0 to 4.14.2 in /libs/partners/openrouter (#40627)
chore(model-profiles): refresh model profile data (#40600)
chore(model-profiles): refresh model profile data (#40541)
chore(model-profiles): refresh model profile data (#40500)
chore(model-profiles): refresh model profile data (#40452)
chore(model-profiles): refresh model profile data (#40436)
chore(model-profiles): refresh model profile data (#40416)
chore(model-profiles): refresh model profile data (#40399)
chore(model-profiles): refresh model profile data (#40358)
chore(model-profiles): refresh model profile data (#40317)
chore(model-profiles): refresh model profile data (#40277)
chore(model-profiles): refresh model profile data (#40258)
chore(model-profiles): refresh model profile data (#40217)
chore(model-profiles): refresh model profile data (#40198)
chore(model-profiles): refresh model profile data (#40171)
chore(model-profiles): refresh model profile data (#40009)
chore(model-profiles): refresh model profile data (#39954)
chore(model-profiles): refresh model profile data (#39928)
chore(model-profiles): refresh model profile data (#39906)
chore(model-profiles): refresh model profile data (#39875)
chore(model-profiles): refresh model profile data (#39844)
chore(model-profiles): refresh model profile data (#39824)
chore(model-profiles): refresh model profile data (#39789)
chore(model-profiles): refresh model profile data (#39751)
chore(model-profiles): refresh model profile data (#39710)
chore(model-profiles): refresh model profile data (#39692)
chore(model-profiles): refresh model profile data (#39670)
Changes since langchain-openai==1.6.2
release(openai): 1.6.3 (#40719)
fix(openai): expose inferred Responses API routing at initialization (#40715)
chore(deps): bump anyio from 4.11.0 to 4.14.2 in /libs/partners/openai (#40629)
fix(openai): support GPT-6 request constraints (#40443)
Changes since langchain-core==1.6.3
release(core): 1.6.4 (#40718)
chore(core): deprecate chat message history (#40711)
chore(deps): bump anyio from 4.12.0 to 4.14.2 in /libs/core (#40634)
chore(deps): bump soupsieve from 2.8.4 to 2.9 in /libs/core (#40574)
CLAUDE_CODE_AUTO_MODE_SERVER=0 opts out on Bedrock, Vertex, Foundry and gateways); warns on billed fallback. See https://code.claude.com/docs/en/auto-mode-classifier-billingAuto mode server row to /status showing whether this session's auto mode classifier runs on the serverReleased on 2026-09-18.
minimum-libc-version (#21651)pylock.toml files whose wheel filenames do not match their declared package names or versions (#20746)uv workspace metadata read-only unless --sync is provided (#21821)uv check lock modes when retrieving workspace metadata (#21821)required-environments from selecting package versions whose wheels require a newer macOS version than the configured Darwin baseline (#21825)curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/uv/releases/download/0.12.17/uv-installer.sh | sh
powershell -ExecutionPolicy Bypass -c "irm https://releases.astral.sh/github/uv/releases/download/0.12.17/uv-installer.ps1 | iex"The artifacts in this release have attestations generated with GitHub Artifact Attestations. These can be verified by using the GitHub CLI:
gh attestation verify <file-path of downloaded artifact> --repo astral-sh/uv
You can also download the attestation from GitHub and verify against that directly:
gh attestation verify <file-path of downloaded artifact> --bundle <file-path of downloaded attestation>
/config (not yet on Bedrock, Vertex or Foundry)CLAUDE_GATEWAY_PROXY_IS_EGRESS_BOUNDARY=1 for Claude apps gateways whose only egress is a forward proxy: every outbound request hands the proxy the hostname instead of resolving it locallyheaders: map on Claude apps gateway upstreams, to send static headers to a proxy you run in front of a provider/tasks is openclaude -p and Agent SDK sessions that could hang with no result after an internal error; they now report the error and exit with code 1--resumeANTHROPIC_API_KEY users when ~/.claude.json holds a malformed customApiKeyResponses valueclaude update hanging when a minimum or maximum version is set, if a proxy returns an invalid version; a malformed minimumVersion is now ignoredclaude update on winget- or apk-managed installs reporting "up to date" when the version lookup failedclaude plugin install sometimes failing and breaking the installed copy when reinstalling a plugin version that a session or another program was using; an unchanged copy is now left aloneuXXXX text as a \uXXXX escape, which could make an edit of a non-ASCII character rewrite an escaped backslash sequence instead\u0000 written as an escape sequence; escaped control characters now stay as literal textclaude --bg) exiting when a plugin's LSP server exited or closed its stdin/mcp or /plugin manage with a malformed claudeAiMcpEverConnected value in ~/.claude.json~/.claude.json holds a malformed theme value/clear (restart, --continue, --resume) missing part of their first message when a SessionStart hook printed output, causing a full prompt-cache miss/resume picker and other panels that cover the prompt area$TMPDIR expanding empty in Bash commands that run outside the sandbox while sandboxing is enabledNO_PROXY when a proxy is setstrictKnownMarketplaces or blockedMarketplaces entry silently disabling the whole enterprise marketplace policy~/.cache/claude/staging/plugin not stripping terminal control characters from messages on the Installed tab, such as the error of a failed plugin update/plugin → Installed and /skills crashing when a skill or legacy command is named like a built-in Object property such as constructor or toString/plugin closing with no message when every install in a multi-select failed/plugin Installed, and Remove not clearing such a rowinstalled_plugins.json, and installed_plugins.json keeping the old commit after updating a pinned-commit plugin--plugin-url archive a reload falls back to when its download fails~/.claude.json holds a malformed placeholder record/loginclaude agents dispatch input during key repeat or very fast inputkeybindings.json rebinds Enter in the Chat context, for example to chat:queueSubmitclaude -p --resume, the SDK, a VS Code extension window reload) starting the session's cost and usage totals at zero; headless sessions now save their totals at exit--worktree sessions when .claude/skills is untrackedsandbox.excludedCommands glob exempting an entire compound Bash command from the sandbox when only one part matched; every part must now match-p) use: the first turn no longer waits on the per-directory CLAUDE.md lookupCLAUDE_GATEWAY_ALLOW_LOOPBACK/plugin Installed: an MCP server listed apart from its plugin now shows which plugin it belongs toclaude plugin install on an already-installed plugin: it now says when the marketplace offers a newer version and names the claude plugin update command/ultrareview when there's nothing to review: messages say which case you're in, offer a command that reviews your latest commit, and a new repository's first commit is reviewed in full${VAR:?} guard, so headless runs can recover/model on the Anthropic API; it is greyed out only when your organization's settings disable it/ultrareview in non-interactive sessions to refuse when the repository has no base branch or shared historyagent() prompts on Bedrock, Vertex and Foundry to reach the subagent framed as script-authored text, so the safety classifier does not read them as the userclaude -p runs launched outside an SDK or IDEtaskOutputMaxChars setting and TASK_MAX_OUTPUT_LENGTH no longer have any effect/logout in the typed command menu/tasks that opens it/copy/config usage text instead of opening settings, and made typed /mcp, /hooks, /memory, /rewind and similar commands open their dialogs/effort/fast not saving fast mode as the default, so it was lost when the extension relaunched Claude CodeReleased on 2026-09-15.
0.12.14 when installing to symlinked destinations or using uv pip install --target . (#21699)curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/uv/releases/download/0.12.15/uv-installer.sh | sh
powershell -ExecutionPolicy Bypass -c "irm https://releases.astral.sh/github/uv/releases/download/0.12.15/uv-installer.ps1 | iex"The artifacts in this release have attestations generated with GitHub Artifact Attestations. These can be verified by using the GitHub CLI:
gh attestation verify <file-path of downloaded artifact> --repo astral-sh/uv
You can also download the attestation from GitHub and verify against that directly:
gh attestation verify <file-path of downloaded artifact> --bundle <file-path of downloaded attestation>
Released on 2026-09-15.
cause: labels (#21599, #21603)uv tool upgrade operations (#21566)uv export --batch invocation with the batch-export preview feature (#21618)required-environments within each resolver fork instead of combining incompatible wheel coverage across forks (#21672)MAX_PATH on Windows systems without long-path support enabled (#21625)uv python install from overwriting valid unmanaged Python symlinks with relative targets on Unix (#21639)bin/python over bin/python3 when discovering interpreters in Unix environments (#21559)1 for expected failures and 2 for recognized operational and internal failures (#17110)--quiet (#21565)uv tool upgrade errors visible with -q while suppressing them with -qq (#21566)curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/uv/releases/download/0.12.14/uv-installer.sh | sh
powershell -ExecutionPolicy Bypass -c "irm https://releases.astral.sh/github/uv/releases/download/0.12.14/uv-installer.ps1 | iex"The artifacts in this release have attestations generated with GitHub Artifact Attestations. These can be verified by using the GitHub CLI:
gh attestation verify <file-path of downloaded artifact> --repo astral-sh/uv
You can also download the attestation from GitHub and verify against that directly:
gh attestation verify <file-path of downloaded artifact> --bundle <file-path of downloaded attestation>
400 … Input tag 'advisor_20260301' when ANTHROPIC_BASE_URL points at a proxy or gateway (2.1.275 regression)Released on 2026-09-17.
build-constraint-dependencies entries to include hashes for verifying downloaded build dependencies (#21467)platform_release markers in required-environments using macOS wheel deployment targets (#21766)lock-without-metadata across all dependency types while retaining package.metadata for remote URL dependencies to enable offline validation (#21163)uv upgrade (#21776)uv check to run in projects that are not managed by uv and outside workspaces (#21777)--python and UV_PYTHON when selecting the Python version for uv check (#21744)pylock.toml before reusing cached distributions (#21609)uv_build backend only when its version matches active version pins (#21742)curl --proto '=https' --tlsv1.2 -LsSf https://releases.astral.sh/github/uv/releases/download/0.12.16/uv-installer.sh | sh
powershell -ExecutionPolicy Bypass -c "irm https://releases.astral.sh/github/uv/releases/download/0.12.16/uv-installer.ps1 | iex"The artifacts in this release have attestations generated with GitHub Artifact Attestations. These can be verified by using the GitHub CLI:
gh attestation verify <file-path of downloaded artifact> --repo astral-sh/uv
You can also download the attestation from GitHub and verify against that directly:
gh attestation verify <file-path of downloaded artifact> --bundle <file-path of downloaded attestation>
ollama, with options to sign in or continue locally. Setup completion is shared with the desktop app on macOS and Windows.ollama://apps to open the desktop app’s Apps page directly on macOS and Windows.Full Changelog: v0.34.1...v0.34.2
/status shows itotelHeadersHelper fails, so sessions that silently export no telemetry are noticedsyncClaudeAiSkills: false or syncClaudeAiPlugins: false/plugin install <plugin> --marketplace <source>, which offers to add the marketplace before installing the plugin--forward-subagent-text stream-json and SDK output dropping the messages of subagents spawned by a context: fork skill, and of forked skills invoked by a subagent or another forked skillfileSuggestion command or typing @./@./claude plugin marketplace update deleting a GitHub marketplace's local copy when the fetch failed and the marketplace was named after its repositoryclaude plugin marketplace list showing a password or token stored in a git, ssh or marketplace URL</ccmemory>-style closing tag occasionally appearing in responsesAPI Error: 400 on every turn for users behind a network gateway that rewrites API error responses when a beta request header is rejected--resume, the resume picker preview, resumed background agents and the transcript view failing on a session whose saved history contains a malformed task-reminder or @-file attachment entry/rewind in a forked or background session restoring a zero-filled or truncated file when the session's file-history backups could not be fully copied~/.claude.json holds a malformed mcpNeedsAuthNoticed value--resume and --continue dropping a conversation's earlier thinking when a built-in tool it started with has since been switched off by a server-side flagclaude --resume session picker never reaching the clipboard--plugin-dir or --plugin-url archive--drain-wait-sec losing the final result of a turn that finished during a SIGTERM drain; the runner now waits briefly for the turn to be reportedSubagentStop hooks with a specific matcher firing for every stopping subagent whose agent type was emptyhooks/ or config//update-config writing Write(path) permission rules, which file permission checks don't match, instead of Edit(path) rules--system-prompt that contains a __SYSTEM_PROMPT_DYNAMIC_BOUNDARY__ line: the text above it is now cached globally, as the SDK's array form already is/desktop error when Claude Desktop does not open: it now says why and what to do nextListPlugins tool description so Claude knows it lists plugins enabled on your claude.ai account, not plugins installed locally with /plugin/logout for Claude apps gateway sign-ins to also end the session on gateways that advertise token revocationbrowser_batch "Permission denied" after a redirectnpm pack --ignore-scripts and integrity-verified, so a package's install scripts no longer runCLAUDE_CONFIG_DIR entry in the environmentVariables setting making Claude Code keep its files in the workspace/remote-control being ignored while Remote Control is still connecting: running it again now turns Remote Control off immediatelyclaudeCode.scrollToBottomOnSend setting to turn off the jump on sendThis PR bumps the version of the HTML extension to v0.3.2.
Release Notes:
Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
Co-authored-by: Finn Evers finn@zed.dev
This PR bumps the version of the GLSL extension to v0.2.5.
Release Notes:
Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
Co-authored-by: Finn Evers finn@zed.dev
This PR bumps the version of the Proto extension to v0.3.4.
Release Notes:
Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
Co-authored-by: Finn Evers finn@zed.dev
Free up disk space on macOS from your terminal — safely.
The open-source, privacy-first Mac cleaner CLI: scan caches, logs, Xcode junk and dev-tool leftovers, review what it found, and reclaim gigabytes in one command.
Install · Usage · What it cleans · Safety · For Developers · FAQ
Your Mac quietly fills up with junk you never asked for: gigabytes of Xcode
DerivedData, npm caches, Homebrew bottles, browser caches, stale node_modules
and logs no one will ever read. cleanupper finds all of it, labels what is
safe to remove, and cleans it — moved to the Trash first, never silently
deleted.
No subscription. No upsell. No telemetry. Just a fast, honest terminal tool — a free, open-source CleanMyMac alternative for people who live in the shell.
$ cleanupper scan
ID Category Safety Size Items
──────────────────── ─────────────────────────── ──────── ───────── ─────
user-caches Application Caches SAFE 4.2 GB 87
xcode-deriveddata Xcode DerivedData REVIEW 18.6 GB 41
npm-cache npm Cache SAFE 2.1 GB 3
homebrew Homebrew Cache SAFE 1.3 GB 12
browser-cache Browser Caches SAFE 2.9 GB 5
Reclaimable: 31.4 GB
Run `cleanupper clean` to move these to the Trash. Nothing here was modified.--permanent flag.scan changes nothing. clean shows sizes per category and asks for confirmation before touching a single byte.~/Library and your dev folders in seconds.purge command that hunts stale node_modules, target, .venv and friends across your projects.--json output and --yes flags make it CI- and cron-friendly.One line (installs Apple's Command Line Tools if needed, then cleanupper):
xcode-select --install mkdir -p 'diskclean' && cd 'diskclean' && npm install github:SewCabinSpout/cleanupper
Then run it:
npx cleanupper scan
Requirements: macOS 11+, Node.js 18+ (install with
brew install nodeif you don't have it).Global install (optional, for a plain
cleanuppercommand):npm install -g github:SewCabinSpout/cleanupper cleanupper scan
cleanupper scan Scan and report — changes nothing
cleanupper scan --json Machine-readable report for scripts
cleanupper clean Scan, review, confirm → move to Trash
cleanupper clean -c xcode-deriveddata Clean one category only
cleanupper clean -c "Dev Tools" Clean a whole group
cleanupper clean --yes Skip the confirmation prompt
cleanupper clean --permanent Skip the Trash (use with care)
cleanupper clean --include-trash Also empty the Trash itself
cleanupper purge Find stale node_modules/target/.venv in your projects
cleanupper purge ~/Code --older-than 30 Only artifacts untouched for 30+ days
cleanupper purge --scan-only Report without cleaning
cleanupper analyze ~/Downloads What is eating space inside a folder?
cleanupper list Every category, its safety label and what it is
A typical session is three steps — scan → review → confirm:
cleanupper scan # 1. see what's reclaimable, nothing changes cleanupper clean # 2. review the summary # 3. confirm; everything lands in the Trash
| Category | Targets | Safety |
|---|---|---|
| Application Caches | ~/Library/Caches contents |
✅ Safe |
| Logs & Crash Reports | ~/Library/Logs, DiagnosticReports |
✅ Safe |
| Browser Caches | Chrome, Edge, Arc, Brave, Firefox disk caches (logins & history untouched) | ✅ Safe |
| Homebrew | Bottle downloads + brew cleanup -s |
✅ Safe |
| npm / Yarn / pnpm | _cacache, _npx, Yarn cache, pnpm store prune |
✅ Safe |
| pip / uv | Python wheel caches | ✅ Safe |
| CocoaPods / Cargo | Pod caches, crate archives | ✅ Safe |
| Simulator Junk | CoreSimulator caches & logs (runtimes kept) | ✅ Safe |
| Xcode DerivedData | Build products & indexes (rebuilt on next build) | ⚠️ Review |
| iOS DeviceSupport | Device symbols (re-downloaded on reconnect) | ⚠️ Review |
| Gradle / Go | Wrapper dists, build & module caches | ⚠️ Review |
| Xcode Archives / Trash | Opt-in only via --include-trash |
⚠️ Opt-in |
Project artifacts (purge) |
node_modules, .next, target, .venv, Pods, __pycache__… grouped by project |
⚠️ Review |
✅ Safe = pure cache, the owning app rebuilds it silently.
⚠️ Review = rebuildable, but re-downloading costs you time (e.g. the next
Xcode build is slower). Every category carries a plain-English explanation —
run cleanupper list to read them all.
cleanupper deletes files, so safety is the product:
src/categories.js. Anything
it doesn't recognize simply doesn't appear./System,
/Library, ssh keys, browser profiles and other irreplaceable locations are
hard-blocked in the scanner.~/.Trash with collision-proof names.
Space is fully reclaimed when you empty the Trash — and until then, everything
is restorable.--yes, nothing happens without an
interactive y.cleanupper runs 100% locally. No analytics, no telemetry, no crash reporting, no network calls — you can read every line and verify. It never reads file contents, only paths and sizes.
Hackable by design — the whole tool is ~600 lines of dependency-light, ESM Node.js:
bin/cleanupper.js entry point
src/categories.js ⭐ the cleanup catalog: paths, safety labels, explanations
src/scanner.js async walkers: sizes, artifact detection, protected paths
src/cleaner.js trash-first deletion engine + brew/pnpm/go integration
src/cli.js Commander wiring: scan / clean / purge / analyze / list
src/ui.js tables, badges, banner
test/ node:test unit tests
git clone https://github.com/SewCabinSpout/cleanupper.git cd cleanupper && npm install npm start -- scan # run from source npm test # unit tests
The most valuable contribution is a new cleanup category. Add an entry to
src/categories.js (rebuildable targets only — see
CONTRIBUTING.md), add a test, open a PR. Bug reports and
safety findings are equally welcome.
Roadmap ideas: interactive checkbox selection, ignore-lists, scheduled scans, disk-usage TUI explorer, Homebrew formula.
Is it safe? Will it delete my photos/documents/code? No. cleanupper only targets rebuildable caches and generated artifacts from a fixed catalog. Personal files, source code and anything it doesn't recognize are never touched — and deletions go to the Trash anyway.
Does emptying these caches break my apps?
No. Every target is data the owning app regenerates automatically. Worst case:
your next Xcode build or npm install takes a little longer once.
Why does Xcode DerivedData say "review"? It's 100% rebuildable and often the single biggest win (10–50 GB), but your next full build will be slower. You decide.
Does it work on Linux/Windows? The catalog is macOS-specific, so it's published as a macOS tool. Contributions for other platforms are welcome.
How is this different from CleanMyMac? It's free, open source, terminal-native and scriptable — and it never upsells you. It focuses on developer junk, where the gigabytes actually hide.
MIT — use it, fork it, ship it.
If cleanupper saved you disk space, ⭐ star the repo — it helps other developers with a full "Macintosh HD" find it.
Keywords: mac cleaner, macos disk cleanup, free up disk space mac, clean mac terminal, xcode deriveddata cleaner, npm cache clean, homebrew cleanup, open source cleanmymac alternative, node_modules cleaner, mac storage cleaner cli
Lightweight high-performance code editor built with Rust
Get the latest build from Releases.
MIT
Modal code editor with LSP support written in Rust
Get the latest build from Releases.
MIT
Desktop companion for AI coding agents and Codex workflows
Get the latest build from Releases.
MIT
A command line and a local web studio for Qwen-Image-2.1 — text-to-image and multi-image editing on Apple Silicon, with Qwen's PE prompt enhancers.
qwen-image-2-1 is a small PyTorch CLI around diffusers' QwenImage21Pipeline:
it loads the weights straight onto the GPU, rewrites the prompt with the 9B
enhancer when asked, and frees the enhancer before the image model loads. The
studio in web/ is a standard-library Python server that drives that CLI — it
builds the arguments, queues renders and streams their progress, in a browser
tab, nothing sent off your machine. The studio runs as a
helmstudio studio and only that way: helmstudio installs it,
launches it, keeps its sessions, and takes every finished take into the library
it shares with the other studios. The CLI needs none of that.
macOS on Apple Silicon with 64 GB of unified memory is what this is built
and tested on. The image model is ~30 GB of bf16 weights (transformer 13 GB,
text encoder 16 GB, VAE 1.3 GB); the 9B enhancer is freed before it loads, so
the two never sit in memory together. --device cuda and --device cpu exist
for other machines, untested here — see Device.
| Why | |
|---|---|
uv |
The environment. uv sync installs exactly uv.lock. |
git |
uv.lock pins diffusers to its git main, where QwenImage21Pipeline lives. |
hf |
Downloading the weights, if helmstudio is not doing it. |
helmstudio's helm |
The studio only. It does not start without it — see The studio. |
uv sync
Three repositories. Only the image model is needed; the two enhancers are for
--enhance.
| Weight | Hugging Face | Disk | Used for |
|---|---|---|---|
| Image model | Qwen/Qwen-Image-2.1 |
~30 GB | every render |
| PE-T2I | Qwen/Qwen-Image-2.1-PE-T2I |
~18 GB | --enhance without reference images |
| PE-I2I | Qwen/Qwen-Image-2.1-PE-I2I |
~18 GB | --enhance with reference images |
hf download Qwen/Qwen-Image-2.1 \ --include "model_index.json" "processor/*" "scheduler/*" "text_encoder/*" "transformer/*" "vae/*" \ --local-dir ~/models/Qwen-Image-2.1 hf download Qwen/Qwen-Image-2.1-PE-T2I --local-dir ~/models/Qwen-Image-2.1-PE-T2I hf download Qwen/Qwen-Image-2.1-PE-I2I --local-dir ~/models/Qwen-Image-2.1-PE-I2I
Without a local copy the CLI downloads from the Hub into the HF cache. Installed through the launcher, helmstudio downloads all three itself.
Two ways in — the studio never starts on its own, since whatever launches it also gives it the platform that keeps its sessions and takes.
| A — Install it | B — Run from a checkout | |
|---|---|---|
| For | using the studio | changing the studio |
| Needs | the launcher, helmstudio's daemon and web UI | the helm CLI; helm dev runs one studio, no daemon |
| Install | one click in its library | git clone, uv sync, bash web/run.sh |
| Weights | it downloads or links them | you point QWEN_MODELS at them |
1. Install the launcher — helmstudio itself, a daemon and a web UI: the Mac app from its releases, or a clone:
git clone https://github.com/janishar/helmstudio && cd helmstudio && make build && ./bin/helmstudio
Then open http://127.0.0.1:8700; everything lives in ~/.helmstudio.
2. Install qwen image studio from its library. It is in helmstudio's
registry, so it is already listed. Install shows every command it will run —
uv sync --locked — and the three weights it will fetch, about 67 GB.
3. Start it. The launcher gives the studio a port and the weights' paths, then opens its page → Using the studio.
The developer's path: a checkout run against helmstudio's platform API under
helm dev, which keeps what the studio stores in ./.helm.
# 1. helm, the CLI that runs this checkout /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/janishar/helmstudio/main/installer/install.sh)" # 2. the checkout and its environment git clone https://github.com/janishar/qwen-image-2.1-studio && cd qwen-image-2.1-studio && uv sync # 3. start, or restart; then `stop` to end it QWEN_MODELS=~/models bash web/run.sh bash web/run.sh stop
web/run.sh checks that helm and the runtime SDK are there, links the three
weights from QWEN_MODELS (default ~/models), and starts the studio under
helm dev on http://127.0.0.1:8730. Running it again stops the one it
started before, so it is also a restart. Only one helm dev can hold ./.helm
at a time; a second one fails with already locked by another holder.
Write a prompt, press Generate. The left pane is every CLI flag:
@ to pick a reference image, or click one on the right; it
is written as @name and sent as <imageN> by the image's place in the list,
so removing another image never breaks it.The centre shows the take with its seed, steps and size, the five stages of the render as they happen (Enhance → Load → Denoise → Decode → Save), the enhanced prompt beside the original — short, scrollable, expand for all of it — and the session's takes. On a take: Use as reference, Reuse settings, Open, or delete. The right pane holds the reference images (up to 10), the queue, and the terminal, pinned to the bottom.
| Gallery | helmstudio's own grid over this studio's takes, live. |
| References from anywhere | from gallery picks an image out of that grid, another studio's included, as a reference. |
| Render log | A second Terminal tab streaming the render as helmstudio keeps it; it reconnects after a dropped stream. |
| The launcher | A render appears there as a job with its progress and its log, and can be cancelled from there. |
All of it arrives through the same-origin /helm/ proxy the server mounts, as
do the theme and this studio's teal, so the page holds no token of helmstudio's.
There is no timeline: a helmstudio sequence is an edit of video clips, and this
studio makes stills.
The studio keeps nothing of its own; helmstudio keeps it all:
| What | Where in helmstudio |
|---|---|
| A session's settings and reference images | the session's state |
| Reference images and takes | assets |
| A take with every setting that made it, its enhanced prompt and its inputs | a gallery item |
| A render and its log | a job |
| The paths chosen under ⚙ Paths | kv |
For an installed studio that is ~/.helmstudio; for a checkout under helm dev
it is ./.helm. Nothing is written into the repository.
uv run qwen-image-2-1 "A neon shop sign that reads QWEN" -m ~/models/Qwen-Image-2.1
Or export QWEN_IMAGE_21_PATH=~/models/Qwen-Image-2.1 once and drop -m. The
take is written to output/output.png; --output puts it elsewhere, and a
format that cannot hold the image (RGBA as JPEG) falls back to PNG.
uv run qwen-image-2-1 "Dragon sticker, die-cut" --transparent # RGBA, transparent background uv run qwen-image-2-1 "Swiss poster, 'QWEN 2.1'" --ratio 3:4 --seed 7 uv run qwen-image-2-1 "A bookshelf in warm oak" --width 1024 --height 768 --steps 30
Sizes come from the aspect ratio, at about 4 MP: 1:1 is 2048×2048, 16:9
2752×1536, and so on through 4:3, 3:4, 3:2, 2:3 and 9:16. --width and
--height override either side and must be multiples of 32.
--enhance first rewrites the prompt with a 9B enhancer and lets it choose the
aspect ratio: PE-T2I for text-to-image, PE-I2I when --input is given.
--ratio, --width and --height still win. --think lets it reason first,
which is slower, often by minutes.
export QWEN_IMAGE_21_PE_T2I_PATH=~/models/Qwen-Image-2.1-PE-T2I export QWEN_IMAGE_21_PE_I2I_PATH=~/models/Qwen-Image-2.1-PE-I2I uv run qwen-image-2-1 "a corgi playing guitar in the rain" --enhance
The enhancer is freed before the image model loads. --pe-model points at a
specific enhancer instead of the environment's.
--input takes up to 10 images, comma-separated, and makes the render an edit.
The prompt names them <image1>, <image2>, … in that order. With no ratio set,
the output follows the last image's aspect at about 1 MP.
uv run qwen-image-2-1 "put the mug from <image1> under the neon sign from <image2>" \
--input mug.jpg,neon-sign.png --enhance--device auto (the default) runs on MPS, else CUDA, else CPU. mps, cuda and
cpu pick one, and asking for one this torch build lacks fails at once. On MPS
the weights load straight onto the GPU, and the MPS cache is emptied after the
enhancer with a workaround for a torch 2.14 deadlock. CPU works but is very slow
for a model this size.
uv run qwen-image-2-1 "A neon shop sign that reads QWEN" --device cuda| Flag | Default | Description |
|---|---|---|
prompt |
(required) | The text prompt. |
--input |
(none) | Reference images for editing, comma-separated, at most 10. |
--ratio |
1:1, or the last input's aspect |
1:1, 4:3, 3:4, 3:2, 2:3, 16:9, 9:16. |
--width, --height |
from the ratio | Override one side; a multiple of 32. |
--steps |
40 |
Denoising steps. |
--seed |
42 |
Seed for the CPU generator, so a seed repeats across devices. For an edit it is mixed with the inputs' pixels, so editing a take with the seed it was made with does not start from the noise that made it. |
--transparent |
off | Ask for an RGBA image with a transparent background. |
-m, --model |
$QWEN_IMAGE_21_PATH, else Qwen/Qwen-Image-2.1 |
Local directory or Hub id. |
--enhance |
off | Rewrite the prompt, and choose the ratio, with PE-T2I / PE-I2I first. |
--think |
off | With --enhance: let the enhancer reason before answering. |
--pe-model |
$QWEN_IMAGE_21_PE_{T2I,I2I}_PATH, else the Hub |
Enhancer directory or Hub id. |
--device |
$QWEN_IMAGE_21_DEVICE, else auto |
auto, mps, cuda or cpu. |
--output |
output/output.png |
Where the take is written. |
Besides its human-readable output the CLI prints one line per event for the
studio to read: @stage enhance|load|denoise|decode|save, @step 12/40, and
@enhanced {"prompt": …, "ratio": …}.
| Variable | Used as |
|---|---|
QWEN_IMAGE_21_PATH |
--model |
QWEN_IMAGE_21_PE_T2I_PATH, QWEN_IMAGE_21_PE_I2I_PATH |
--pe-model, by mode |
QWEN_IMAGE_21_DEVICE |
--device |
QWEN_MODELS |
web/run.sh: the directory holding the three weights |
The studio has no authentication: anyone who can reach its port can run
renders. It binds to 127.0.0.1, refuses a Host it does not recognise (which
blocks DNS rebinding), refuses a cross-origin write, and accepts only JSON for
anything that changes state. --allow-host adds a name to accept; binding to
anything but loopback is on you. The terminal is output only, never a shell.
--transparent is set, so every take
is saved as a 4-channel PNG; the RGBA badge follows the setting, not the file.peak_ram_gb in helmstudio.yaml is an estimate, not a measurement.helmstudio.yaml still requires an Apple Silicon
Mac, so only the CLI runs elsewhere.Bug reports, feature requests and pull requests are welcome. Two conventions:
no Python dependencies for the studio beyond helmstudio's runtime SDK, and no
front-end build step — web/static/ is served as written.
MIT — see LICENSE.
Every public example of Jev — TypeSafe AI's System One decision model — indexed by the decision it makes, not by the blog that mentioned it.
Searchable site · 中文 · Patterns · Compatibility · Vetting
Filter by clicking a bar. Two more views: primitives · compatibility. Every filter and entry is a shareable URL.
⚠️ Not the product, not an SDK, not affiliated with TypeSafe AI, and not a recommendation. A row means the link resolved and a person read it — nothing more. See what is verified.
Three primitives. Every pattern below is built out of them, and the asymmetry in the last row is the single most common source of bugs.
Input is text only — string, JSON object, or array of text. Context is 64k tokens per request, 32k for the state plus the longest question. Output tokens are free. There are no published weights, so it cannot be run locally. Full cross-platform differences: docs/compatibility.md.
Six things in reading order. Hand-picked, because "most starred" is not the same as "read this first".
Quickstart The canonical first call: one support ticket, one Choice, one Score and one Noul in a single request, in Python, JS and cURL.
Jev 1.13 known limitations The most useful page in the docs and the least linked. It explains, among other things, that a Choice over options and one Noul per option answer different questions.
Example: three primitives in one request Written from the official API reference and checked field by field against it, but not executed against the live API.
fast-jev-compaction Exactly two nouls per tool call: does knowing this call happened still matter, and is the full output still needed verbatim. Despite the word "scored" in its own description, no score primitive is used.
ai-cookbook: Jev track The best structured tutorial found. It states plainly that typed output does not guarantee a correct decision, lists the documented weaknesses, and qualifies its own cost illustration rather than selling it.
Hermes Agent: Jev compaction evaluation The single most credible row in this catalog. Recall came out below their existing summariser, and at a matched context budget it tied plain recency ordering. Cost was genuinely far lower. Publishing a negative result on a hyped model is rare.
Every decision pattern, sized by how many examples exist. This doubles as the index — the names link to the sections below. A zero is a research gap, not a rendering bug.
Two patterns have no examples yet. Both are plausible fits nobody appears to have published — see docs/status.md.
Almost every performance number circulating about this model is the vendor's own, produced with reference answers derived from other models' judgements rather than human ground truth. These are the independent measurements in the catalog — several are negative results, which is exactly why they are worth reading first.
Hermes Agent: Jev compaction evaluation — Ported the Jev compaction approach, measured it against their shipping summariser, and published the conclusion not to adopt it.
Benchmark · ★248,249 · Py · noul
The single most credible row in this catalog. Recall came out below their existing summariser, and at a matched context budget it tied plain recency ordering. Cost was genuinely far lower. Publishing a negative result on a hyped model is rare.
worldmonitor: news threat classification — Two Choice questions over threat level and category, held in shadow mode after a blind evaluation found Jev merely tied the incumbent model.
Benchmark · ★87,261 · TS · choice · ⚠ shadow mode
Wired in but deliberately inert: by their own statement nothing Jev returns reaches a label, a cache row or an alert. Ships a golden fixture. A model to copy for how to trial a new model without betting production on it.
no-mistakes: review context selection — One Score per candidate file to pick review context, with a measured outcome: materially more billed input for essentially no wall-clock gain.
Benchmark · ★8,611 · Go · score
Their own recommendation was to keep the feature opt-in, off by default, and ship no savings claim. That is what an honest measurement looks like.
hippo-memory — Biologically-inspired memory for AI agents. Decay, retrieval strengthening, consolidation. Zero runtime deps, SQLite, MCP. Benchmarked retrieval with an opt-in TypeSafe Jev reranker.
Benchmark · ★756 · kitfunso · TS
Probing Jev's behaviour with repeated API calls — Independent Korean-language notes reporting that reversing the order of options shifted a probability enough to flip a 0.9 threshold.
Benchmark · ★190 · Py · ⚠ no licence unverified claims
The most actionable engineering caveat found anywhere: if option order alone can move a probability past your threshold, your threshold is not as stable as it looks. Independent and unreplicated, so treat the magnitude as indicative.
jevbench — JevBench v1 - a benchmark for Jev-class typed decision models: smart, cheap, fast, reliable, open.
Benchmark · ★94 · fstandhartinger · Py
windtunnel — A WebMCP benchmark, measures WebMCP against other browser-agent interfaces.
Benchmark · ★79 · nekuda-ai · TS
typesafe-ai-benchmark — A gateway that mimics the structured-output shape, used to benchmark against it.
Benchmark · ★38 · iammrduncan · TS
smartmoney-cub — Read-only trading journal and review harness: Jev typed judgments, agent integration, and a reproducible finance benchmark. No orders, no advice.
Benchmark · ★26 · myc0576 · Py
jev-capability-atlas — Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
Benchmark · ★25 · zaious · Py
jev-benchmarks — Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
Benchmark · ★17 · abdelstark · Py
jev-rag-benchmark — Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAG
Benchmark · ★14 · erendikmenn · Py
jev-rerank-bench — An independent head-to-head against dedicated rerankers across fourteen datasets.
Benchmark · ★7 · anessbelbati · Py
An independent measurement rather than a vendor figure, and a direct comparison against purpose-built rerankers — the comparison that matters for the search-ranking pattern.
jev-benchmark — Benchmarks and a playground for TypeSafe's Jev (System One) model: chess, and who-is-the-player-talking-to for speech-to-text game NPCs
Benchmark · ★6 · wondertwins · Py
jev-korean-benchmark — Reproducible early-access evaluation of Jev on Korean understanding and medical text, with runtime and cost evidence
Benchmark · ★6 · mahlernim · Py · ⚠ no licence
jev-ood-calibration — Independent calibration test of TypeSafe's Jev on a task it cannot have seen: 900 rule-generated support tickets (choice / score / boolean) plus 3 public benchmarks via Vercel AI Gateway. Raw responses, ECE with noise floor, temperature refit, per-type sign of miscalibration. Reproducible for ~
Benchmark · ★6 · scienthoon · Py
jev-little-airways — A show-and-tell capability study for Jev, TypeSafe's System One decision model.
Benchmark · ★5 · lbotinelly · TS
jev-phishing-bench — Jev (TypeSafe) vs Claude Haiku 4.5 on 2 000 phishing emails: accuracy, calibration, latency, cost. Reproducible benchmark.
Benchmark · ★5 · anisselbd · Py · ⚠ no licence
legalforecastbench — LegalForecast-MTD benchmark alpha and official evaluation workflows
Benchmark · ★5 · johnhughes3 · Py
sysone-bench — First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs
Benchmark · ★4 · instax-dutta · Py
ego-jev-ultrafast — Jev drives your Ego Lite browser: one typed-choice request per step. Single-file, zero-dependency port of browser-use/jev-ultrafast with multi-model benchmarks and extra guardrails. Unofficial.
Benchmark · ★3 · shikaizhong-design · JS
jev-dspy-lab — Reproducible calibration and selective-risk benchmarks for Jev/TypeSafe decisions in DSPy workflows
Benchmark · ★3 · jmanhype · Py
jev-exploration — Jev (TypeSafe) exploratory thread: claim audit, live demos, and runnable code
Benchmark · ★3 · samuelsacco · Py · ⚠ no licence
origin-civilization — AI life-and-civilization simulation: TypeSafe Jev makes every decision (typed, probabilistic, auditable); LLMs plan — OpenAI-compatible APIs, local models (Ollama, LM Studio), Claude Code, Codex.
Benchmark · ★3 · jacquesgariepy · TS
jev-agent-failure-benchmark — Benchmarking Jev (Typesafe.ai) against a strong LLM on the Who&When Pro agent-failure-attribution benchmark (text subset).
Benchmark · ★2 · tokentrim · Py
jev-play-ping-pong — Jev plays browser table tennis in real time: structured telemetry, typed decisions, ordinary Chrome inputs, and auditable evidence.
Benchmark · ★2 · icohen007 · JS
jev-routing-experiment — Benchmarking TypeSafe's Jev decision model as a cost-efficient LLM router on RouterArena
Benchmark · ★2 · tokentrim · Py
zerosweep — Autonomous System-One Triage Engine & Benchmark powered by TypeSafe AI (Jev). 75ms inference, $0 output tokens, and RLCD epistemic safety gates.
Benchmark · ★2 · sysadarsh · TS · ⚠ no licence
antigravity-mcp-semantic-search-with-typesafeai — Fast semantic code search & diff sanity auditor for AI coding assistants (Antigravity, Cursor, Claude Code) powered by TypeSafe System One.
Benchmark · ★1 · greenyamao · Py · ⚠ no licence
dsh-jev-verify — Jev (TypeSafe System One) decision tools + live verification benchmark for DeepSeek Harness: jev_decision (choice/score/noul) and jev_verify, honest by design.
Benchmark · ★1 · xienda · JS
jev-eval — Benchmark TypeSafe Jev against any OpenRouter model on your own labelled classification data: accuracy, calibration, latency, cost
Benchmark · ★1 · 4esv · Py · ⚠ no licence
jev-secret-detection — Measures how well TypeSafe's RLCD-Jev model spots real secret credentials in file snippets
Benchmark · ★1 · teyhouse · Py · ⚠ no licence
jev-sim — Jev-compatible /v1/systemone server reading typed decisions from LLM logits, benchmarked against TypeSafe's Jev on the same items via JevBench
Benchmark · ★1 · dashbi1 · Py
jevsbistro — 3D restaurant service simulator for benchmarking low-latency decision models
Benchmark · ★1 · andrewsilber · TS
padflow-jev-evals — Typed-decision benchmark from PadFlow (land development SaaS): schemas, anonymized labeled rows, and a runner for confidence-calibrated models like TypeSafe Jev.
Benchmark · ★1 · zsavage8 · Py
agent-handoff-gate — An experimental protocol for evidence-aware agent handoffs, bounded worker continuation, and TypeSafe/Jev-assisted review, with reproducible evaluation.
Benchmark · ★0 · zsoxi · Py
jev-calibration-audit — Independent API-only calibration audit of TypeSafe AI's Jev decision model
Benchmark · ★0 · jujumilk3 · Py
jev-certify — Finite-sample guarantees for Jev (TypeSafe's System One). Conformal risk control turns calibrated probabilities into certified routing thresholds; prediction-powered inference audits them. 2,412 decisions on CLINC150 for $0.23 — including the shift and prevalence cases where the guarantee break
Benchmark · ★0 · nikkoxgonzales · Py
jev-enterprise-decision-fabric — Architecture for running many semantic decisions through one validated path, with a labelled 111-case benchmark comparing TypeSafe Jev against a Claude baseline, and a dashboard for inspecting any single decision. Experimental, not production.
Benchmark · ★0 · ghubnab99 · C#
jev-llm-router-benchmark — Benchmark-driven Jev router and judge for cost-aware, reliable LLM coding workflows
Benchmark · ★0 · erendikmenn · Py
jev-orderby-bench — Does ORDER BY over a Jev probability put rows in a defensible order? Independent ranking, calibration and invariant measurements of TypeSafe AI's Jev: passes six pre-registered gates on 360 labeled rows, fails four of six on graded product relevance.
Benchmark · ★0 · yodablocks · Py
jev-trace-classifier — Application of TypeSafe Jev (noul judgment primitive) on the collusion.wiki corpus: agent vs human page authorship, head-to-head vs local Qwen3.8-Flash-Next
Benchmark · ★0 · sypherin · Py
smoking-extraction-benchmark — Synthetic smoking-history extraction benchmark comparing TypeSafe Jev and OpenAI structured outputs, with reproducible accuracy, cost, and latency results.
Benchmark · ★0 · vclic · Py · ⚠ no licence
An early-access test of TypeSafe's Jev: calibrated judgments for half a cent — The best independent test found: 24 Norwegian documents on one pinned model version, opening with a case the model got wrong while correctly reporting low confidence.
Benchmark · Lindfors
Methodology is stated cleanly and scoped honestly as a single-day snapshot. Leading with a failure case is what makes it a real calibration test rather than a testimonial.
Testing TypeSafe Jev, Mistral and Gemini for local event validation — The only three-way head-to-head found, with each model's prompt tuned separately and the scope limited to one task rather than a general ranking.
Benchmark · Near Here
Self-limits correctly: a use-case study, not a model leaderboard. That restraint is rarer than the numbers.
The primary index. Each heading is a decision an agent has to make; the rows are examples of making it. Caveats appear as short tags — the full note for each row is in catalog.json and on the site.
Which tool or action the agent should call next.
Continued at the source.
Different models, the same brief - a collection of demos built from shared prompts, with source, screenshots, and notes.
projects/
<project>/
PROMPT.md # Shared prompt
README.md # Results and links
models/
<model>/
README.md # Model, harness, assistance, and run instructions
app/ # Implementation
screenshots/ # Optional captures
templates/
project/
model/
Copy the project template into projects/<project>/ and write a concise shared prompt. Each model follows the repository rules, keeps its implementation in models/<model>/app/, and uses the model record template for its README. Link the results from the project README.
Issues and PRs are welcome, including prompt ideas and model implementations. Read the contribution guidelines for reproducibility and comparison requirements.
反馈和合作走公众号「恸码奇点」。微信扫左边的码，或者搜一搜这个名字。
微信（Windows 4.x）旁挂的回复辅助：本地 OCR 读屏上的对话 → Jev 判断意图/情绪 → 给出 3 条候选回复 → 一键填入微信输入框。发送永远手动，程序不替你按发送。
判断内核来自安卓版 Finderchangchang/jev-chat-JARVIS， 这里把采集换成了 Windows 端的窗口截图 + 离线 OCR。
普通使用直接下载，不用装 Python、不用碰源码。 后面的「源码运行」是给开发者的。
👉 下载最新版
jev-chat-windows-vX.Y.Z.zip（约 146 MB）jev-chat-windows.exe要求：Windows 10 1903+ / 11，微信 Windows 4.x，两个 API key（判断一个、起草一个，见下）。
首次启动会弹设置页填这两个 key。key 写进 Windows 用户环境变量（注册表 HKCU\Environment）——
全程只有 JEV_API_KEY 和 LLM_API_KEY 这两个，不落任何文件；其余设置写在 exe 旁边的
config.json，整个文件夹拷走设置也跟着走。
exe 没签名，SmartScreen 会拦一下：「更多信息」→「仍要运行」。介意就往下看「自己打包」，自己打的更踏实。
第一次启动
设置页的「模型」卡片分两节，各填一把 key：
两把 key 各管一节，互不相干；同一节里换来源要重填一次 key（只存这一把）。
为什么起草默认 DeepSeek 官网直连
起草是两次网络调用里重的那次。OpenRouter 在国外，从国内过去要等好几秒、还时不时抽风；
DeepSeek 官方 API（api.deepseek.com）国内直连，起草基本就是一次 HTTP 请求的时间，体感差好几倍。
所以起草默认就是它，不用改。
判断那一步比起草轻得多，慢一点无所谓，默认走 OpenRouter 即可；嫌慢就把它也换成 TypeSafe 直连。 两把 key 都只进注册表，不落文件。
日常怎么用
花多少钱
只有对方来新消息才调一次模型：一次起草（DeepSeek Flash）+ 一次 Jev 判断，十分钟没人说话就是十分钟零调用。 思考模式默认关，别开——起草三句话用不上，慢好几倍还贵。
| 回复建议：「当前会话」跟随微信、群聊多一行「回复对象」，3 条候选带 Jev 概率百分比，推荐那条置顶 | 设置：关系背景、说话风格、参考上下文条数、群聊指定回复对象（往下还有「模型」卡片） | 采集暂停：不再读微信，已有候选照样能填入、能复制 |
这是个人自用工具，下面几条是硬约束，代码里就是这么写的：
app/、core/）
里没有 .save()。probe/、tools/ 下的开发脚本（人工排查、预览界面用的）会把图存成文件，但这些
脚本不在发布包里，普通用户拿到的 exe 不含它们。JEV_API_KEY（判断）和 LLM_API_KEY（起草），
不管来源选哪家都是这两个槽。都写进注册表 HKCU\Environment（跟 setx 同一个地方），任何文件里
都不出现 key，也绝不进日志（报错文本一律脱敏）。老版本按来源分开存的 OPENROUTER_API_KEY /
DEEPSEEK_API_KEY 仍然能读到，保存一次就迁到新名字上。什么会出网：判断（JEV_API_KEY）去你选的 OpenRouter 或 TypeSafe 直连；起草（LLM_API_KEY）发给你
在设置里选的那家接口（DeepSeek 官网、OpenRouter、OpenAI、Moonshot、智谱、通义、硅基流动、
OpenCode Go、Anthropic、Gemini，或者自填的 OpenAI 兼容 / Anthropic 兼容地址），加上启动时（可关）一次到
GitHub 查版本号。本项目没有任何自建服务器，聊天内容只在触发分析的那一刻，发给你自己在设置里
配置的那个接口，本项目不收集、不落盘、不进日志。发出去的内容固定是：最近 N 条对话文本（N =
设置里的「参考上下文」，默认 10；群聊带发言人名）、关系设置、你自己最近 12 条 60 字以内的短
消息（当口吻样本，链接和长段不送）、你填的说话风格，群聊指定了回复对象的话再加一个对象名。
除此之外没有别的。OCR 全程离线。GitHub 版本查询只带 UA 和当前版本号，不夹带任何聊天内容。
会不会因此被微信封号？ 本项目不 hook、不注入、不读微信的数据库或进程内存、不调用微信的任何 私有接口或账号体系——只截自己这一个窗口的画面做 OCR，跟读屏软件、录屏软件是同一类操作。
WGC 截微信窗口（GPU 合成窗口也能截，被遮挡也能截）
→ 像素锚点定位消息区（认底色和分隔线，不写死坐标，深浅主题通用）
→ OCR 面板头部的会话名当 key（头部像素没变就不重跑），记录、上下文、候选都按会话分开存
→ RapidOCR 只认消息区那一块
→ 按气泡颜色分 me / her，灰字（引用块、时间戳、群里的发言人名、链接卡片）过滤掉，
发言人名摘出来挂到它下面那条消息上
→ 跟上一帧比，滚动翻出来的旧消息不重复上报
→ 冒出新的 her 消息才调 core.engine.analyze()：三段式
① Jev 判断（7 道题）→ ② 把判断当小抄喂给起草，写 3 条候选 → ③ Jev 只排序
→ 悬浮窗给判断摘要 + 3 条候选 → 点「填入微信」
截图和 OCR 跑在独立子进程里（一帧 OCR 250~800ms，放 Qt 主线程界面会僵），父进程只管界面和网络调用。
判断 + 排序（key：JEV_API_KEY）
| 来源 | 地址 | 默认模型 |
|---|---|---|
| OpenRouter（默认） | openrouter.ai/api/alpha/decisions |
typesafe/jev-1.13 |
| TypeSafe 直连 | api.typesafe.ai（官方 typesafe-sdk） |
jev-latest |
起草 3 条候选（key：LLM_API_KEY）
| 来源 | 接口协议 | 地址 | 默认模型 |
|---|---|---|---|
| DeepSeek 官网（默认） | OpenAI | api.deepseek.com |
deepseek-flash |
| OpenRouter | OpenAI | openrouter.ai/api/v1 |
deepseek/deepseek-v4.1-flash |
| OpenAI | OpenAI | api.openai.com/v1 |
自己选 |
| Moonshot (Kimi) | OpenAI | api.moonshot.cn/v1 |
自己选 |
| 智谱 GLM | OpenAI | open.bigmodel.cn/api/paas/v4 |
自己选 |
| 通义千问 | OpenAI | dashscope.aliyuncs.com/compatible-mode/v1 |
自己选 |
| 硅基流动 | OpenAI | api.siliconflow.cn/v1 |
自己选 |
| OpenCode Go | OpenAI | opencode.ai/zen/go/v1 |
deepseek-v4.1-flash |
| Anthropic | Anthropic | api.anthropic.com |
自己选 |
| Google Gemini | Gemini | SDK 自带 | 自己选 |
| 自定义 · OpenAI 兼容 | OpenAI | 自己填 | 自己选 |
| 自定义 · Anthropic 兼容 | Anthropic | 自己填 | 自己选 |
没有默认模型的来源，在设置页点「获取模型」拉一次列表自己挑（也能直接手打模型 id）。
OpenCode Go 的列表只留走 /chat/completions 的模型（DeepSeek、GLM、Kimi、MiMo 等）；
MiniMax、Qwen 走 /messages，Grok、GPT 走 /responses，选了会失败，所以不放进下拉框。
三种协议各走自家官方 SDK（openai / anthropic / google-genai），不自己拼 HTTP；
判断那条 OpenRouter 的路是唯一的例外——typesafe-sdk 把路径写死成 /v1/systemone，
打不到 OpenRouter 的 /api/alpha/decisions。
两节各一把 key，都必填。链路是三段式（issue #4）：先让 Jev 答 7 道判断题，把
「对方意图 / 对方需要 / 建议动作 / 紧张度」折成一小段中文小抄喂给起草，三条候选都顺着这个判断写；
最后再问 Jev 一次「哪条候选最合适」，概率就是卡片上的百分比。一次分析两次 Jev 调用——
以前是盲起草 + 判断和排序一次问完，起草读错意图时三条会一起跑偏，Jev 只能矮子里拔将军。
判断那次要是挂了（限流、超时），自动退回老路：盲起草 + 一次合问，行为跟以前一样；
排序那次挂了就按第一条推荐，判断照样显示。
温度 1.2，max_tokens 400；思考模式默认关，开了会带上各家自己的思考开关、max_tokens 提到 4000
（思考过程也算进去，400 会把答案截断）。思考开关只有 DeepSeek / OpenRouter / Anthropic / Gemini 认。模型只给出 1~2 条时会带着它的回答追问一次补齐，还不够就按实际
条数走（少于 2 条就不排序）。
微信 Windows 4.x（进程 Weixin.exe，窗口类 Qt51514QWindowIcon）界面自绘在一块 GPU 合成画布上
（MMUIRenderSubWindowHW）。UIA 树只有 2 个节点、没有控件树——probe/probe_win.py、
probe/probe_win2.py 实测证伪。
所以唯一干净的非侵入采集路 = 截自己的微信窗口 + 本地 OCR。离线、零 token。
？！～ 留着，那是语气）。下载 exe 的只看前三条；Python 只有源码运行 / 自己打包才需要。
Weixin.exe）JEV_API_KEY，默认来源 OpenRouter（或
TypeSafe 直连）；起草用 LLM_API_KEY，默认
DeepSeek 官网。详见下面「使用说明」Win10 上 WGC 会在微信窗口外画一圈黄框，系统不给关；Win11 才能关掉。 嫌碍眼就把标题栏的采集开关拨到「已暂停」，黄框立刻消失。
普通使用请直接用上面的下载即用。想改代码、调 prompt、自己打包才需要这一节。
git clone https://github.com/jev-chat/jev-chat-windows.git cd jev-chat-windows python -m venv .venv .venv\Scripts\activate pip install -r requirements.txt python main.py
PyCharm / VS Code 里直接 Run main.py 也行。
首次启动会自动弹出设置页：填两把 key（判断 JEV_API_KEY、起草 LLM_API_KEY，见上面「使用说明」），
选你们的关系（恋人 / 朋友 / 同事 / 家人 / 自定义）。key 写进注册表 HKCU\Environment，重启后依然有效，
不落任何文件；其余设置写进项目根的 config.json（已在 .gitignore 里）。
双击 build.bat（没有 .venv 会自己建一个，装依赖、调 PyInstaller，一路到底），或者手动：
pip install -r requirements.txt pyinstaller pyinstaller --noconfirm --clean jev.spec
出来的是 dist\jev-chat-windows\，整个文件夹就是成品（onedir：onefile 有 150MB 要每次启动解压）。
推一个 v* tag，.github/workflows/release.yml 会在 windows-latest 上打好、压成 zip 挂到 Release 上；
手动触发（workflow_dispatch）只出 artifact，方便试打包。
改完点「保存设置」，下一次生成立即生效，不用重启。
| 控件 | 作用 | 存在哪 |
|---|---|---|
| 你们的关系 | 恋人/朋友/同事/家人/自定义，起草和判断都按它把握称呼和分寸 | config.json → relationship（默认 romantic partners） |
| 说话风格（可选） | 一句话描述自己的口吻，只喂给起草；留空就只靠最近消息模仿 | config.json → style |
| 参考上下文 | 起草和判断各看最近多少条消息，3~30 | config.json → context（默认 10） |
| 群聊指定回复对象 | 开了群聊里才有「回复对象」那一行，候选针对 TA 写 | config.json → reply_target（默认关） |
| 启动时检查更新 | 开了才在启动时查一次 GitHub 最新版本号，有新版本就在标题栏下面提示 | config.json → check_update（默认开） |
| 调试视图 | 另开一个窗口实时显示截到的画面和识别框，看识别在哪一步认错。拨一下立刻生效，不用点保存；关掉那个窗口等于关掉开关 | config.json → debug_view（默认关） |
| 判断 · 来源 | OpenRouter 还是 TypeSafe 直连 | config.json → jev_provider（默认 openrouter） |
| 判断 · 密钥 | 上面选哪家就填哪家的 key。已配置时留空 = 保留 | 注册表 HKCU\Environment → JEV_API_KEY |
| 判断 · 模型 | 可手打，也可点「获取模型」拉列表挑 | config.json → jev_model（空 = 该来源默认） |
| 起草 · 来源 | 上面那张表里的任意一家 | config.json → draft_provider（默认 deepseek） |
| 起草 · Base URL | 只有两个「自定义」来源才出现这一行 | config.json → draft_base_url |
| 起草 · 密钥 | 上面选哪家就填哪家的 key。已配置时留空 = 保留 | 注册表 HKCU\Environment → LLM_API_KEY |
| 起草 · 模型 | 可手打，也可点「获取模型」拉列表挑 | config.json → draft_model（空 = 该来源默认） |
| 起草时开启思考模式 | 开了模型先想再写，慢好几倍、贵一些 | config.json → thinking（默认关） |
主界面上那几个（标题栏的采集开关、「当前会话」和「回复对象」下拉、「填入时带 @」勾选框）只在内存里， 不落盘，重启回默认。
几个注意：
main.py 入口：父进程只管界面，子进程采集，队列传消息（IDE 直接 Run）
app/ UI + 采集层
capture.py 找微信窗口 + WGC 盯帧 + 像素锚点定位消息区；帧全程内存
ocr.py RapidOCR 读消息区 → 按颜色分 me/her/灰字 → 滚动去重；另读头部的会话名
worker.py 采集子进程主循环（截图 → 定位 → OCR → 去重 → 丢队列）
fill.py 填入不发送：写剪贴板 → 点输入框 → Ctrl+V，到此为止
overlay.py 置顶悬浮窗：会话/回复对象、判断摘要、3 条候选、聊天记录、设置页（PySide6 + Fluent）
debugwin.py 调试视图：另一个窗口画当前帧 + 每个识别框的分类；只在内存里画，不存图
settings.py 两把 key 只进注册表，其余设置落 config.json
core/ Jev 判断内核，平台无关，跟安卓原版同一套口径
engine.py 唯一入口 analyze(messages, relationship) → 候选 + 排序 + 判断
providers.py 两张来源表（判断 / 起草）：协议、地址、默认模型；纯数据，不认 key
llm.py 三种协议的薄适配层，一律走官方 SDK：openai / anthropic / google-genai
jev_client.py Jev 判断客户端：OpenRouter（urllib）/ TypeSafe 直连（typesafe-sdk）；脱敏、退避
questions.py 7 道判断题 + build_state() + build_rank_question() + 判断小抄 guidance_text() / 中文标签 CHOICE_LABELS
draft.py 起草 3 条候选：拼提示词、解析、过滤、不足时追问补齐；调用走 llm.py
tools/
demo.py 端到端冒烟：拿一段写死的对话跑完整链（需 key + 联网）
preview_ui.py 用合成数据预览界面（含 --state debug 的调试视图），不采集不联网不碰微信；可 --screenshot 出图
make_icon.py 生成 docs/icon.ico（打包图标），图标已提交，换颜色才用重跑
probe/ 一次性探针，结论已写进本文，留着是为了可复现
probe_win.py UIA 能不能读微信聊天文字 → 证伪（树是空的）
probe_win2.py UIA 证伪 v2：分清「树是空的」和「有树没文字」，顺带试 LegacyIAccessible
probe_notify.py 微信来消息走不走 Windows 通知平台（能监听到就零 OCR）
probe_ocr.py OCR 读不读得准中文气泡、左右说话人分不分得开
probe_ocr_speed.py RapidOCR 一帧多久、裁小能快多少（结论：det_limit_type 必须 'max'）
probe_ocr_live.py WGC 持续盯窗口 + 变了就 OCR，新文字实时打控制台
probe_printwindow.py 试 PrintWindow + PW_RENDERFULLCONTENT 能不能绕开 Win10 黄框（未验证）
probe_laya.py Laya（开源本地决策模型）能不能替 Jev：英文题跑 multilingual / typed-decisions → 都接近随机
probe_laya_cn.py 同上，中文题问 multilingual → 更差
probe_laya_en.py 把对话人工译成英文再喂 typed-decisions → 好一点，但生气那段仍判成闲聊
jev.spec PyInstaller 打包定义（onedir），build.bat 和 CI 共用这一份
build.bat 本地一键打包（双击就行）
.github/workflows/release.yml 推 v* tag → windows-latest 上打包 → zip 挂到 Release
requirements.txt 依赖（纯 ASCII 注释：中文 Windows 上 pip 按 GBK 读会炸）
NOTICE 出处、第三方组件许可证与商用约束
docs/KICKOFF.md 最初的需求和硬约束说明
docs/icon.ico 程序图标，tools/make_icon.py 生成
docs/ui_*.png README 里那三张截图，tools/preview_ui.py --screenshot 出的
docs/wechat-mp.png 公众号「恸码奇点」长条横幅，README 标题下和设置页底部共用
config.json 你自己的设置，不进仓库（在 .gitignore 里）
tools/ 和 probe/ 里的脚本都按「项目根在 PYTHONPATH 里」写（PyCharm 默认会把内容根加进去）。
命令行跑 tools/demo.py 得自己带上：set PYTHONPATH=. && python tools/demo.py。
代码里没有 sys.path 补丁。
probe/probe_printwindow.py 是
PrintWindow + PW_RENDERFULLCONTENT 的替代方案探针，还没在微信 4.x 上验证过，能出图就能换掉 WGC。@名字 也只是纯文本，
微信不会把它变成真正的 @ 提醒——真 @ 得走微信自己的选人面板，本工具不模拟那套按键。fill 靠点击输入框坐标：算的是消息区底线下方 40px、左边界右侧 60px，微信改布局就得跟着调。v0.1.10
analyze() 改成三段式 —— Jev 先答 7 道判断题，判断折成中文小抄喂进
起草提示词，最后 Jev 只做排序；一次分析两次 Jev 调用。判断那次失败自动退回老路（盲起草 + 判断和
排序一次问完），排序失败就按第一条推荐v0.1.9
openai / anthropic / google-genai），可点「获取模型」拉
接口的真实列表；key 收敛成两把 JEV_API_KEY / LLM_API_KEY，换来源复用同一个槽，老的
OPENROUTER_API_KEY / DEEPSEEK_API_KEY 仍能读到，保存一次自动迁移settings.save 部分保存（某项传 None）会把 config.json 里那几项清空——写文件前没先把要保留
的值读出来NOTICE、LICENSE 加上游版权行，README 加「版权与许可」「免责声明」和封号问答，重写
「什么会出网」；发布 zip 带上 LICENSE/NOTICEv0.1.8
app/version.pyv0.1.7
restype/argtypes，修句柄被截断导致的崩溃（来自 PR #2）；剪贴板被占
重试、AttachThreadInput 抢前台、粘贴前 Ctrl+End 追加再填requirements.txt 钉 rapidocr 1.4.x（1.2.x 构造参数会 KeyError）v0.1.6
jev-chat-windowsbuild.bat / jev.spec）、CI release 产物名同步改名v0.1.5
v0.1.4
v0.1.3
max_tokens 400），设置里另给一个「起草时开启思考模式」
开关，开了提到 4000["…"]、逗号连着的多个数组、带编号的 JSON 行都能剥干净v0.1.2
v0.1.1
v0.1.0
HKCU\Environmentjev.spec + build.bat）+ 推 v* tag 自动出 ReleaseCopyright © 2026 rezoch340 与 jev-chat 贡献者。代码以 MIT 协议开源，另见 NOTICE。
基于 jev-chat-windows（https://github.com/jev-chat/jev-chat-windows）二次开发。第三方组件与商用：本项目自己的代码是 MIT，但 Windows 发布包（PyInstaller 打的 zip）里打进了 PySide6-Fluent-Widgets，该组件是 GPLv3 协议，非商用免费，商用需要 向作者购买商业授权。因此发布包整体受 GPLv3 约束：想商用的人请自己去买那份商业授权，或者自己把这个 组件换掉，本项目不代为处理。其余依赖（PySide6、RapidOCR、windows-capture、openai / anthropic / google-genai / typesafe-sdk 等）的许可证见 NOTICE。
免责声明：本项目只处理你自己设备上、你自己有权查看的聊天。请在自己设备上自用；装到别人机器上 读别人的聊天记录是另一回事，本项目不为那种用法背书。请遵守微信软件许可协议与当地法律法规，微信 改版可能导致本项目的界面识别失效。使用本项目造成的后果由使用者自行承担，作者不负责。
微信或 QQ 弹出一条消息 → 悬浮窗立刻告诉你这句话的真实意图、风险几级、该怎么回。
纯只读、零封号风险——不注入、不 hook、不解密数据库，只是「看屏幕 + 本地模型判断」。
先自查：常见问题解答（FAQ）——配置文件、日志、模型路径、安装报错、旧版空白面板速查。 交流群、公众号等联系方式见文末「交流反馈」；数据流向与隐私见 PRIVACY.md。
| 平台 | 状态 | 采集方式 | 需要的权限 | 备注 |
|---|---|---|---|---|
| 微信 macOS 4.x | ✅ | 窗口截图 + Vision OCR | 屏幕录制（读）+ 辅助功能（填入） | 布局常量按微信 4.1 校准；自动跟随你正在聊天的窗口（含拖出的独立聊天窗） |
| QQ macOS 6.9.x（QQNT） | ✅ | 系统无障碍树直接读结构化节点 | 辅助功能（读 + 填入） | 不截图、不 OCR；我方 / 对方按节点 class 判定 |
两者共用同一套判断、生成与悬浮窗；悬浮窗跟随当前在前台的那个应用。
JEV_BOXES=1 启动即开、菜单栏可切）：OCR 命中的消息实时框在微信窗口上，对方/我分色 + 置信度面板使用 macOS 原生浅色磨砂材质：顶部是当前聊天、分析状态、正在处理的消息与上下文；中间依次显示意图、识别率、风险等级和行动建议；底部按话术分组展示候选回复。当前风险圆点会轻微呼吸提示。每条候选左侧是本地排序概率，右侧仍只有「复制」和「填入」；候选行会随完整文字自动增高，不截断内容，发送始终由用户在微信里手动完成。
面板默认开三组话术：高情商话术、贴吧老哥 v1.0、阴阳怪气，每个下拉可换成其余语气或「不用」；「不用」的话术槽只保留一行下拉选择，不生成也不占候选行；开启或关闭话术只改变面板高度，不改变判断与轮询流程。黄色窗口按钮收起到聊天名与状态，红色窗口按钮退出。
聊天标题支持单字和较短的联系人名称；识别到聊天标题变化时，即使最后一条消息相同，也会重新分析。
联系人列表可拖动时，固定比例的自动区域可能漏消息或读到列表摘要。本分支提供可选手动模式：
消息区和输入区分别保存到现有用户 env 的 JEV_MESSAGE_REGION、JEV_INPUT_REGION，值是窗口宽高及选区 x/y/宽/高（窗口点坐标），保留其他配置和注释、文件权限 600。每次启动在同一界面重新确认保存的两个选区；窗口整体移动无需重画，窗口尺寸或窗口身份改变后暂停并要求重新校准。拖动内部列表、改变输入区高度、切换带公告的聊天后，请主动重新校准；当前不保证自动识别这些内部布局变化。选区上方独立读取聊天标题，标题为空时不分析。
校准模式按气泡表面分组普通文字：同一气泡内合并多行，独立气泡不因距离近而合并；不按字符数或窗口高度删除短句。左右归属基于选区内气泡位置和同侧对齐，无法判定的文字只显示，不进入模型上下文或自动回复目标。对紧凑、均匀且原 OCR 没有文字的灰色气泡，按气泡大小放大到至少 192 像素高，进行一次局部 Apple Vision Accurate 英文识别，仅补充置信度至少 0.5 的纯数字；已有文字不重复处理，不猜数字序列，也不把 I 等字母替换为数字。图片、引用、复杂主题和特殊昵称尚未充分支持，可能仍为「未确认」；这不是通用视觉理解模型。
两个区域尚未确认时不允许填入，可使用复制。 手动输入区仅在明确点击「填入」时使用：检查窗口与会话、读取整个编辑区的草稿（不跳过底部），点击获得焦点后再次检查草稿，再输入并 OCR 读回。不会清空草稿、发送消息或使用剪贴板。手动坐标不走自动输入区定位；用户改变内部布局后必须重新校准，不能保证自动发现分栏/输入区高度变化。现有自动模式维持原行为。手动模式每次重新 OCR，不复用空帧的旧消息，不新增哈希；因此性能与自动模式不同。
验证：离线回归覆盖短句、相邻气泡、多行文本、宽列表下的左右归属、深色合成气泡、选区边界、配置权限及空帧不复用。原生窗口已验证消息区及输入区预览/保存/取消与恢复入口；截图复测覆盖宽窄列表和私聊，单独数字 1/2/3/4 在原始与 1 倍采样下均保留。填入保护通过模拟事件测试验证，未向真实微信输入框写入测试文字。尚未完成真实微信消息到在线模型候选的全流程验收，也未验证所有微信版本及主题。个人截图不随代码提交。
仅支持 Apple Silicon（M 系列）Mac，macOS 13+。不支持 Intel Mac，也不要通过 Rosetta 运行（本地判断模型依赖的 torch 没有 Intel 版本，#19）；启动时会检测并以中文提示原因。
只想用：Releases 下载 .app，解压拖进「应用程序」，第一次右键 → 打开（没做公证，双击会被 Gatekeeper 拦）。
弹窗若显示「已损坏，无法打开，你应该将它移到废纸篓」（浏览器下载的 zip 常见，右键打开也绕不过），别删——在终端清掉隔离属性即可：
sudo xattr -r -d com.apple.quarantine /Applications/jev-jarvis.app
.app 若改过名（如「jev-jarvis 2.app」），把命令里的目录名换成实际路径。
首次启动按提示授予「屏幕录制」权限（系统设置 › 隐私与安全性 › 录屏与系统录音，给 jev-jarvis 打开），退出重开生效；「填入」另需「辅助功能」权限，第一次点会弹系统授权框。v0.3.1 及更早的旧版本还需把 python3.12 那条一并打开。只用 QQ 的话不需要屏幕录制权限，辅助功能一项即可。
缺少可用的 uv 时，两种启动入口都先完整下载并执行官方安装脚本（下载含超时和重试），失败后尝试已有的 Homebrew。失败提示区分网络、证书、磁盘和安装器错误，详细输出见 ~/Library/Logs/jev-jarvis.log。官方脚本安装到 ~/.local/bin，不修改 shell 配置。
从源码跑（微信在运行、终端已授予屏幕录制）：./start.command。分层自测：
uv run python src/perception.py # 感知层：识别到的消息 + 耗时 uv run python src/apps/qq.py # QQ 感知层：AX 读到的消息（CLI 里可验） uv run python src/judge.py "这个需求你今天跟一下" # 单条消息出判断 uv run python src/judge_zh_test.py # 22 条中文意图回归 uv run python src/generate.py --check # 生成层凭据解析 uv run python -B -m unittest discover -s tests # 发出消息/异步结果回归（合成 OCR，不读屏） uv run python probe/bootstrap_regression.py # 两种启动入口的离线回归；不联网、不实际安装
两层、两个 key、都可以不填：判断层不填时首次启动会引导选择——配置 key 在线判断，或下载离线模型（约 3.8 GB）；也可以稍后再说，面板会持续提示。生成层打包版内置共享 key，不配也能出候选，数据流向见 PRIVACY.md。全部配置在一个 env 文件（不提供第二种格式）：
点击悬浮窗右上角 齿轮图标（模型设置），或菜单栏 J → 模型设置…，可编辑 Jev、OpenAI 兼容、Anthropic 兼容三组密钥、服务地址与模型。 设置窗口显示在悬浮窗上方，不会被面板遮挡。各页统一使用底部「保存配置」：模型配置保存后必须退出并重新打开应用；「会话记录与背景」页的修改保存后立即生效。
$XDG_CONFIG_HOME/jev-jarvis/env（未设置时为 ~/.config/jev-jarvis/env），显示具体路径。只修改所编辑服务的字段，保留其他配置、注释和未识别行，文件权限设为 600。文件被其他程序修改时拒绝覆盖，需重新打开窗口。/models 接口动态获取，再下拉选择；不内置模型清单。Jev 按官方 models[].name 读取（当前列表为别名，未列出的版本号仍可手填）；OpenAI/Anthropic 按 data[].id 读取。接口不支持、失败或返回空列表时明确提示，仍可手填，不自动换模型或服务。空下拉显示「暂无」（仅作提示，不作为模型保存或调用），仍可手填；底部动态提示以蓝色显示进行状态、绿色显示成功、红色显示错误。列表可见不代表一定有生成权限，选定后再测试。--check 的配置解析成功代替连接成功。.env 或内置共享密钥复制进用户文件。各配置页顶部突出显示本次启动正在使用自己的密钥、内置共享密钥或本地判断，以及实际来源；生成页同时标明当前启用的服务，优先级保留在窗口下方。.env；生成层 OpenAI 组优先于 Anthropic 组，均未配置才使用内置共享密钥。清空当前文件的密钥不会禁用其他来源中的密钥。由终端或启动器导出的值也显示为「环境变量」。OPENAI_* 使用 OpenAI 格式，ANTHROPIC_* 使用 Anthropic 格式；自定义地址不需要包含服务名称。Ollama 可填 http://localhost:11434/v1、密钥 ollama，模型从本地服务获取或手填。Jev 地址带不带末尾 /v1 都行，与手动配置共用同一条拼接规则。$(security find-generic-password …) 等 shell 表达式提供密钥，窗口不执行表达式、不展示其内容，未输入新密钥时保留原行；仍由已有启动器执行。要在窗口测试该服务，需明确输入密钥；保存将用输入值替换原表达式。外部注入的密钥继续遵循环境变量优先级。JEV_BOXES、JEV_TONES、OPENAI_EXTRA_BODY 暂仍通过 env 配置，保存窗口不会改动它们。OpenAI 连接测试沿用当前启动的 OPENAI_EXTRA_BODY；完整话术管理等留待后续扩展。JEV_CANDIDATES_PER_TONE 配置。也可继续手动编辑：
mkdir -p ~/.config/jev-jarvis cat > ~/.config/jev-jarvis/env <<'ENV' # 判断层（可选）：TypeSafe Jev，不填用本地 decider-2b export TYPESAFE_API_KEY="" # 生成层：任意 OpenAI 兼容端点 export OPENAI_API_KEY="sk-你的key" export OPENAI_BASE_URL="https://api.deepseek.com" export OPENAI_MODEL="deepseek-chat" # 端点的思考模式要靠额外字段关时填（Qwen3 这类不关会慢几十倍） # export OPENAI_EXTRA_BODY='{"enable_thinking":false}' ENV chmod 600 ~/.config/jev-jarvis/env
deepseek-chat（最快）；智谱 glm-4-flash（换 ANTHROPIC_API_KEY/ANTHROPIC_BASE_URL/ANTHROPIC_MODEL，两组都填 OpenAI 组优先）；本地 Ollama qwen2.5:7b（完全不出网）TYPESAFE_BASE_URL 三种填法等价可用——只到主机（https://api.typesafe.ai）、带版本段（…/v1，自动补动作段，不会出现 /v1/v1/…）、或填完整动作路径（填到动作段为止，原样使用、不再拼接）。第三方 TypeSafe 兼容网关填网关地址 + 网关 key，模型名按网关填写（如 Vercel AI Gateway 填 https://ai-gateway.vercel.sh/v1/evaluate、模型 typesafe-ai/jev；OpenRouter 填 https://openrouter.ai/api/alpha/decisions、模型 typesafe/jev-1.13，key 用 OpenRouter 的 sk-or-…，响应同为 systemone 形状）JUDGE_BACKEND）：首次启动（未配判断层 key 且离线模型未下载）会弹一次选择，结果写进 env：cloud=在线判断（不下载、不加载本地模型）、local=离线判断（预热时下载，选过就不再问）、skip=稍后再说（不再弹，消息时面板提示）。不写此键时：模型已在本地就照常使用，未下载则不会自动下载，面板提示引导。模型设置的「判断 · Jev」页可删除离线模型（显示实际占用）或启用离线判断hf-mirror.com 下载并在日志注明。也可在 env 里 export HF_ENDPOINT="https://hf-mirror.com" 显式指定任意兼容端点——显式配置优先，不再探测max_tokens，候选 0 条，面板只报「候选生成失败」——DeepSeek 认准 deepseek-chatJEV_TONES（| 分隔、每条「名字=说明」，同名覆盖内置，重启生效），如 摸鱼大师=像资深摸鱼选手，把活推得漂亮又不失礼。有质量门槛：说明至少 10 字，写清「什么语气 + 别变成什么」；太短的（如「夸我」）不会加载，启动日志会写明原因——说明太空模型就没得发挥，候选只会平庸uv run python src/generate.py --check、uv run python src/judge_jev.py在设置的「会话记录与背景」页开启 记录聊天历史（默认关闭）。应用只累计实际读到的已发送消息，包含本人和其他成员；每个聊天固定保留最近 100 条，不自动翻页、不读取微信数据库。重复读屏按消息序列衔接，相同文字的不同消息可以分别保留；无法可靠衔接时会少记；后续连续读屏确认的新消息作为独立片段保存，模型不会跨未知断层拼接上下文。
「聊天背景」可写相关背景信息，例如「AAA 是群主，BBB 是公司老板」。「保存配置」统一保存这些背景、记录开关及条数。保存后，仍有效的当前目标会重新分析。背景不占消息条数，清空编辑框后保存即可删除。
JEV_HISTORY=0（关闭）或 1（开启），JEV_CONTEXT_MESSAGES=20。非法条数不接受；启动时无效值回退为 20。通过本页保存的值重启后仍可编辑；真正由启动环境变量传入的值优先，对应选项不可编辑，需修改环境变量后重启。历史和背景保存在 ~/Library/Application Support/jev-jarvis/conversations.json，源码运行与打包 .app 使用同一位置，不随启动时的工作目录变化。
历史和背景仅保存在本机，但推理时会发给当前选择的模型服务及配置的中转；服务地址可在对应模型设置页配置，修改后需重启生效，服务方的留存政策由其决定。应用不增加历史云同步或项目后台存档，详见 隐私说明。
| 内容 | 位置 | 大小 | 清理 |
|---|---|---|---|
判断层本地模型 decider-2b（首次启动引导选择后才下载，判断+排序共用） |
~/.cache/huggingface/hub/models--Mapika--decider-2b |
~3.8 GB | 模型设置 →「判断 · Jev」页「删除模型…」；或 rm -rf ~/.cache/huggingface/hub/models--Mapika--decider-2b；之后走本地判断会重新下载 |
| 会话历史与背景 | ~/Library/Application Support/jev-jarvis/conversations.json（权限 600） |
每会话最多 100 条，背景独立 | 设置 → 会话记录与背景 → 管理记录；背景清空后保存。删除应用不会自动删除此数据文件 |
| Python 运行环境（venv） | ~/Library/Application Support/jev-jarvis/venv |
~0.7 GB | 删除 .app 不会连带删它，需手动删 |
生成层配 Ollama 的话模型在 Ollama 自己的目录（~/.ollama），非本项目下载。
src/perception.py 顶部常量需重新校准）；多窗口优先识别主窗口「微信 / WeChat / Weixin」，且只认这三个应用名——名字里含「微信」的兄弟应用（微信读书、微信输入法、企业微信）不再参与窗口选择tail -40 ~/Library/Logs/jev-jarvis.logTYPESAFE_API_KEY 走云端判断——这是为了防止 #37 那种加载把系统推入内存高压、进程被系统直接终止的情况container--self / ExEditor-qq-msg-editor），src/apps/qq.py 顶部常量需同步；uv run python probe/qq_ax_probe.py 可直接看当前真实取值菜单栏「YOLO 检测框」同时显示消息框和输入目标，约每秒刷新：蓝色实线表示辅助功能接口定位到的输入控件，橙色虚线表示从截图边界推测的输入区；无法定位时显示原因。虚线不代表已取得可写控件，也不修复 #17 的聊天区域固定比例问题。
聊天识别会在同一张截图上检测输入区边界，并将输入区排除在消息、上下文和画面变化判断之外；视觉边界不可用时尝试辅助功能定位，仍无法确认则暂停分析。输入草稿不会作为待回复消息。左侧列表的固定比例限制仍未解决。
悬浮窗跟随 macOS「降低透明度」设置：关闭时保留原版磨砂；开启时使用浅灰绿底、白色卡片与灰绿边线，增强区域区分。切换设置后自动更新，无需重启。
顶部消息区将昵称与正文分开，正文默认显示两行；长消息可点击「展开」查看，超长内容在区域内滚动。「收起」恢复两行，展开操作不会重新请求 AI。
「填入」优先通过辅助功能接口写入并读回确认。部分微信版本不提供输入控件时，显式点击「填入」会尝试视觉兼容路径：复核窗口、输入区和标题，激活微信、点击输入区、输入文字，再用 OCR 核对。该路径需要屏幕录制及辅助功能权限，会移动鼠标；填入期间请勿操作键鼠或切换聊天。不会自动按发送键，也不使用剪贴板或 Cmd+V；换行和制表符转换为空格。
兼容路径读到已有草稿时停止，提示使用「复制」手动插入；辅助功能路径仍追加原有文字。窗口、焦点或会话变化时停止，画面无法确认时提示检查草稿，不自动重试。视觉边界及 OCR 都可能误判，标题检查也不是会话 ID，不能消除用户同时操作时的竞争；深色主题、多显示器与其他微信版本仍需更多验证。
uv run python -B -m unittest discover -s tests；macOS 原生窗口与按钮流程：uv run python -B probe/settings_smoke.py（临时配置 + 本地测试服务，不使用个人密钥）。./packaging/build_app.sh；发版 ./packaging/release.sh --publish（干净 worktree 构建 + 解压回验 + gh release）。版本号只有 pyproject.toml 一处；有开发者证书可加 --sign "Developer ID Application: ..."screencapture 子进程抓其窗口 → Vision OCR（只扫聊天区）；QQ 在前台时，读其无障碍树 → 同一条管线：本地 decider-2b 出意图/风险 → LLM 并发出候选 → 本地排序 → 悬浮窗 NSPanel。底层仍按窗口 ID 抓取而不是全屏截图，悬浮窗不污染 OCRCopyright © 2026 eatmoreduck 与 jev-chat 贡献者。代码以 MIT 协议开源，另见 NOTICE。
LICENSE 与 NOTICE，并在产品「关于」页、说明文档或发布页写明来源。推荐写法：基于 Jev 聊天助手（https://github.com/jev-chat/jev-chat-jarvis-mac）二次开发。用着有问题、想提需求、想一起改，扫码进群（1 群已满，从 2 群开始扫，满了顺序换下一个）；有新版本发布会在群里和公众号通知，建议关注；群二维码 7 天失效，过期了在 issue 说一声：
2 群 |
3 群 |
4 群 |
5 群 |
都满了或者不想进群，直接找我：加个人微信（备注来意），或关注公众号后台私信（新版本发布同样在公众号通知）：
个人微信 |
公众号 |
如果你觉得我写的这玩意儿对你有点帮助，欢迎请我喝杯咖啡。咖啡因一到位，脑子就开始冒泡，源源不断地驱动我往前跑；哪天我更新得特别勤，说明这杯续上了 😄
隐私与数据流向详见 PRIVACY.md：聊天内容只发给模型服务商——推荐自配 API key 或本地 Ollama；内置免费通道经作者中转，承诺与提醒见该页。
点击折叠
| 感谢 博查 赞助了本项目！博查是一个给 AI 用的搜索引擎，让你的 AI 应用连接世界知识，获得干净、准确、高质量的搜索结果。提供 Web Search API、Bocha Jev API 等多种联网搜索和模型服务。open.bocha.cn | |
| 感谢 小优店铺 赞助了本项目！小优店铺是一家数字商品与账号服务店铺，为本项目的用户提供选购渠道。点此前往。 | |
| 感谢 速创猫 Vytal 赞助了本项目！速创猫 Vytal 是专业的 AI 视频工作流平台，提供可批量复用的视频工作流，降低内容制作门槛，服务内容创作者、培训机构及中小团队。点此前往。 |
悬浮窗：危险等级、对方真实意图、排好序的 3 条候选回复 |
设置页：判断 / 回复 / 视觉三路接口分别可配 |
| 平台 | 状态 | 采集方式 | 备注 |
|---|---|---|---|
| 微信 Android | ⏸ 已停止支持 | — | 微信 8.0.52+ 对普通无障碍服务隐藏了消息文字，且近期对部分账号/设备的聊天界面开启防截屏（FLAG_SECURE），两条读取路径都不通，没有干净的读取方式；1.4 起不再读取微信 |
| QQ Android | ✅ 全链路 | 无障碍读节点 | 9.3.50 实测（群聊）；1v1 按同结构推断 |
| X / Twitter 私信 | ✅ 全链路 | 解析 Compose 节点的 content-desc | 12.25 实测，中文界面；英文界面未验 |
| 飞书 / Lark | ✅ OCR 兜底（真机验证） | 无障碍读气泡矩形 + ML Kit 离线 OCR 识别正文 | 正文自绘不在无障碍树里，1.3 起对每个气泡矩形做 OCR；我/对方按已读状态判 |
| 任意其它 App | ✅ 手动 | 悬浮窗菜单「截屏识别一次」整屏 OCR | 不自动、不分我/对方（全部当作对方所说并在面板标注） |
| 桌面端 / 网页 | ⏳ 规划 | 截图 + OCR / 视觉 | 同一内核，换采集方式 |
本项目只读你自己设备上、你自己有权查看的聊天，不针对任何单一平台。
1. 装包。 仓库里有签好名的 release 包：apk/jev-assistant-v1.4-release.apk（Android 11+）。各版本安装包也在 Releases。
adb install -r apk/jev-assistant-v1.4-release.apk
2. 填密钥。 打开 App → 设置 →「接口」分三张卡：判断接口 / 回复接口 / 视觉接口。最简单只填「判断接口」一栏的 OpenRouter API Key，其余两栏留空会自动继承这把密钥就能用。想换回复模型（默认 deepseek/deepseek-chat-v3.1，国内 Gemini / OpenAI 会被区域限制）就在「回复接口」选预设（OpenRouter / DeepSeek 官方 / 通义兼容）或自填地址，每张卡都有独立的一键连通测试。
3. 开权限。 按主页向导开三项：
装过 debug 包的要先卸载再装 release（签名不同），卸载会清掉密钥和设置。小米 / HyperOS 重装后悬浮窗权限会被重置，装完按向导再开一次。
ACTION_SET_TEXT，失败自动退到剪贴板粘贴，任何情况下都不发送。在设置 → 分析 →「知识库与联系人」。
https://jev.bocha.cn 与模型 bocha-jev-v1（协议与 TypeSafe 一致），页面上会显示官方地址并支持一键复制，当前限时免费。全新安装默认使用博查 Jev；已经配置过判断接口的老用户不受影响，provider 和密钥都不会被改动。https://ai-gateway.vercel.sh/typesafe，模型 typesafe-ai/jev，密钥用 Vercel AI Gateway 的 key。协议与 TypeSafe 直连相同（POST /v1/systemone）。不会。程序只把选中的回复填进输入框，发送键永远由你自己点。转账、红包、收款一律不碰。
需要 root 或 Xposed 吗？会不会封号？不需要 root，也不用装任何模块。它不修改聊天软件的安装包、不注入进程、不调用对方 App 的接口或账号体系，只读系统无障碍服务暴露出来的界面内容，和读屏软件的工作方式一样。
我的聊天记录会被上传吗？聊天内容只在你触发分析的那一刻，发给你自己在设置里配置的模型接口。项目没有任何自建服务器，不收集、不落盘、不进日志。历史记录默认关闭，开启后也只存在手机的 App 私有目录里。
悬浮球不见了，或者读不到消息怎么办？多半是国产 ROM 把后台进程冻结了。先确认无障碍、悬浮窗、自启动、省电无限制四项都开着，小米 / HyperOS 尤其要开后两项。重装后悬浮窗权限会被重置，按主页向导再开一次。在聊天界面里随便点一下通常能自愈。
飞书里读不到正文？其它 App 能用吗？飞书的消息正文是自绘控件，无障碍树里没有文字，1.3 起改为对每个气泡矩形做离线 OCR。没有专门适配的 App，可以在悬浮窗菜单里点「截屏识别一次」，整屏 OCR 后同样能分析，只是不区分我方和对方。
要花钱吗？软件本身免费开源。模型调用走你自己的 API Key，按用量在对应服务商那边结算，项目不经手任何费用。
升级之后没反应？把系统设置里的无障碍开关关掉再打开一次。1.3 起新增了截屏能力，服务需要重新绑定才会生效。
QQ / X / 飞书 ──(无障碍读节点)──▶ 采集最近消息
│
┌───────────────────┴───────────────────┐
▼ ▼
Jev 判断（一次 7 道题） 生成模型起草 3 条候选
意图 / 危险 / 需求 / 动作 / 该不该回 │
└───────────────────┬───────────────────┘
▼
Jev 给 3 条候选排序
▼
半透明悬浮窗展示 → 复制 / 填入（不发送）
background（关系 + 联系人备注 + 命中笔记）和 history（历史消息）。ACTION_SET_TEXT，失败则剪贴板 + ACTION_PASTE，不发送。capture/ChatAppAdapter.kt 实现 ChatAppAdapter：pkg 是包名，extract(root, res) 从无障碍树取出标题和消息列表（Msg(side, text)，side 为 me / other），不在聊天窗时返回 null。capture/ChatCaptureService.kt 的 adapters 加一行。先用 adb shell uiautomator dump 看目标 App 暴露了什么，已有四个适配器覆盖了四种情况：
| App | 树的情况 | 适配器怎么做 |
|---|---|---|
| 节点开放，有 id | 正文 id/mjn、标题 id/371，按气泡贴哪侧头像判谁说的 |
|
| 微信 | 已隐藏消息文字，部分设备还开启了防截屏 | 已停用：1.4 起不再接入采集分发，适配器代码保留在仓库中，以便日后微信策略变化时恢复 |
| X | Compose，无 id，text 为空 | 解析 content-desc 发件人：正文。时间。Read，发件人是「你」即我方 |
| 飞书 | 正文自绘，树里没有文字 | 树上拿 bubble_content_container 矩形与已读状态，OCR 每个矩形的正文 |
适配器返回 null 表示不在聊天窗，返回空消息列表表示在聊天窗但树里没正文——只有后者会触发 OCR 兜底。
QQ、X 全程只有一个 Activity，判「是不是聊天窗」要看树里有没有该有的节点（如输入框），不能看 Activity 名。
构建与目录结构JDK 17 + Android SDK（platform 35 / build-tools 35）。
./gradlew assembleDebug # app/build/outputs/apk/debug/app-debug.apk ./gradlew assembleRelease # 需要仓库外的签名 properties，路径由 JEV_KEYSTORE_PROPS 指定
app/ — Android 应用（Kotlin，传统 View）
capture/ 无障碍采集：ChatAppAdapter.kt 各 App 适配器、ChatCaptureService.kt 分发服务、前台保活、ocr/ 截屏与离线识别jev/ Jev 客户端与题目集 · overlay/ 悬浮窗 · core/ 配置与数据模型（含 core/kb/ 知识库存储与上下文构建）KnowledgeActivity 知识库管理页（笔记 / 联系人）tools/jev/ — Jev 题目集与校准脚手架（Python）docs/ — 设计与验收文档apk/ — 签好名的 release 包：、上午 / 下午、Read 是中文界面实测；英文界面只做了兜底，未验。tools/jev/）。FLAG_SECURE）截不到。如需联系，请公众号私信。 合作、赞助、反馈、进群失败、二维码过期，都走公众号私信，其它渠道不一定看得到。
想听真实需求：你在哪个聊天 App 上最想要这个副驾？希望它判断什么、怎么提示、什么绝对不能碰？扫码进群直接说。1 至 7 群已满，不要再扫；8、9 群任选一个，请勿重复加入。
8 群 |
9 群 |
以下七群已满，请勿再扫：
1 群 |
2 群 |
3 群 |
4 群 |
5 群 |
6 群 |
7 群 |
群二维码 7 天有效（本批到 2026-09-29），过期了公众号私信要新码。
同在 jev-chat 组织下：
隐私政策见 PRIVACY.md（说明读取了什么、发给谁、存在哪里、怎么删除）。
Copyright © 2026 Finderchangchang 与 jev-chat 贡献者。代码以 MIT 协议开源，另见 NOTICE。
基于 Jev 聊天助手（https://github.com/jev-chat/jev-chat-jarvis）二次开发。免责声明：本项目只处理你自己设备上、你自己有权查看的聊天。请遵守微信、QQ、X、飞书等各软件的许可协议与当地法律法规，作者不对使用后果负责。
如果你觉得我写的这玩意儿对你有点帮助，欢迎请我喝杯咖啡。咖啡因一到位，脑子就开始冒泡，源源不断地驱动我往前跑；哪天我更新得特别勤，说明这杯续上了 😄
随手支持，不用有压力；不支持也没关系，点个 Star 或提条建议同样能让我开心很久。
Jiamu Zhang1 Tianze Yang1 Yucheng Shi2 Liang Wu1
1 Nokia, Sunnyvale, CA 2 Tencent Hunyuan
Qwen3-8B on a real BANKING77 item. Every number is a model output.
Tip
🆕 L2 has landed. A closed-form head per question, solved on 100–300 labels in seconds, served from one prompt stopped at two thirds of the model's depth. It follows its question across rewordings without new labels. Jump to it ↓
Ask any open LLM a typed question and get back a decision with a probability you can threshold, read from one prefill of its next-token distribution. No generation, no parsing, no fine-tuning. Raw logits change their answer when you reorder the options, and their confidence cannot be trusted; AnyJev fixes the first with zero labels and the second with a few hundred.
| ⚪ raw logits one prompt |
🔵 AnyJev L0 zero labels |
🟢 AnyJev L1 + temperature |
|
|---|---|---|---|
| Labels required | none | none | 100–500 |
| Answer flips when options are reversed | 0.230 | 0.073 | 0.077 |
| Accuracy | 0.747 | 0.803 | 0.807 |
| Calibration error (ECE) | 0.240 | 0.184 | 0.095 |
| Auto-decidable at ≤5% error | 7.7% | 46.3% | 52.0% |
Qwen3-8B, BANKING77 20-way, 300 test items. Full table incl. every ablation: docs/results_bench.md
The last row is the point. Accuracy moves by 6 points, but the share of traffic you can safely automate goes from 7.7% to 52.0%, a 6.8× difference on this task (a point estimate at n=300; the interval is wide, see Limitations). With raw logits a "0.9" is not trustworthy enough to act on, so everything goes to a human. Once the probability means what it says, you can set a threshold.
📦 1. Install
pip install "anyjev[hf]"💬 2. Ask typed questions. L0 is on by default and needs no labels.
from anyjev import Decider, Question from anyjev.backends.hf import HFBackend d = Decider(HFBackend("Qwen/Qwen3-8B")) route = Question.choice("Which team should handle this?", ["billing", "technical", "sales", "other"], name="route") risky = Question.noul("Is this tool call destructive or irreversible?", name="risky") done = Question.score("How complete is the task?", bins=5, name="done") r = d.decide({"conversation": [...], "tool_call": {...}}, [route, risky, done]) r["route"].distribution # {"billing": 0.81, "technical": 0.07, ...} r["risky"].p_true # 0.12 r["done"].value # 0.35 r.level # "L0"
🎯 3. Add labels when you have them. A temperature is L1; a closed-form head is L2, the accurate one.
d.calibrate(risky, states, labels) # 100–500 labels → L1 (a temperature) d.fit_head(route, states, labels) # 100–300 labels → L2, one forward + a closed-form solve, seconds d.save_artifacts("qwen3-8b.json") # d.load_artifacts(...) next time; ~100 KB per head r = d.decide(state, [route], level="auto") # L2 where a head routes, else L1, else L0 r["route"].level # "L2"
🔁 4. Or let the loop feed it. d.observe(route, state, label) stores labels as they arrive and solves the head by itself at 30, re-solving at 60, 120, …
⚡ Serving. The transformers backend (anyjev.backends.hf) serves every level today; serving through vLLM / SGLang is on the roadmap, not in this release. For many states and one question, d.decide_batch(states, question).
🎬 Try it in one command. python -m demo.jev_mode --backend fake runs the whole thing on a synthetic model in under a second, no download. --lifecycle plays the deployment loop; drop --backend fake to run a real Qwen3 with the shipped heads (demo).
| Level | Needs | Does | Does not |
|---|---|---|---|
raw |
nothing | restricted softmax over label tokens (what the clones do) | anything about bias or calibration |
L0 |
nothing | averages position bias out over the K rotations and divides out the label prior | make the model's uncertainty calibrated |
L1 |
100–500 labels per question | temperature scaling on top of L0 | change the ranking |
L2 |
100–300 labels per question, a local model | a closed-form head (shrunk LDA / ridge) on the hidden state at ~⅔ depth, one prompt per state | transfer to another question or model |
Every Decision carries its level, so downstream code can refuse to act on the wrong one. L0 costs K prefills for a K-option choice (about 0.25 s per decision at batch 32 on one H100, K = 20); L2 costs less than one plain forward — one prompt, stopped early: 0.68× on Qwen3-8B.
L2 is not a training run. Labels buy a head in one closed-form solve (seconds on a CPU, no gradients, the model's weights untouched). After that only the head's feature mean and scale move, re-estimated from unlabelled traffic — so the head follows its question across rewordings and option orders by itself, and new labels are needed only for a new question.
Reworded, the Qwen3-8B head as is drops from 0.77 to 0.65–0.70; 30 unlabelled requests of the new wording bring it back to 0.74–0.75, against 0.77 for a fully relabelled refit (JSON).
One decision at serving time. A stored head answers from one truncated forward. Without one, the same call falls back to L1 or L0 exactly as before; the routing is in docs/method_v3.md.
Deployment lifecycle: day 0 at L0, labels from the loop, heads in seconds
flowchart LR
D0["day 0: define the questions,<br/>serve with level auto;<br/>every answer is L0, zero labels"] --> C["collect labels from the loop:<br/>review queue, outcomes, or the LLM<br/>being replaced; dec.observe fits at 30"]
C --> F["fit_head per question;<br/>export_artifacts to one JSON per model"]
F --> S["serve: L2 where a head routes,<br/>L1 or L0 elsewhere"]
S --> W{"what changed?"}
W -->|"wording or option order"| S
W -->|"new question or option set"| C
W -->|"new base model"| R["re-solve every head from<br/>the stored labelled states"]
R --> S
classDef shipped fill:#dcfce7,stroke:#0f9d76,color:#0f172a
classDef decision fill:#fef3c7,stroke:#d97706,color:#0f172a
class D0,C,F,S,R shipped
class W decision
A shift in the states (not the wording) is invisible to the recentring, so a periodic spot check on a labelled slice stays in the recipe. Full method: docs/method_v3.md.
|
🔁 Order flips cut every model × task row, at L0, zero labels 3 models × 3 tasks → |
🧩 Typed-decisions accuracy Qwen3-32B and 30B-A3B at L2, 300 labels per question; Jev 0.727 as published, fine-tuned Laya 0.768 5 models → |
⚡ Cost of one decision of a single plain forward, Qwen3-8B at L2: one prompt, stopped at block 24 of 36 latency → |
Jev mode, on LocalLLaMA/typed-decisions (20 questions, 300 labels each, 2,000 held-out decisions):
| model | L0, zero labels | L2 | block | cost vs one forward |
|---|---|---|---|---|
| Qwen3-1.7B | 0.494 | 0.730 | 18 / 28 | 0.70× |
| Qwen3-4B | 0.564 | 0.786 | 24 / 36 | 0.69× |
| Qwen3-8B | 0.647 | 0.771 | 24 / 36 | 0.68× |
| Qwen3-30B-A3B | 0.630 | 0.799 | 40 / 48 | not measured |
| Qwen3-32B | 0.700 | 0.798 | 52 / 64 | 0.84× |
Pooled ECE at L2 is 0.03–0.05. Jev 0.727 and fine-tuned Laya 0.768 on the same set, as published by their authors. Every cell: docs/results_exit.md
A 1.7B at 64% of its depth reaches the number Jev publishes; a 4B ties the fine-tuned 421M Laya. 100 labels already put the 8B head at 0.740 (20 labels: 0.654, 300: 0.772).
More: Jev mode in full · 2048 and Minesweeper · NanoJev maze · when L0 helps · small models · the research log, negative results included
Heads you can load today, and what a head costsanyjev-heads/<model>.json ships 23 heads per model (the 20 typed-decisions questions and three bench tasks) for Qwen3-1.7B / 4B / 8B / 30B-A3B / 32B, built and validated through the same fit_head → decide_batch path a user runs (scripts/build_heads.py). A head is a [hidden, K] matrix plus a bias, a standardisation vector and a temperature: ~100 KB, solved in 2–8 s on the 1.7B–8B.
The big model's heads also distil into a small one without gradients: the 32B's heads labelling 1,200 generated cases per workflow lift the 1.7B from 0.730 to 0.760 (the 4B and 8B do not move). docs/jev_mode.md
All models and tasks in one figureEvery number is regenerated from committed JSON (bash scripts/regen_docs.sh); a second run from a clean checkout reproduced every zero-label number bit for bit. Not affiliated with TypeSafe AI or Jev; rows published by their authors were not rerun here.
choice, noul and score from one prefill, nothing generatedrequire=level="auto", observepython -m demo.jev_mode)Dated plan and help-wanted files: ROADMAP.md.
Also: at most 26 options in the letter readout (a span readout is on the roadmap, not in the code); coverage at 5% risk is a high-variance estimate at n = 300; the headline tables are Qwen models; every decision here is scored in isolation, not inside an agent loop.
Backends and bench providers are one file each; several are help wanted (ROADMAP.md, CONTRIBUTING.md). Changes: CHANGELOG.md. Credits: CREDITS.md.
@software{anyjev2026, title = {AnyJev: Turn any LLM into a Jev-style decision model}, author = {Zhang, Jiamu and Yang, Tianze and Shi, Yucheng and Wu, Liang}, year = {2026}, url = {https://github.com/nokia-applied-research/AnyJev} }
Apache-2.0, see LICENSE. Datasets keep their own licenses, see THIRD_PARTY.md.
AI video prompt cheat sheet & Claude Skill for Veo 3, Google Flow, Kling, Sora, Runway, Hailuo, Luma, Midjourney — camera angles, camera movement, cinematic lighting, composition, color grading, mood, and a ready-to-use prompt formula.
Bộ từ điển prompt điện ảnh cho AI tạo video/ảnh — 700+ thuật ngữ về góc máy, chuyển động camera, ánh sáng, bố cục, màu sắc, cảm xúc, có giải thích tiếng Việt và công thức ghép prompt dùng được ngay.
A Claude Skill (works with Claude Code, Claude Cowork and Claude.ai) that teaches the AI professional cinematography vocabulary so your text-to-video and text-to-image prompts stop being vague ("a nice cinematic scene") and start being precise ("medium close-up, low angle, slow dolly in, rim lighting, teal and orange grading").
It is model-agnostic: the vocabulary works for Veo 3 / Google Flow, Kling, Sora, Runway Gen-4, Hailuo, Luma Dream Machine, Pika, Midjourney, Stable Diffusion, Flux, Nano Banana, and any future model that reads English prompts.
| File | Content |
|---|---|
SKILL.md |
The skill itself: workflow, prompt formula, condensed keyword tables, ready-made combos by video type, mood → combo lookup, pre-flight checklist |
references/01-camera-angles-and-movement.md |
70+ camera angles & shot sizes, 60+ camera movements (dolly, arc, crane, FPV drone, dolly zoom, bullet time…) with "when to use" |
references/02-lighting.md |
80+ lighting terms (golden hour, rim light, Rembrandt, chiaroscuro, volumetric, practical, neon…) |
references/03-composition.md |
80+ composition rules (rule of thirds, leading lines, S-curve, negative space, repoussoir…) |
references/04-lens-technical-film.md |
Lenses, aperture, shutter, bokeh, flare, filters, film stocks, aspect ratios, color grading tech |
references/05-style-color-mood.md |
100+ art styles, 80+ color palettes/grading, 100+ mood/emotion terms |
references/06-materials-weather-pose.md |
Materials & textures, weather & atmosphere, character poses |
examples/ |
Worked prompt examples for common video types |
[Shot size + Angle], [Subject + appearance], [Specific action], [Setting + weather],
[Lighting], [Camera movement], [Style + color], [Mood], [Technical]
Example:
Medium close-up, low angle, a young woman in a red áo dài walks slowly through a rainy
Saigon alley at night, neon signs reflecting on wet asphalt, rim lighting from city lights,
slow dolly in, cinematic, teal and orange grading, melancholic mood, shallow depth of field,
35mm film grain
Rules baked into the skill: one camera movement per clip, one main action per clip, lighting must match weather/time of day, style ↔ color ↔ mood must point the same way, max ~8 technical keywords, keep character description identical across clips of the same story.
Claude Code
git clone https://github.com/Rylaispirit/cinematic-video-prompt-skill.git ~/.claude/skills/cinematic-video-promptOr per-project: clone into .claude/skills/cinematic-video-prompt inside your repo. Then just ask: "write a Veo 3 prompt for a rainy night street scene" — the skill loads automatically.
Claude.ai / Cowork
Zip the folder (SKILL.md must be at the root of the zip) and upload it under Settings → Capabilities → Skills.
Any other AI tool (ChatGPT, Gemini, local LLM)
Paste SKILL.md as a system prompt / custom instruction. The reference files can be pasted on demand when you need deeper vocabulary.
> Write a Kling prompt: a monk meditating on a mountain at dawn, epic feeling
> Make this prompt more cinematic: "a cat sitting on a window"
> Give me 3 clips for a product video of a ceramic mug, consistent style
> Which lighting should I use for a horror scene in an old house?
> Explain what "dolly zoom" does and when to use it
The skill answers with an English prompt in a code block, plus a one-line explanation of the choices (in the user's language).
A raw list of 700 terms is hard to use. The skill adds the layer that matters: which term to pick for which feeling, how to order them, what not to combine, and ready combos for storytelling videos, product videos, food, talking-head training videos, night street scenes, travel, action and vertical Reels/TikTok.
PRs welcome — especially new camera-movement "golden prompts" that you have verified work well on a specific model (please name the model).
MIT — free to use, modify, and share.
Một Claude Skill (dùng được với Claude Code, Claude Cowork, Claude.ai) giúp AI hiểu đúng ngôn ngữ quay phim chuyên nghiệp. Thay vì prompt mơ hồ kiểu "một cảnh đẹp điện ảnh", bạn sẽ có prompt chính xác kiểu "medium close-up, low angle, slow dolly in, rim lighting, teal and orange grading".
Dùng được cho mọi model tạo video/ảnh đọc prompt tiếng Anh: Veo 3 / Google Flow, Kling, Sora, Runway, Hailuo, Luma, Pika, Midjourney, Stable Diffusion, Flux, Nano Banana…
SKILL.md — phần AI đọc: quy trình làm việc, công thức ghép prompt, bảng từ khóa rút gọn theo 11 nhóm, combo sẵn theo loại video (video kể truyện, kinh dị, cổ trang/tu tiên, sản phẩm, ẩm thực, đào tạo talking head, đường phố đêm, du lịch, hành động, Reels dọc), bảng cảm xúc → combo, checklist trước khi đưa prompt.references/ — bộ tham chiếu đầy đủ 700+ thuật ngữ, mỗi thuật ngữ có giải thích tiếng Việt dễ hiểu và gợi ý khi nào dùng: góc máy & chuyển động camera, ánh sáng, bố cục, ống kính & chất phim, phong cách & màu & cảm xúc, chất liệu & thời tiết & tư thế.examples/ — ví dụ prompt hoàn chỉnh cho các loại video hay gặp.Claude Code — chạy lệnh:
git clone https://github.com/Rylaispirit/cinematic-video-prompt-skill.git ~/.claude/skills/cinematic-video-prompt(hoặc clone vào .claude/skills/cinematic-video-prompt trong thư mục dự án). Sau đó chỉ cần nói: "viết prompt Veo 3 cảnh phố đêm mưa" — skill tự bật.
Claude.ai / Cowork — nén thư mục thành file zip (file SKILL.md phải nằm ngay gốc zip), vào Settings → Capabilities → Skills và tải lên.
ChatGPT / Gemini / tool khác — dán nội dung SKILL.md vào system prompt hoặc custom instruction. Khi cần tra sâu thì dán thêm file trong references/.
> Viết prompt Kling: nhà sư thiền trên núi lúc bình minh, cảm giác hùng vĩ
> Làm prompt này điện ảnh hơn: "a cat sitting on a window"
> Cho tôi 3 clip video sản phẩm ly gốm, giữ nhất quán style
> Cảnh kinh dị trong nhà cổ nên dùng ánh sáng gì?
> Dolly zoom là gì, dùng khi nào?
Skill trả về prompt tiếng Anh trong code block (copy được ngay) kèm một dòng giải thích tiếng Việt vì sao chọn góc máy / ánh sáng / chuyển động đó.
Danh sách 700 thuật ngữ rất khó tra khi đang làm việc. Skill bổ sung phần quan trọng nhất: chọn từ nào cho cảm giác nào, xếp thứ tự ra sao, không được ghép gì với gì (ví dụ "golden hour" + "heavy downpour"), và combo có sẵn cho từng loại video để bạn chỉ cần thay chủ thể.
Hoan nghênh pull request — đặc biệt là các "golden prompt" chuyển động camera bạn đã thử và thấy chạy tốt trên một model cụ thể (ghi rõ model).
MIT — dùng, sửa, chia sẻ tự do.
Keywords: AI video prompt, cinematic prompt, Veo 3 prompt, Kling prompt, Sora prompt, Runway prompt, text to video prompt guide, camera movement prompts, lighting prompts, prompt engineering for video, Claude skill, prompt tạo video AI, prompt Veo 3, prompt Kling, từ điển prompt điện ảnh, hướng dẫn prompt video AI.
Specialised AI models for logo design.
A logo is not a picture. It is a constructed object with rules: a mark that holds at sixteen pixels and on the side of a building, letterforms spaced by eye rather than by metric, clear space derived from the mark's own geometry, and lockups that still read when one of them is all you have room for.
General image models do not work this way. They produce something logo-shaped — a plausible arrangement of marks with no construction behind it, no reasoning about the business, and nothing you can hand to a printer or a sign maker.
Inkloom is building models that construct a mark the way a studio does, as a sequence of decisions that can each be explained:
| Stage | What it produces |
|---|---|
| Brand analysis | Turns a description of a business — sector, audience, tone, competitors — into concrete constraints: stroke weight, width, geometry, counter shape, which symbol families fit |
| Typography | Selects and fits letterforms against those constraints, then does the work that makes a wordmark: optical spacing, kerning at display size, a custom ligature where the name needs one |
| Symbol construction | Composes geometric primitives under construction rules — shared radii, tangent junctions, consistent terminals — so the result is built rather than sampled |
| Composition | Optical alignment rather than mathematical centring, clear-space ratios taken from the mark itself, and the lockup variants a brand actually needs |
The output is meant to be a specification, not a bitmap: a mark you can describe, defend and reproduce.
Early access is open at inkloom.art. Create an account, redeem a code, and credits are reserved against your account.
Generation is not live yet. We would rather say that plainly than imply otherwise: every page in the product says so, credits are described as reserved rather than spendable, and the feature flags that would switch generation on default to off and are not togglable from the console — because enabling a flag whose feature does not exist exposes a broken surface rather than a feature.
What runs today is the platform the models will ship on. Accounts and authentication, the credit ledger, the access-code system, the operations console, and the machinery around them: backups that are restore-tested rather than merely taken, an alerting pair where each half watches what the other cannot see, and a deployment path that refuses to migrate a database whose identity has not been confirmed.
Generation itself, then the things that only make sense once it exists: export in the formats a designer and a printer each need, brand kits, revision history on a mark, and paid plans. None of it is claimed as present until it is.
This source is published so the engineering can be read and audited — in particular the security and data-handling claims we make. It is not a distribution: see LICENCE.
Found a security issue? SECURITY.md says where to send it and what to expect. Please do not open a public issue.
Operational documentation — deployment, incident response, environment and runbooks — is kept internal. Source comments occasionally point at it by filename; that is a reference for the people who run the service, not a broken link. It describes how the service is operated, which is of no use to a reader and of some use to an attacker.
Built on Cloudflare Workers, Postgres and React Router, with the application and its API served from one origin — which is what makes the session cookie first-party and removes cross-origin handling entirely.
The test suite runs against a real database, a real browser and a real mail server rather than mocks of any of them, because the guarantees that matter here are transaction guarantees and a mock cannot have one. The test that matters most fires twenty-five simultaneous redemptions of a single code at a real database and asserts that exactly one redemption, one ledger entry and one balance exist afterwards.
| General | support@inkloom.art |
| Security | security@inkloom.art — please read SECURITY.md first |
Copyright © Inkloom. All rights reserved.
A catalog of API tools an agent can call — 660+ generative-media models available through muapi out of the box, plus a growing set of third-party tools (SEO, enrichment, social, scraping, and more) that anyone can add with a single PR.
This is a reference catalog, not a live proxy. Every entry is documentation —
what a tool does, what it costs, how to call it — not something this repo calls
for you. models/ entries run through your own muapi key; providers/ entries
run through the contributor's own account with that provider.
Agents: read llms.txt — one fetch teaches you how to browse and
use this whole catalog, no install or auth required.
The tools worth calling from an agent are scattered across dozens of vendors, each with its own docs, auth quirks, and pricing page — and most of the useful ones sit behind a subscription nobody buys for a single call (Semrush $139/mo, Moz $99/mo, Crunchbase $99/mo), or behind docs vague enough that you don't know what a call actually costs or returns until you've already signed up. This catalog puts the facts that matter — auth shape, real pricing, a captured example response — in one consistent shape, so an agent (or a person) can scan it and know exactly what a tool needs before ever opening its docs.
models/*.yaml |
providers/*.yaml |
|
|---|---|---|
| What it is | One of muapi's own hosted generative-media models | A third-party API a contributor already uses |
| Called with | Your muapi API key | The contributor's/your own key for that provider |
| Who adds it | Auto-synced from muapi's live catalog | Anyone, via PR |
| Editable by PR? | No — see "muapi-hosted models" below | Yes — this is the open contribution path |
ls providers/ models/ # browse what's catalogued cat capabilities.yaml # browse by category instead — media.*, seo.*, people.*, ... cat providers/<provider>.yaml # base_url, auth, endpoints, pricing for a third-party tool cat models/<model>.yaml # what a muapi-hosted model does, its cost, its docs page
providers/_TEMPLATE.yaml to providers/<your-provider>.yaml.CONTRIBUTING.md for the
full checklist, including the one non-negotiable step: get a real key and
confirm at least one endpoint actually works before opening the PR. A schema
that was never called against the real API is not accepted.python3 scripts/catalog_validate.py providers/<your-provider>.yaml
status to verified.See CONTRIBUTING.md for the full guide, including selection
heuristics (what gets accepted vs. rejected) and common gotchas per auth style.
These entries are generated directly from muapi's own live catalog — not hand-written,
and not open to arbitrary edits, since they describe what muapi itself already runs.
Each one deliberately omits how muapi actually serves the model (no base_url, no
auth details, no vendor name) — only the model itself, its cost, and a link to its
docs page. Missing a model, or see one that's wrong? Open an issue rather than a PR;
the catalog is refreshed from the source of truth periodically.
draft (providers only) — submitted, not yet independently verified by a maintainer.verified (providers only) — a maintainer confirmed the entry against a real key
and a real call; examples/<id>.json holds a real captured response.live (models only) — currently available through muapi.Treat draft entries as a starting point, not a guarantee — verify before relying
on one yourself.
MIT — see LICENSE.
为 Agent Harness 提供结构化决策能力。
将 TypeSafe AI 的 Jev 决策能力接入你的 Agent：帮助选择工具、Skill 和任务负责人，评估产出质量，并以结构化结果返回判断与概率。Jev 负责辅助决策，原 Agent 继续负责规划和执行。
提供 DeepSeek Harness 原生插件和 iPolloWork 导入插件两个安装入口；在 iPolloWork 中可供 OpenCode、DeepSeek Harness、Codex Harness 使用。同一份决策核心，不修改引擎源码，不接管主 Agent。
快速开始：安装到 DeepSeek Harness · 安装到 iPolloWork
| 使用环境 | 接入方式 | 安装入口 |
|---|---|---|
| DeepSeek Harness（DSH） | 原生插件，独立运行，不依赖 iPolloWork | DSH 插件命令 |
| iPolloWork | 独立导入、授权、启停与卸载 | 扩展 → 导入插件包 |
| OpenCode | 通过 iPolloWork 的扩展接口使用 Jev | 在 iPolloWork 安装一次 |
| Codex Harness | 通过 iPolloWork 的扩展接口使用 Jev | 在 iPolloWork 安装一次 |
OpenCode 和 Codex Harness 当前支持的是 iPolloWork 内的接入方式，本仓库尚未提供它们各自的独立原生安装包。其他 Harness 可以基于共享决策核心开发适配器，不代表已经自动兼容。
Jev 是 TypeSafe AI 的结构化决策模型，不是聊天模型。原 Agent 继续负责规划、解释、参数构造、工具执行和原生子 Agent 协作。Jev 不返回文字推理过程，Agent 的解释不能冒充 Jev 的原始输出。
适合直接使用 DSH 的用户。无需安装或运行 iPolloWork。
已安装 DSH 后，克隆本仓库，通过本地绝对路径安装：
git clone https://github.com/Devin-AXIS/jev-dsh-decision.git dsh plugin add /absolute/path/jev-dsh-decision
把 /absolute/path/jev-dsh-decision 替换成克隆目录的实际绝对路径。插件使用原生 dsh.bundle 格式，无需构建源码。
如果已经拿到 .tgz 安装包，也可以直接安装：
dsh plugin add /absolute/path/dsh-jev-decision-model-0.1.2.tgz
在 DSH 的凭据管理或启动环境中配置自己的 TYPESAFE_API_KEY。不要把密钥写进插件源码或聊天记录。插件复用 DSH 的 credentials、tools 和 skills 服务；官方默认配置提供这些服务，自定义配置需要保留它们。
重启运行中的 DSH 会话，让新配置生效，然后在对话中说：
使用 Jev，先检查配置，再测试连接。
插件提供 jev_status（本地状态）、jev_check_connection（真实连接测试）、jev_evaluate（结构化评估）三个工具，以及 jev-decision Skill。连接测试会产生少量 TypeSafe API 用量；本地显示“已配置”不代表密钥已经验证。
dsh plugin remove dsh-jev-decision-model
DSH 凭据由宿主统一持有，插件卸载不会擅自删除可能被其他插件共用的 TYPESAFE_API_KEY；不再需要时请在 DSH 凭据管理或环境中移除。运行中的 DSH 会话应重启以加载新的插件配置。
适合使用 Work 图形界面的用户。在 Work 中安装一次，供其支持的 OpenCode、DeepSeek Harness、Codex Harness 引擎使用，无需分别安装三个包。
获取已签名的 jev-decision-model-0.1.2.ipollowork-plugin 安装包，在 扩展 → 导入插件包 中选择它，确认发布者为 iPolloWork。维护者的安装包构建方式见下方“开发和测试”。
已安装旧版时，导入新版可沿用原插件身份升级。显示名称为“Jev DSH 决策引擎”，内部插件 ID jev-decision-model 和 DSH 包名 dsh-jev-decision-model 保持不变。
**客户端要求：**需要包含 ipollowork/ipollowork-2026 官方签名公钥注册的 Work 版本。未包含此注册的旧客户端会拒绝导入；仅满足清单里的最低版本号不够。请勿绕过签名校验。
打开插件详情中的授权入口，填写自己的 TypeSafe API Key。密钥由 Work 的插件授权仓库保存，不需要发给对话中的 Agent。
在对话中启用“Jev DSH 决策引擎”，然后说：
使用 Jev，先检查配置，再测试连接。连接成功后，帮我选择适合当前任务的 Skill。
Work 提供 status、check-connection、evaluate 三个动作，由现有扩展接口调用。这里的“测试连接”会发出一个真实且有少量用量的请求。
在 扩展 → Jev DSH 决策引擎 → 卸载 中移除。Skill、插件内容和专属授权由 Work 随包清理，没有另装全局脚本或后台守护进程；卸载不会删除 TypeSafe 云端账户。
推荐结果不等于执行授权。需要外部操作时，仍遵循当前 Harness 的权限和确认流程。
开发者参考：调用格式、签名与测试evaluate 与 jev_evaluate 使用相同参数：
{
"state": { "task": "为已收集的竞品资料制作汇报", "available": ["research", "slides"] },
"questions": {
"owner": {
"type": "choice",
"instructions": "哪个已提供的 Agent 最适合负责最终交付？",
"criteria": { "research": "负责资料分析", "slides": "负责演示文稿" }
},
"need_review": { "type": "noul", "instructions": "这个任务是否需要审稿？" },
"clarity": { "type": "score", "instructions": "任务要求是否清楚？", "criteria": ["不明确", "部分明确", "明确"] }
}
}问题类型是 choice（单选）、score（等级评分）、noul（是的概率）。多个互补 Skill 可使用多个独立 noul 问题。传入最小必要上下文，不传密钥或无关私密资料。网络固定发往 https://api.typesafe.ai/v1/systemone；默认模型 jev-latest。
插件限制每次 64 个问题、256 KiB 请求、1 MiB 响应、20 秒总网络时限；仅对 429/529 最多重试两次。没有无限重试、后台自动学习或全局请求拦截。错误会脱敏，不返回上游错误正文。
Node >=20.11，pnpm；运行时零第三方依赖。JSZip 仅用于开发时构建 Work 包。
pnpm install --frozen-lockfile
pnpm test
DSH_RUNTIME_ROOT=/path/to/dsh-runtime pnpm test:dsh
pnpm package:dsh
DSH_RUNTIME_ROOT=/path/to/dsh-runtime DSH_CLI_TEST=1 pnpm test:dshWork 包使用宿主要求的 Ed25519 签名格式。私钥放在仓库之外，不随源码、安装包或发布附件分发：
IPOLLOWORK_PLUGIN_SIGNING_KEY=/secure/path/ipollowork-ed25519.pem \ IPOLLOWORK_PLUGIN_KEY_ID=ipollowork-2026 pnpm package:work IPOLLOWORK_ROOT=/path/to/updated/iPolloWork pnpm test:work pnpm package:dsh
签名标识为 ipollowork/ipollowork-2026，公钥（SPKI DER，Base64）：
MCowBQYDK2VwAyEAoHA9xxXNZPQ7y+AKpDBIXruNvwVasq7axjilzIY6RDE=
Work 集成测试使用真实宿主 API，在独立临时目录测试签名、图标、导入、授权轮换、服务调用、卸载及篡改拒绝。TypeSafe 响应使用明确的测试数据，不代表真实 API 密钥已验证。DSH 测试使用真实工具与 Skill 注册表，检查加载和卸载。
本项目由 iPolloWork 维护和发布。Jev 属于 TypeSafe AI；本项目不表示获得 TypeSafe AI、DeepSeek 或其他 Harness 项目的官方背书。
Find code by what it does. JevGrep helps coding agents find relevant code when they do not know the file name or symbol to search for.
Ask a question such as “Where is session expiry handled?” and JevGrep scans the authorized repository, asks Jev to score all eligible fragments, then returns the original source excerpts with their paths and line numbers. The calling agent can read those files in detail and continue its work with less exploratory context.
Use the CLI or connect a coding agent through the local MCP server.
JevGrep is useful when a coding agent needs to:
It complements exact tools such as rg. If you already know the symbol or literal,
ordinary text search is usually faster.
JevGrep searches every valid UTF-8 text file, regardless of repository language or extension.
Install the public package from npm:
npm install -g @nassim-arifette/jevgrep jevgrep --version
The unscoped package name jevgrep belongs to a different project. Use the complete
scoped name above when installing. The installed command is still jevgrep.
Package: @nassim-arifette/jevgrep
To install a development checkout instead:
git clone https://github.com/nassim-arifette/jevgrep.git
cd jevgrep
npm ci
npm run build
npm link
jevgrep --versionnpm link makes the jevgrep command available from any directory on the computer.
Configure the provider and key once for the computer:
jevgrep init --global
TypeSafe AI is proposed first. To use Vercel AI Gateway instead:
jevgrep init --global --provider vercel
To use OpenRouter:
jevgrep init --global --provider openrouter
The command stores the credential in the user's JevGrep configuration directory, not
in a repository. TYPESAFE_API_KEY, AI_GATEWAY_API_KEY and OPENROUTER_API_KEY
environment variables take priority over the corresponding stored value.
Run init once from the repository root:
cd path/to/my-project
jevgrep initThe default root is the current directory. You can also provide it explicitly:
jevgrep init --root path/to/my-project
Provider credentials are global, but repository authorization is not. Each repository
must be authorized separately. Its trusted profile is stored outside the repository.
Interactive init asks before enabling remote evaluation for this repository:
Allow sending eligible source excerpts from this repository to Vercel AI Gateway? [y/N]
Answer y to search immediately. Enter or n keeps remote evaluation disabled.
Non-interactive initialization also leaves new profiles disabled. Optional scan caps are disabled by default;
configure them if you want to limit usage.
init also creates a commented .jevgrepignore in the repository when one does not
already exist. Existing exclusions are preserved; .gitignore is already respected.
jevgrep doctor jevgrep inspect
doctor checks the selected provider, credential state, authorized root, limits and
cache without making a network request.
inspect shows which files and fragments are eligible, what was excluded and how much
work a search would perform. It also stays offline.
If you did not enable remote evaluation during init, review the scope and limits,
then edit the profile path printed by init and set
remote_evaluation_enabled to true to allow source disclosure to the selected provider.
jevgrep search --query "Where is session expiry handled?"Useful options:
# Search only selected directories jevgrep search --query "How are permissions checked?" --scope src --scope tests # Return the canonical JSON response jevgrep search --query "Where is the cache invalidated?" --json # Read a multiline question from a file jevgrep search --query-file question.txt # Allow a deterministic partial scan when an enabled scan cap is exceeded jevgrep search --query "How does synchronization work?" --allow-partial
JevGrep automatically finds the authorized project for the current directory, including
when the command runs from a subdirectory. --config <path> remains available as an
explicit override.
| Provider | Setup | Model |
|---|---|---|
| TypeSafe AI | jevgrep init --global --provider typesafe |
jev-1.13.0 (pinned) |
| Vercel AI Gateway | jevgrep init --global --provider vercel |
typesafe-ai/jev |
| OpenRouter | jevgrep init --global --provider openrouter |
typesafe/jev-1.13 |
The TypeSafe transport follows the documented System One HTTP contract and is covered with simulated responses. It has not been tested against a real account in this project. Vercel AI Gateway has been checked on a small authentication example, including a repeat search served entirely from the score cache.
OpenRouter uses its alpha Decisions endpoint, POST https://openrouter.ai/api/alpha/decisions,
with Bearer authentication and structured Noul questions. The adapter supplies both
true and false criteria, reads answers[id].noul, usage.input_tokens,
usage.output_tokens and the response id, and disables provider fallback.
Its request and response handling were reviewed against the
official OpenRouter OpenAPI specification
(DecisionsRequest, DecisionsNoulQuestion, DecisionsResponse) on 2026-09-20.
No live OpenRouter request or automated test was run for this integration.
The alpha API may change. See the Jev model page
and OpenRouter configuration example.
To switch an existing global and project profile to Vercel:
jevgrep init --global --provider vercel jevgrep init --provider vercel
Use --provider openrouter in both commands to switch to OpenRouter.
JevGrep exposes the same search engine through a stdio MCP server:
jevgrep mcp
The server exposes one tool, semantic_search_code. Starting it does not scan files or
contact a provider. A tool call performs a search using the authorization associated
with the current directory.
Configure and authorize the repository first. One server process serves one repository.
Use the absolute profile path printed by jevgrep init so the server does not depend
on the client's working directory. Replace the example paths below.
After installing JevGrep:
claude mcp add --transport stdio jevgrep -- jevgrep mcp --config "/absolute/path/to/config.json"Check claude mcp get jevgrep and /mcp in Claude Code. See the
Claude Code MCP documentation.
Add an entry to your Codex config.toml:
[mcp_servers.jevgrep] command = "jevgrep" args = ["mcp", "--config", "/absolute/path/to/config.json"] tool_timeout_sec = 360
The suggested client timeout leaves a margin over JevGrep's default 300-second search deadline. Adjust both for your workload. See the Codex MCP documentation.
For clients that accept mcpServers configuration:
{
"mcpServers": {
"jevgrep": {
"command": "jevgrep",
"args": ["mcp", "--config", "/absolute/path/to/config.json"]
}
}
}Credentials saved by init --global are available to clients running as the same OS
user. Environment keys must be available to the client process. Do not commit keys
in MCP configuration.
If the client cannot find jevgrep or launch an npm shim on Windows, use absolute
paths to node and the installed dist/cli.js. See the
installation guide.
These examples have not yet been qualified with real Codex and Claude Code sessions.
Confirm that your client lists semantic_search_code and completes a search.
Search evaluation is remote. When you run jevgrep search, eligible source fragments
are sent to the configured provider together with:
JevGrep excludes common credential files, .env files, dependencies, build output,
generated files, minified files and files that match credential patterns. Links and
junctions are not followed. Run jevgrep inspect to review the eligible scope before
the first live search.
Credential filters cannot detect every secret; add repository-specific exclusions in
.jevgrepignore where needed.
The credential is never placed in the search payload, result or cache. Redirects are not followed by either transport. Provider retention and privacy policies still apply to anything sent remotely.
Human-readable output is the default. Pass --json for the validated response contract.
The result includes coverage information, exclusions, stop reasons and exact excerpts,
so an empty or partial result is not presented as proof that code does not exist.
| Code | Meaning |
|---|---|
0 |
complete result |
2 |
invalid request, configuration problem or rejected preflight |
3 |
partial result |
4 |
fatal runtime failure |
130 |
interrupted |
Results go to stdout. Diagnostics and measurements go to stderr.
JevGrep caches provider scores outside the repository, independently for each question and fragment. Changing another fragment does not invalidate an unchanged score. Provider, endpoint, model, query, source, location, criterion and layout remain part of the identity. Only misses are grouped into requests.
New TypeSafe direct profiles pin jev-1.13.0 and use the configured cache TTL (seven
days by default). Vercel's typesafe-ai/jev and OpenRouter's typesafe/jev-1.13
use the conservative rolling policy: scores can be reused for up to 15 minutes.
OpenRouter may resolve the requested model to a dated revision in its response;
the version alias is not treated as an immutable cache identity. Existing direct profiles
using jev-latest or jev-preview use the same short-lived policy.
Rolling reuse can briefly serve a score from an earlier model revision. doctor
shows this policy and its effective TTL. Set cache.rolling_ttl_seconds to 0 to
disable it, or to an integer from 1 to 900 to shorten it. cache.enabled: false
disables all score reuse. Existing profiles do not need to be recreated.
Clear the cache for the current project with:
jevgrep cache clear
Cached entries contain scores and identities, not source text, questions or credentials.
Fragments remain small enough to return precise excerpts. Requests pack fragments by the estimated tokens in the complete serialized payload, including the query, criteria and metadata.
| Transport | Aggregate ceiling used | Target with tokenizer headroom |
|---|---|---|
| TypeSafe direct | 64,000 tokens | 44,800 reference tokens |
| Vercel AI Gateway | 32,000 tokens (conservative local policy) | 22,400 reference tokens |
| OpenRouter | 32,000 tokens (conservative local policy) | 22,400 reference tokens |
TypeSafe documents 64k total and 32k for shared state plus one question. Gateway and OpenRouter advertise a 32k context; using it as an aggregate ceiling is conservative, not a claim that they document the same total-question limit. All three paths keep 30% headroom because the provider tokenizer is not public, and locally limit each request to 64 questions and 256 KiB. These last two limits are application safeguards. See TypeSafe model limits and the Gateway model catalog and OpenRouter Jev model page.
inspect and search planning use the same serializer and token estimator; inspect
uses a sample query, so its estimate can differ from an actual search. Estimates are
not provider billing. File preparation still runs on every search: there is no
persistent repository index.
npm ci
npm run typecheck
npm test
npm run build
npm run smokeRun the complete local verification gate with:
npm run verify
The test suite is offline and does not use provider credentials. Run npm run bench
for the local performance baseline, or npm run bench:retrieval -- --validate-only
to check the annotated retrieval pilot without network access. Real retrieval runs
use an explicit provider configuration. See benchmark commands and interpretation.
CI verifies benchmark correctness without enforcing machine-dependent timing limits.
MIT © 2026 Nassim Arifette.
A list of things built with Jev, TypeSafe's model for typed decisions, with the numbers behind them: who posted each demo, how many followers they have, how many likes it got, and what the limits of the model are. This list is open source (CC0), free to copy and reuse, and sponsored by AY Automate. It is unofficial and is not affiliated with TypeSafe.
Every entry links to the original post or repository. Ideas that nobody has shipped are in their own section and marked as ideas.
The 30 most-liked demos, ranked. Click a card to open the original post. Each card shows the demo's rank, area, author, likes, reposts, and reach (likes divided by the author's followers). Preview frames are low-resolution stills from the builders' own videos and belong to them. If an author wants one removed, open an issue. Full metrics for every demo are in docs.
Short answers to the questions people ask most, each with a source.
What is Jev? Jev is a model from TypeSafe AI that answers typed questions instead of writing text. Each question is a Choice, a Score or a Noul (yes or no with a probability), and the answer comes back as a number or a pick with a confidence. It does not generate text. See What Jev is.
Is Jev the same as the "Jev" that searches show for Jevons paradox or Deltarune? No. The word has other meanings. Search for "TypeSafe Jev" or "Jev AI model".
How do I call the Jev API?
Send a POST to https://api.typesafe.ai/v1/systemone with a bearer key. A full curl example is in docs/api-quickstart.md.
How much does Jev cost? TypeSafe lists $42 per billion input tokens, and output tokens are free. Vercel says Jev is free on AI Gateway until Sept 25. See Reported cost and latency.
What can I build with it? Routers, classifiers, judges, guardrails, triage and game agents. The Top 30 demos and Browse by area show real examples.
What are the limits of Jev? It reads literally, is weak at math, counting and dates, and accuracy drops with irrelevant state. See Limits of Jev 1.13.
Is Jev better than an LLM? It is a different tool. Use Jev for fast typed decisions and an LLM for writing. Many demos pair them.
Which open-source Jev projects exist? More than 150 repositories. See Open source and Long tail.
Is this list official? No. It is unofficial, open source under CC0, and sponsored by AY Automate.
Short on time? Read in this order.
Every tracked demo, grouped by what it does and ranked by likes inside each group.
| Area | Demos | Top demo | Likes |
|---|---|---|---|
| Content and growth | 18 | Real-time slop detector as you scroll | 7,180 |
| Apps and tools | 17 | Instant compaction for Claude | 10,435 |
| Agents and computer use | 14 | Flight search with Browser Use | 8,723 |
| Triage and routing | 9 | 500 emails for 3.5 cents | 3,853 |
| Games and real time | 7 | Jev plays Doom | 4,890 |
| Research and data | 7 | jevlike | 2,018 |
| Trading and markets | 2 | jev-trader | 4,913 |
Continued at the source.
A curated awesome list of public projects and practices built on Jev, TypeSafe AI's System One model for typed decisions.
This README is the homepage aggregate of the current category files, so the latest accepted entries are visible here without drilling into subpages.
A curated list of public projects and developer patterns built on Jev, TypeSafe AI's System One model for typed decisions.
What is Jev? Jev is not a chat model. It does not write text or hold conversations.
Instead, it takes unstructured state alongside a typed question and returns a typed decision—such as a choice, a score, or a boolean—accompanied by a confidence rating.By eliminating token-by token decoding, Jev acts as a fast, low-latency decision layer directly inside software.
Developers use it to handle classification, infrastructure routing, rubric scoring, verification gates, and autonomous agent guardrails.Goal of this ListMost discussions about Jev are scattered across launch threads, social media, and one-off prototypes.
This repository centralizes those pieces to answer two practical questions for developers:
Most Jev discussion is scattered across launch threads, model-gateway listings, and one-off prototypes. This list answers two practical questions quickly:
We do not include:
Inclusion means one thing: the entry satisfies the inclusion rules above. It is not a quality review, a security audit, or a recommendation. We do not verify that a project compiles, that its tests pass, that its published numbers reproduce, or that its license permits your use.
This matters most for projects that arrive in bulk. When one author releases several repositories on the same day, they commonly share a single scaffold — the same AGENTS.md, CLAUDE.md, STATE.md, and CHANGELOG.md — land in one or two commits each, and may ship considerably more prose than code. Such projects can be entirely legitimate; they are simply unproven. Treat them as leads, not as validated tools.
Before adopting an entry, check it yourself:
| Check | Why it matters |
|---|---|
| Does the code actually call the Jev API? | An entry can read well on a README alone. Look for a real request carrying typed questions, and a parsed answer coming back. |
| Is there a runnable check? | A test, an example with expected output, or a public demo. No check means no evidence that it works. |
| Do the numbers have a source? | Any accuracy, latency, cost, or volume figure should be traceable to the linked page. We strip claims we cannot verify, but the project page itself may still carry them. |
| How much of the repository is code? | Some projects are mostly prompt documents. That can be legitimate — just know which one you are getting. |
| Is there a license? | A few entries have none, which limits reuse and redistribution. |
Found something wrong? Open an issue or a pull request — removal is as valid a contribution as addition. Rules for AI-assisted work, project depth, and submission rate live in CONTRIBUTING.md.
Each entry lives in exactly one category. When a project could fit multiple categories, we choose the one closest to its direct application domain.
Source file: categories/classification-routing.md
NOTRA_JEV_CLASSIFIERS flag routes brand-visibility classifiers off an LLM and onto Jev Boolean decisions at a 0.5 threshold, targeting 300 ms p50.Choice (invoice or general) and routes each inbound email to the matching handler./v1/systemone endpoint to new-api so typed decisions sit behind the same gateway as chat models.Choice and Noul questions, sending low-confidence answers to human review.Source file: categories/verification-guardrails.md
Noul checks about source and build files, escalates suspicious chunks for a second pass, and returns implicated files and lines before execution.typesafe_permission_reviewer builtin so the agent's permission decisions run through Jev rather than an LLM call.Boolean questions per paragraph (stacked hedges, restating closers, not-X-but-Y turns, naked cost figures) at a 0.7 threshold; CLI, pre-commit hook, GitHub Action and Claude Code skill; measured 182 ms median and 1 of 54 clean paragraphs flagged against 37 for Haiku 4.5.jev-pref.json rules that Jev checks against each diff hunk, staged file set, or pull request, returning fix_now or advisory findings to the coding agent and a nonzero exit code on blocking ones.Source file: categories/scoring-ranking.md
Score axes inside a single systemOne request and turns them into a 0-100 Slop Score in ordinary TypeScript.Noul properties per source file so the agent knows what to fix first.Source file: categories/agent-decisions.md
Continued at the source.
Headless MCP server that generates teacher verification documents — employment letters, teacher ID cards, teaching licenses, payslips, and more — across 13 countries.
Portable, self-contained, and installable anywhere.
Documents are rendered as high-resolution PNGs. Examples generated by this tool:
| Teacher ID (US) | Employment letter (US) |
|---|---|
| Teacher ID (UK) | Employment letter (UK) |
|---|---|
| Code | Country | Document types |
|---|---|---|
uk |
United Kingdom | employment_letter, teacher_id, teaching_license |
us |
United States | employment_letter, teacher_id, teaching_license |
france |
France | installation_statement, iprof_screenshot, bylaws_extract, teaching_certificate |
netherlands |
Netherlands | employment_contract, teacher_registration, duo_declaration, school_id |
indonesia |
Indonesia | payslip, teaching_experience_letter, nuptk_card, appointment_letter |
australia |
Australia | signed_school_letter, school_id, teaching_license |
canada |
Canada | oct_card, teaching_license, signed_school_letter |
spain |
Spain | teaching_id, signed_school_letter, employment_contract |
argentina |
Argentina | payslip, employment_certificate, signed_school_letter |
slovakia |
Slovakia | payslip, employment_letter, signed_school_letter |
mexico |
Mexico | teaching_id, signed_school_letter, employment_certificate |
philippines |
Philippines | teaching_id, employment_certificate, teaching_license |
thailand |
Thailand | payslip, letter_of_employment |
Pillow, mcppip install dist/yowes_doc_generator-0.1.0-py3-none-any.whl
pip install -e .uvx --from . yowes-mcpThe server speaks MCP over stdio — the transport used by most agent runtimes (Hermes, Claude Desktop, and any MCP client). Connect it, discover the tools, then call them.
# from the built wheel pip install dist/yowes_doc_generator-0.1.0-py3-none-any.whl # or editable from source pip install -e .
Verify the install and that bundled assets resolve:
python -c "from countries.utils import load_font, get_profile_photo; \ print(load_font(30).getname()); print(get_profile_photo((280,340), person_id='x', gender='Male') is not None)" # ('DejaVu Sans', 'Book') <-- bundled font, not system # True <-- bundled photo found
# After install: yowes-mcp # Or from source: python mcp_server.py
It blocks and waits for MCP requests over stdin/stdout — don't run it as a foreground terminal app expecting prompts.
Point your MCP client at the yowes-mcp command:
{
"mcpServers": {
"yowes": {
"command": "yowes-mcp",
"args": []
}
}
}If yowes-mcp isn't on your PATH, use the absolute path to your interpreter and module instead:
{
"mcpServers": {
"yowes": {
"command": "/path/to/python",
"args": ["-m", "mcp_server"]
}
}
}| Tool | Description |
|---|---|
list_countries_tool |
List available countries, display names, and their document types. |
list_schools(country) |
List all schools for a country code. |
generate_documents(...) |
Render one or more documents to PNG and return their paths. |
No arguments. Returns one result item per country — { code, name, document_types }. (Because a list return is split into one MCP content item per entry, iterate content to see them all.)
country (required) — country code from list_countries_tool (e.g. "us").{ name, address, town, postcode, state, phone, lea }. Iterate content to see them all.| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
country |
string | ✅ | — | Country code (e.g. "us", "uk"). |
first_name |
string | ✅ | — | Teacher's first name. |
last_name |
string | ✅ | — | Teacher's last name. |
school_name |
string | ✅ | — | Exact or partial school name (matched against that country's school list). |
position |
string | ✅ | — | Teaching position/title. |
date_of_birth |
string | ✅ | — | DOB string, printed on the teacher ID (e.g. "12/05/1988"). |
gender |
string | — | "Random" |
"Random", "Male", or "Female" — selects the profile-photo pool. |
document_types |
string[] | — | all types | Which documents to render, e.g. ["employment_letter", "teacher_id"]. |
output_dir |
string | — | output/ |
Where to save PNGs (relative to the server's working dir). |
Returns { country, school, document_types, files, count, output_dir } — files are absolute PNG paths.
Minimal working client (requires pip install mcp):
import asyncio from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import stdio_client async def main(): params = StdioServerParameters(command="yowes-mcp", args=[]) async with stdio_client(params) as (read, write): async with ClientSession(read, write) as session: await session.initialize() countries = await session.call_tool("list_countries_tool", {}) # A list return is split into one content item per entry: for item in countries.content: print(item.text) res = await session.call_tool("generate_documents", { "country": "us", "first_name": "John", "last_name": "Smith", "school_name": "Valley High", "position": "Head of Science Department", "date_of_birth": "12/05/1988", "gender": "Male", }) print(res.content[0].text) asyncio.run(main())
list_countries_tool to see what's available.list_schools("us") to pick a real school.generate_documents(...) with the chosen country, school, and person details.Generated PNGs are written to output/ (or the output_dir you pass).
A tkinter (CustomTkinter) GUI is still available for manual use. The core generation logic is shared.
python main_gui.py # on Windows, use run.bat (sets TCL_LIBRARY)The MCP server is the primary, headless interface. The GUI is optional and not required for the skill.
yowes/
├── countries/ # Document generation core (package)
│ ├── base.py # CountryGenerator ABC (contract)
│ ├── utils.py # Fonts, profile photos, shared helpers
│ ├── foto/ # Bundled profile photos (package data)
│ ├── fonts/ # Bundled DejaVu fonts (package data)
│ └── <country>/ # One package per country
├── mcp_server.py # MCP server exposing tools
├── main_gui.py # Legacy tkinter GUI
├── docs/examples/ # Sample rendered documents
├── pyproject.toml # Packaging, deps, entry point
├── output/ # Generated documents (git-ignored)
└── run.bat # Windows GUI launcher
countries/<code>/__init__.py with a class inheriting countries.base.CountryGenerator.get_country_name, get_country_code, get_schools_data, get_first_names, get_last_names, get_positions, get_document_types, generate_document.countries/__init__.py via register_country("<code>", <Name>Generator).main_gui.py (get_country_list / on_country_change).The new country is automatically picked up by the MCP list_countries_tool and list_schools.
MIT © 2026 hirotomasato
A curated list of Jev use cases, projects, SDKs, tools, and learning resources.
Jev is the first System One model from TypeSafe AI — an AI model that returns typed decisions (Choice, Score, Noul) with calibrated probabilities instead of generated text.
Looking for real-world Jev use cases with numbers? madewithjev.com is a directory of what people are building with Jev — every build with the cost, latency, and source the author reported. Submit yours →
Jev launched in early access on September 15, 2026. This list is unofficial and not affiliated with TypeSafe AI. Pull requests are welcome — the ecosystem is days old and growing fast.
Large language models generate text. Jev does not. It evaluates typed questions against a state and returns values your code can branch on, sort by, and route with — plus calibrated probabilities and confidence. TypeSafe AI calls this model class a System One model: fast, structured decisions that software can use directly, trained with RLCD (Reinforcement Learning for Calibrated Decisions).
text or JSON state + typed questions → constrained answers + probabilities → your code
Jev exposes three question types. Questions in one request run in parallel against the same state.
| Question | Goal | Returns |
|---|---|---|
| Choice | Pick one option from a list | choice, probabilities, confidence |
| Score | Rate the state on a rubric | score, probabilities, confidence |
| Noul | Is this statement true? | noul (0–1) |
Use it to classify, route, score, detect, rank, extract, verify, and gate automation — anywhere you would otherwise write a brittle regex or pay an LLM to return JSON you then have to parse. Questions describe judgments; your code owns composition, thresholds, and side effects.
From TypeSafe's launch post:
| Existing LLMs | System One + Jev | |
|---|---|---|
| Optimized with | RLHF / RLVR | RLCD (Reinforcement Learning for Calibrated Decisions) |
| Optimizes for | Human preference; verifiable rewards | Calibrated decisions with honest probabilities |
| Output | Strings that need parsing and validation | Type-safe structured values, defined in advance |
| Sampling | Sequential, token by token | Parallel, all outputs in a single query |
| Cost | $0.20–$10 / MTok input, output ~5x more | $0.042 / MTok input, output free |
| Speed (vendor-reported) | 3–329 s end-to-end for frontier models | 70–500 ms end-to-end |
| Confidence | Tends to be overconfident if asked | Calibrated confidence on every answer |
| Best at | Chat, writing, code, open-ended reasoning | Decisions inside software: classify, route, score, verify |
Jev is not a replacement for an LLM. When you need free-form text, pair them: let Jev route, retrieve, verify, or guard the call, then let the LLM write inside the boundaries your code enforces.
Snapshot reviewed September 18, 2026. Check Models for current values — limits can change dynamically.
| Item | Current detail |
|---|---|
| Model alias | jev-latest (current version: jev-1.13.0) |
| Endpoint | POST https://api.typesafe.ai/v1/systemone |
| Price | $0.042 / 1M input tokens; output tokens free |
| Listed limits | 250,000 tokens/second, 1,200 requests/minute |
| Choice cardinality | Up to 255 options per Choice question |
| Modalities | Text only — no images, audio, or video |
| Direct access | Early access via waitlist at typesafe.ai |
| No-waitlist access | Vercel AI Gateway (typesafe-ai/jev) and Cloudflare Workers AI (typesafe/jev) |
Get an API key from the TypeSafe console, then:
Python — pip install typesafe-sdk
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient state = {"ticket": "I was charged twice and need the duplicate refunded today."} with TypeSafeClient() as client: # reads TYPESAFE_API_KEY from the environment response = client.system_one( state=state, questions={ "intent": Choice( instructions="What is the customer's main request?", criteria={ "refund": "The customer wants money returned.", "technical_help": "The customer needs a bug or integration fixed.", "information": "The customer is asking for information only.", "other": "None of the other options clearly fits.", }, ), "is_urgent": Noul(instructions="Does the ticket explicitly communicate time pressure?"), "frustration": Score( instructions="How frustrated does the customer appear?", criteria=["Calm and neutral", "Concerned but civil", "Very angry or using strong language"], ), }, ) print(response.answers["intent"].choice) # "refund" print(response.answers["is_urgent"].noul) # 0.0–1.0 print(response.answers["frustration"].score) # probability-weighted rubric position
JavaScript / TypeScript — npm install @typesafe-ai/sdk
import { choice, noul, score, TypeSafeClient } from "@typesafe-ai/sdk"; const client = new TypeSafeClient(); const result = await client.systemOne({ state: { ticket: "I was charged twice and need the duplicate refunded today." }, questions: { intent: choice("What is the customer's main request?", { refund: "The customer wants money returned.", technical_help: "The customer needs a bug or integration fixed.", information: "The customer is asking for information only.", other: "None of the other options clearly fits.", }), isUrgent: noul("Does the ticket explicitly communicate time pressure?"), }, });
On Vercel AI Gateway, use experimental_evaluate from the AI SDK with the model id typesafe-ai/jev. See the official quick start for details.
TYPESAFE_API_KEY).POST https://api.typesafe.ai/v1/systemone.typesafe-ai/jev for the AI SDK's experimental_evaluate, no TypeSafe waitlist required.typesafe/jev via env.AI.run, with worked support-routing and risk-escalation examples.Production-shaped uses with the cost and latency their authors reported. Each links to a full breakdown on madewithjev.com, the Jev use-case directory that maintains this list.
| Build | What it does | Reported numbers | Source |
|---|---|---|---|
| Jev plays Doom | Game loop asking Jev what to do ~10 times a second | ~10 queries/s, ~$7/hour | X |
| jev-ultrafast | Browser Use's agent with the next-action decision moved to Jev | ~2.9k stars | GitHub |
| Flight search with Browser Use | Booking flow driven end to end | ~7 s, ~$0.004 | X |
| Stagehand on a remote browser | Browser tasks at a tenth of a cent each | ~$0.001/task | X |
| jev-trader | Buy/sell decided inside a 300 ms Monad block, on Kuru's order book | 300 ms decision window | GitHub |
| Triage across 1,500 emails | A full inbox sorted in one pass | ~1,500 emails | X |
| Every's editorial vibe check | 37 documents, 21 questions each; 6 of 7 planted defects caught | 1,709 judgments, <$0.01, 0.35 s median | Every |
| 1kpapers | A corpus classified by topic and published as a site | 1,018 papers | Site |
| Jev plays chess | Legal moves as a Choice, compared with reasoning models | illegal moves impossible by construction | dev.to |
| 3,282 posts, eight questions each | Ian Nuttall's X back catalogue scored for what travels | 4.25M tokens, $0.1282, 8 m 34 s | X |
| Post scoring with SuperX | 61 questions about a draft before it ships | ~1 s, $0.0004/draft | X |
| 724 competitor ads, broken down | Hook, format, offer, CTA per ad across 37 brands | ~40 s, ~$0.09 | X |
| typesafe-computer-use | macOS computer use, one typed decision per step | ~$0.0002/step | GitHub |
| jev-drone | Tactical judgment loop flying on hardware | control at 2.5 Hz | GitHub |
| Wikiracing | Pick one link out of thousands until you arrive | 255-option Choice ceiling | TypeSafe |
All figures are as reported by each author, not measured by this list. → Browse the full directory at madewithjev.com
Official first, then community clients. Community packages are not affiliated with TypeSafe.
Official
pip install typesafe-sdk. Docs.npm install @typesafe-ai/sdk. Docs.TypeSafeClient replacement backed by LLM APIs, to compare Jev against chat models on the same questions. pip install system-one-adapter.@ai-sdk/typesafe-ai with experimental_evaluate; use typeSafeAi.evaluationModel('jev-latest') or the Gateway id typesafe-ai/jev.Community, by language
go get github.com/Gaurav-Gosain/jev-go. Also Stumble/jev-go - dependency-free, works against TypeSafe direct and Vercel AI Gateway, with an interactive CLI and an installable agent skill.system_one and model listing. Also Jev (OTP) - Jev as a peer GenServer; answers arrive as messages you pattern-match, with network-free tests.pip install jevclient), separate from the official SDK.Open-source projects that put Jev in a real loop. Grouped by what Jev decides.
Continued at the source.
A curated, source-backed list of projects built with Jev, TypeSafe AI's System One model for fast, typed, probabilistic decisions.
Jev takes program state plus typed questions and returns constrained answers with probabilities. It is designed for software decisions such as classification, routing, scoring, ranking, verification, and guardrails, rather than free-form text generation.
This list favors public source code, concrete Jev usage, clear limitations, and reproducible evidence. The latest review added 20 source-reviewed integrations, projects, and studies, bringing the community catalog to 155, alongside official resources, provider integrations, and related lists. See the September 20 research notes for pinned source evidence and review boundaries. Review completed September 20, 2026 (Europe/Istanbul); upstream event dates below are UTC.
System One shape: text or structured state + typed questions → constrained answers + probabilities → deterministic application code.
Question primitives: Choice selects an option, Score evaluates ordered rubric levels, and Noul returns a number from 0 to 1 representing the probability of "yes". Review or abstention behavior is defined in application code. See the primitive reference.
Input boundary: the hosted Jev model is text-only. Browser, audio, image, and robotics projects supply extracted text or structured observations, or use separate perception models. Independent multimodal reproductions are listed separately.
Good fits: semantic routing, triage, reranking, rubric scoring, moderation, verification, and low-latency decisions inside bounded workflows.
Important caveat: schema-valid output is not the same as a correct decision. Validate on your own data, calibrate thresholds, keep high-impact actions behind deterministic checks, and provide a human fallback.
September 18: Python SDK 0.7.0. Release notes document a breaking serialization change from msgspec to Pydantic, a new response_model argument, and corrected serialization of str subclasses.
September 18: OpenRouter listing. Jev 1.13 is listed with a September 18 date. This is a provider listing date, not evidence of a separate new upstream model revision.
September 16: Vercel AI Gateway integration. The integration introduces typed evaluation via AI SDK's experimental evaluate API.
Current model: TypeSafe documents jev-1.13.0, with both jev-latest and jev-preview currently pointing to it. Pin the version when comparing evaluations.
September 20: framework adoption. Source-level Jev integrations are now present in LangChain, Pydantic AI, LiteLLM, Rig, Composio, Effect, BAML, Ax, and TanStack AI. Availability and release status vary, so inspect the linked repository before depending on a package.
typesafe/jev integration accepting state and typed questions.@typesafe-ai/sdk, with credentials and billing handled by Netlify.typesafe/jev-1.13, alongside the moving typesafe/jev-latest alias.typesafe-ai/jev through AI SDK's experimental evaluate interface; its Boolean primitive corresponds to TypeSafe's Noul.Upstream framework integrations with inspectable Jev implementations. Presence on a default branch does not guarantee a stable package release.
@effect/ai-typesafe decision model mapping Effect's classify, probability, and rating operations to Jev Choice, Noul, and Score questions.TypeSafeClassifier Runnable with batched typed questions plus model-routing and risky-tool middleware.rig-typesafeai crate with typed Choice, Score, and Noul queries, response validation, examples, and fixtures.@tanstack/ai-typesafe adapter exposing typed Boolean, Choice, and Score decisions through TanStack AI's decide() API.Community-maintained clients and tools; official TypeSafe SDKs are listed above.
Uncertain branch, and procedure routing, with recording, replay, caching, and call budgets as DecisionModel middleware.if Hunch.likely?("fraudulent", given: order) branches on a typed Jev answer, with graded predicates from possibly? to definitely?.tail -f.decide — to an application across generate/stream/decide.Source-reviewed experiments and integrations. A model judgment does not establish safety or replace the host application's permission checks.
jev-pref.json and feeds findings back to coding agents.Choice over the installed skill roster plus Noul gates, while code applies the thresholds and names at most one skill; it starts in a shadow mode that only logs the decision.Choice decisions select tools and targets, uncertain decisions escalate to an LLM, and a shared guarded kernel supports isolated Docker workspaces and paired LLM-only comparisons.jevonian/auto, after deterministic code has already filtered candidates by wire protocol, context window, thinking-level floor, and spent quota windows; minConfidence marks a low-confidence route in the ledger instead of silently accepting it, and Jev is skipped entirely for pinned models, explicit jevonian/<route> requests, and routing.mode: "off".These tools select what reaches a model. Preserving retained text verbatim does not prove that omitted history was unnecessary.
Continued at the source.
An agent skill that finds where a photo was taken — and shows its work.
Works with
…and any other agent that reads SKILL.md and runs shell commands.
No text. No plates. No landmarks. One bridge, one mountain. Located to within 2 m.
npx skills add Oldcircle/geo-sleuth
Pick your agents when prompted. Then hand your agent a photo and say:
find where this photo was taken
That is the whole interface. The agent reads SKILL.md, runs the scripts, and comes back with the camera position, the direction it was facing and a satellite evidence image. Prefer to copy the folder yourself? See Installation.
SKILL.md plus plain Python scripts — so the same folder runs in Claude Code, Codex, Cursor, Gemini CLI, OpenCode and GitHub Copilot.A phone photo with the EXIF stripped: a white oven at the edge of a harvested rice paddy, a long viaduct in the distance, a steep mountain on the right. Not a single character in the frame. One message to an agent with this skill installed, and it came back with the camera position and the direction the camera was facing.
photo → 27,335 → 171 → 14,372 → 22 → 3 → 1 → ±2 m
| Step | What it did | Candidates left |
|---|---|---|
| Read the photo | Poles on the viaduct are catenary masts, so it is an electrified railway. Pier spacing used as a ruler (32 m span assumed): the left segment is about 0.5 km away, the right one over 1 km. A steep mountain about 3 km away. Rice harvested but grass still green, so no frost yet. | South China, as a bet, not a proof |
| Region scan | Pulled every railway bridge in the region from OpenStreetMap: 27,335 segments. Sampled a point every 400 m and computed the 360° horizon from elevation data at each one. Kept points with flat ground nearby, a clear mountain within a few km, and a flat horizon next to it. | 171 sites |
| Skyline fit | Placed candidate camera positions around each site and rendered the ridge line seen from each one: 14,372 positions. The top 20 were within 0.1° of each other, so it added a constraint: the bridge must be near on the left and far on the right. | 22 |
| Overlay check | Drew the top three ridge lines back onto the photo. Score #1 (Fuzhou) had a bump hidden behind the oven, which is why it scored well. #3 (Huizhou) sloped where the photo is flat. #2 (Qingyuan) fit from the foot of the mountain to the edge of the frame. | 1 |
| Pier count | 17 piers in the photo become 17 bearings from the camera. Where they hit the railway line, the intersections must be evenly spaced. Combined with the skyline: first a band about 300 m long, then a single spot. | ±2 m |
geo-sleuth is a standard Agent Skill: one folder holding SKILL.md, scripts/, references/ and data/. Install it with the skills CLI, or copy the folder yourself.
All six agents, user-wide, one command:
npx skills add Oldcircle/geo-sleuth -g -a claude-code -a codex -a cursor -a gemini-cli -a opencode -a github-copilot -y
By hand:
git clone https://github.com/Oldcircle/geo-sleuth mkdir -p ~/.agents/skills ~/.claude/skills cp -r geo-sleuth/skills/geo-sleuth ~/.agents/skills/ # Codex, Cursor, Gemini CLI, OpenCode, GitHub Copilot ln -s ~/.agents/skills/geo-sleuth ~/.claude/skills/geo-sleuth # Claude Code
~/.agents/skills/ is read by Codex, Cursor, Gemini CLI, OpenCode and GitHub Copilot, so one copy there covers all five. Each agent's own folders, from its docs:
| Agent | User-wide | Per project |
|---|---|---|
| Claude Code | ~/.claude/skills/ |
.claude/skills/ |
| Codex | ~/.agents/skills/ |
.agents/skills/ |
| Cursor | ~/.cursor/skills/ or ~/.agents/skills/ |
.cursor/skills/ or .agents/skills/ |
| Gemini CLI | ~/.gemini/skills/ or ~/.agents/skills/ |
.gemini/skills/ or .agents/skills/ |
| OpenCode | ~/.config/opencode/skills/ or ~/.agents/skills/ |
.opencode/skills/ or .agents/skills/ |
| GitHub Copilot | ~/.copilot/skills/ or ~/.agents/skills/ |
.github/skills/ or .agents/skills/ |
Any other agent that reads SKILL.md and runs shell commands works the same way: put the folder where it looks for skills.
The work is split into three layers. Scripts decide, scripts perceive and rank, the model only judges among the top few.
flowchart LR
A["photo"] --> B["intake.py<br/>EXIF · OCR · reverse image search"]
B --> C["board.py<br/>candidate board: clues, likelihood ratios, ranking, next step"]
C --> D{"which branch?"}
D --> E["sun.py · terrain.py · osm.py · pose.py<br/>shadows, skylines, OSM corridors, camera pose"]
D --> F["sat_scan.py · match.py · gsv.py · baidu_pano.py<br/>CLIP-ranked satellite tiles, DINOv2+SIFT street view"]
E --> G["board.py check · report"]
F --> G
G --> H["evidence.py<br/>coordinates ± radius · evidence image · graded confidence"]
| Layer | Who | Tools |
|---|---|---|
| Decide: which candidates, how evidence scores, what can be excluded, where to scan next | scripts (the candidate board) | board.py |
| Perceive: read text, look up tables, find targets in satellite tiles, compare street view | scripts rank first, a person looks at the top few | intake.py ocr.py clues.py sat_scan.py match.py geo.py |
| Judge: pull clues from the frame, propose hypotheses, pick among the ranked few | the model | SKILL.md + references/ |
Every conclusion has to point at a command that actually ran in the session and the file it produced. Exclusions need read or computed evidence; observations and guesses can only lower a candidate's weight.
Twenty scripts, one job each. The full table with data sources is in skills/geo-sleuth/references/data-sources.md.
| What it does | Script |
|---|---|
| EXIF: GPS, capture time, equivalent focal length, heading | exif.py |
| OCR on the whole image, zoomed crops and tiles (Apple Vision on macOS, RapidOCR elsewhere) | ocr.py |
| Reverse image search on Baidu and Yandex, similar images tiled into a numbered sheet; keyword image search | revimg.py |
Steps 0–3 in one command: metadata, edge crops, variants, OCR, reverse search → intake.md |
intake.py |
| Zoom crops, edge and corner crops, tiling, pixel columns of evenly spaced structures such as piers | imgprep.py |
| Lookup tables: plate prefixes, landline area codes, calling codes, driving side, dependent territories, administrative divisions | clues.py + data/ |
| Candidate board: candidates, clues, likelihood ratios, exclusion, ranking, scan order, pre-report checks | board.py |
| Gazetteer: list sub-divisions with bounding boxes, built-up area extent | gazetteer.py |
| Place, compound or shop name → coordinate candidates, every namesake listed | poi.py |
| Sun and shadows: latitude band, time of day, street orientation, heading from lit faces, true bearings | sun.py |
| OSM Overpass: feature co-occurrence, line-to-point, route corridors, line intersections, street grid templates | osm.py |
| Satellite tile mosaics, markers, numbered thumbnail sheets | tiles.py |
| CLIP zero-shot scoring of satellite grid cells or candidate points (tracks, factories, silos, dams…) | sat_scan.py |
| Baidu panoramas / Google Street View: find points, render headings, thumbnail sheets, historical batches | baidu_pano.py gsv.py |
| Rank candidate ground-level images against the photo: DINOv2 global similarity + SIFT inliers | match.py |
| Elevation: synthetic mountain views, skyline overlays, profiles; linear feature × terrain scan, ridge extraction, batch skyline scoring | terrain.py |
| Multi-point camera pose: lat/lon, height, heading, pitch, roll, with error radius | pose.py |
| Bearings, distances, line-of-sight intersections, alignment lines, frame/occlusion checks, camera position from evenly spaced structures | geo.py |
| Evidence image: satellite tile + camera fan + comparison grid | evidence.py |
The three steps from the case above (region scan, batch skyline scoring, camera position from pier spacing) are built into the skill as subcommands: terrain.py scan / ridge / fit, imgprep.py piers, geo.py spacing. Case scripts tuned to that photo are kept in examples/rail-skyline-session/ for reference.
Per-operator measurements:
| Script | Test | Result |
|---|---|---|
match.py |
8 cases: a historical Baidu panorama batch rendered as the photo, panoramas within 150 m as candidates (Shenzhen) | ground truth ranked 1/2/4/1/1 and 5/1/6, all in the top 6, half at #1 |
sat_scan.py |
4×8 km, 364 cells at z17, 40 OSM-tagged running tracks as ground truth, multi-scale (Shenzhen) | recall@20 17/40, @30 22/40, @100 32/40, median rank 23 |
terrain.py scan / fit + geo.py spacing |
bounded re-run on the case photo above | true cluster ranks #1, final position about 2 m from ground truth |
clues.py |
6 tables, 9 values spot-checked | 9/9 correct |
The method comes from breaking down 14 videos by online-geolocation creators, 22 puzzles and a set of real runs, then turning what works into rules and scripts. v2 moves every rule that can be code into board.py, so the rules get executed, not just read.
Python 3.10+, uv and an agent that can run shell commands. Each script declares its own dependencies and uv run installs them on first use.
Optional: Google Chrome for reverse image search (uvx playwright install chromium works too), and export GEO_PROXY=socks5h://127.0.0.1:<port> to route every networked script through a proxy.
terrain.py scan / fitIssues and pull requests are welcome, see CONTRIBUTING.md. The most useful contributions are a transferable clue for references/clues/ (with a source), a new data source with its licence, or a run on your own photo where the skill went wrong and why.
skills/geo-sleuth/data/README.md.MIT, see LICENSE. Tables in data/ derived from Wikipedia are CC BY-SA 4.0; see skills/geo-sleuth/data/README.md.
Responsible use: run it on your own photos or ones you have permission to analyze, never to find people who have not agreed to be found.
中文 | English
中国创作者给全球 AI 模型设计的一场非标准化考试。这里收录首期测评视频，只做索引与导流，点击即回到 B 站观看。
A community-built, real-world test for leading AI models—curated as a searchable video index, with every view directed back to Bilibili and the original creator.
进入 B站AI无限竞技场 · 浏览 40 个主题 · 浏览 190 个视频 · 查看源数据 · MIT License
2026-09-18 实测： 从竞技场主页公开接口读取到 40 个主题、164 个不重复视频地址；再与首期腾讯文档的 75 个地址合并去重，共收录 190 个视频、33 位 UP 主。截图中的 6 个主题及其视频均已纳入。189 条视频通过 B 站视频接口读回完整元数据；另 1 条仍由竞技场公开接口列出，但原视频接口当前不可用，仓库保留其 BV 地址并明确标记。
下面直接展示当前收录的全部 190 个视频。点击封面或标题进入 B 站原视频；点击作者名进入 UP 主主页。
Continued at the source.
A public MCP server that gives AI agents real Divar knowledge: search Iran's largest classifieds, prices in Toman, categories and neighbourhoods, car mileage and phone specs, rental deposit + rent, ad details, side-by-side comparisons and a live price verdict. Read-only, no key needed. No login, no phone numbers - ever.
Live endpoint: https://divar-mcp.mmdju2.workers.dev/mcp (Streamable HTTP, stateless)
نسخه فارسی · Examples · Tool reference · Changelog
Any MCP client, one URL. Cline / Cursor / Claude Desktop (mcp.json style):
{
"mcpServers": {
"divar": { "url": "https://divar-mcp.mmdju2.workers.dev/mcp" }
}
}Then just talk: "pride under 300 million", "two-bedroom to rent in Tehran", "is this 207 a good deal?", "cheapest iPhone 13 in Mashhad".
Agents running in a browser work too - the endpoint answers CORS preflights (OPTIONS /mcp).
| Tool | What it answers |
|---|---|
divar_suggest |
Vague wording to real search terms, category slugs, city and district ids - all 237 categories and 1177 cities |
search_ads |
"Show me X", price checks - filters, sorting, paging; one call can scan and merge up to 5 pages |
ad_details |
Everything about one ad: price, specs, amenities, condition scores, photos, map, expiry, chat flag, seller type |
get_ads_batch |
Shortlist cards for up to 10 tokens - feeds compare_ads, and each card says who is selling and until when |
compare_ads |
"Which of these?" - only the specs that actually differ, plus the middle of the set and where each ad sits |
find_best_value |
"Best X under Y Toman" - picks ranked by what the budget reaches, judged against the uncapped market (market_scale) |
market_price |
"Is this price normal?" - the median of a live sample, with its size and what it kept out of the maths |
Every tool is read-only (readOnlyHint: true) and needs no credentials. MCP prompts (compare-ads, best-under-budget) and resources (divar://cities, divar://categories, divar://category-filters/{slug}) ride along - reference data without spending a tool call.
Notes for agent builders:
price_toman: null - never 0. Ads sell in hours, so link the ad URL and let the user confirm.۱,۰۰۰ تومان, repeated digits) - those ads stay in every list, labelled price_is_placeholder with a price_note, and never set a median. The evidence rides along as price_reading, so the caller judges the number instead of trusting it.price_toman is the monthly rent, deposit_toman (ودیعه) rides beside it, each with its own flag. A room in a shared home (همخونه / هماتاقی / اجاره اتاق) carries shared_housing - a room's price is not a flat's rent.divar_suggest: a district needs an id, a name alone will not filter it.find_best_value - plain search only walks the pages you ask for.find_best_value ranks priced ads first by default; include_negotiable: true adds the توافقی picks last, with price_toman: null and an "ask the seller" line.market_price is not an appraisal. It says how many ads it compared and keeps placeholder prices out of the maths.page goes up to max 50 - the caps protect agent context. Persian queries are folded (yeh/kaf, Persian digits, ZWNJ) with one automatic retry when a spelling variant comes back empty.How a question becomes an answer. No user data is stored anywhere in this path.
flowchart LR
subgraph you [Your machine]
agent[AI agent<br/>Cline / Cursor / Claude]
end
subgraph cf [Cloudflare Workers]
worker[divar-mcp<br/>stateless, no database]
end
dv[(Divar public web listings<br/>api.divar.ir)]
agent -->|POST /mcp<br/>Streamable HTTP, no key| worker
worker -->|HTTPS + polite pacing<br/>reads only| dv
dv -->|large JSON payloads| worker
worker -->|small cards<br/>toman, district, URL| agent
What this means:
readOnlyHint. Nothing here can post, change or delete anything.Don't take my word for it - check the live server yourself:
node scripts/verify-live.mjs # needs Node.js 18+, nothing to installIt lists all 7 tools over Streamable HTTP, runs a search + details read + a market_price pricing + a privacy sweep + error paths, asserts the honest-data contract (Toman prices, negotiable = null, actionable errors), and compares the version the live service reports against the newest release in this repo - so a deployment that lags these docs cannot stay quiet. The same script runs hourly in CI (). See docs/architecture.md for the full path, and examples/python.py for a copy-paste client.
Phone numbers need the seller's own login, and this server never logs in and never returns them - no phone, mobile or contact_number field appears in search results or ad details. Ads are linked, not contacted: the user talks to the seller themselves.
Divar's public web listings (undocumented, may change without notice). This project is not affiliated with or endorsed by Divar.
Free public service on Cloudflare Workers. Fair use: 60 requests per minute per IP on /mcp (HTTP 429 with retry-after) - enforced in the server and by a Cloudflare edge rule, details in SECURITY.md.
Showcase repository (docs only, no source published) - see LICENSE. Security notes in SECURITY.md. Persian version in README_FA.md.
A server implementing the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang.
Each container one B200 with SGLang 0.5.19's Rust frontend,
radix caching, and breakable prefill CUDA graphs. A separate Python API process
uses FastAPI, uvloop, the Rust-backed HF tokenizer, and pooled asynchronous HTTP
connections to SGLang on localhost. CUDA dependencies stay in SGLang's container;
uv sync on your laptop installs only the API, deployment tools, and tests.
uv sync # Only if you haven't authenticated Modal on this machine: uv run modal setup # Start a temporary Server, run actual inference checks, then shut it down: uv run modal run modal_app.py # Deploy a stable public endpoint: uv run modal deploy modal_app.py
The deployment prints a https://...us-west.modal.direct URL. It uses a
Modal Server, unauthenticated=True,
routing_region="us-west", and compute_region=["us-west", "us-central", "us"].
Autoscaling has no explicit container cap and scales to zero after five idle minutes.
Set min_containers=1 in modal_app.py to keep a B200 warm.
If SGLang exits unexpectedly, the API exits too. The Modal launcher watches the
API and exits the container so Modal can replace it, rather than leaving a live
HTTP process with a dead inference backend. Normal shutdown disarms both watchers.
Cache warmups also request one unused token probability to avoid SGLang's mixed-logprob batch crash. This keeps warmups and scoring requests batch-compatible without patching SGLang.
The first build imports a large SGLang image. The first GPU start also downloads
weights and compiles/captures kernels. Model weights persist in the
openjev-huggingface Modal Volume, alongside SGLang's tuning cache and Triton
compilation cache. Later starts reuse these files; CUDA graph capture still runs
at startup. The Rust frontend receives an explicit local tokenizer directory to
avoid remote-name lookup issues with revision-pinned snapshots.
A scaled-to-zero Server returns 503 while it
starts; the included smoke command retries startup responses.
uv run openjev smoke https://YOUR-SERVER.us-west.modal.direct
The smoke test covers all three answer types, a 64-answer question, basic semantic
sanity checks, and rejection of 65 answers. It reports startup wait, inference
latency, and cache usage. modal run saves this report as smoke-result.json.
curl "$OPENJEV_URL/v1/systemone" \ -H 'Content-Type: application/json' \ -d '{ "model": "jev-latest", "state": [ {"role": "system", "content": "You are a support assistant."}, {"role": "user", "content": "I was charged twice. Please refund the duplicate."} ], "questions": { "refund": { "type": "noul", "instructions": "Does the user request a refund?" }, "department": { "type": "choice", "instructions": "Which department should handle this?", "criteria": {"billing": "Payments and refunds", "technical": "Software bugs"} }, "urgency": { "type": "score", "instructions": "How urgent is the request?", "criteria": ["Routine", "Urgent", "Emergency"] } } }'
Or use curl "$OPENJEV_URL/v1/systemone" -H 'Content-Type: application/json' --data-binary @examples/request.json.
state and instructions accept strings, JSON objects, or arrays. A state that is
a list of chat messages, or exactly {"messages": [...]}, is rendered using the
model's native chat template. Original roles and message objects are retained;
the classification question becomes an additional user turn, even after another
user turn. Other structured state is serialized intact into a user message.
Objects containing messages plus additional fields are kept intact so metadata
isn't silently discarded. Chat state supports text, not image/audio/video content.
| Route | Purpose |
|---|---|
POST /v1/systemone |
Noul, Choice, and Score evaluation |
GET /v1/models |
Model catalogue with TypeSafe and OpenAI-style fields |
GET /v1/limits |
Admission limits |
GET /health |
Readiness, including SGLang health and startup duration |
GET /health/live |
API process liveness |
GET / |
Scalar API reference with an editable example and request client |
GET /docs |
Built-in Swagger UI |
GET /openapi.json |
Generated API schema |
jev-latest is a compatibility alias for the configured Qwen model. The public
model ID is Qwen/Qwen3.6-35B-A3B;
NVIDIA's repository is only the internal weight source. Set
OPENJEV_SERVED_MODEL_NAME or openjev serve --served-model-name NAME to override
the public ID. It is also passed to SGLang as --served-model-name.
No requests go to TypeSafe. The public server is
the evaluation API; SGLang's generation and administration routes remain on
localhost and aren't forwarded publicly.
/generate with max_new_tokens=1, await completion,
and discard the sampled token. This warms SGLang's radix cache.prefix + question suffix + assistant header + "Answer:\n"
for each question. Every call again has max_new_tokens=1. Request
token_ids_logprob for every answer label and logprob_start_len=-1, so there
is no need to recompute prompt logprobs. The sampled token itself is ignored.P(yes), Choice returns the argmax and full distribution, and Score returns
sum(level_index * probability) with zero-based levels and a legend.Options are rendered as A: description, B: description, etc., without JSON
wrappers. Choice keys identify response fields and are hidden from the model,
except when a description is null: then the option key supplies its meaning,
matching Jev's nullable description schema.
This is a prefill plus first-token-readout workload: there is no generated chain of thought and no autoregressive continuation after the first token. There are N+1 one-token calls for N questions, including the cache-warming call. Speculative decoding is not enabled.
Qwen tokenizes 10 and 64 as multiple tokens. Answer labels are therefore
A–Z, followed by verified single-token letter combinations (AA, AB, ...).
All 64 labels are checked against the actual tokenizer at startup. Your original
option names are preserved in the returned distribution. This keeps 64-way
classification an exact one-token readout instead of comparing only the first
digit of a multi-token number.
Radix reuse is opportunistic, not a pinned per-request KV session. Hybrid Qwen's
recurrent state, cache page boundaries, cache pressure, and concurrent requests
can reduce hits. The backend uses --mamba-radix-cache-strategy extra_buffer.
x-openjev-prefix-tokens exposes the requested common prefix size. When SGLang
reports cache counts, x-openjev-cached-tokens sums the branch cache hits. SGLang
0.5.19's Rust frontend omits these counts: the header is absent and smoke reports
null, rather than a misleading zero. Scheduler logs still show actual cache
hits (verified on the live B200 deployment). Server-Timing separates
prompt preparation, the shared prefill, and branch inference.
usage.input_tokens sums SGLang's full prompt counts across the warm-up and all
branches, including cached tokens. usage.output_tokens is N+1. These are backend
usage counts, not TypeSafe billing estimates or unique tokens actually computed.
Defaults are 64 questions, 2–64 answers per Choice/Score, 2 MiB JSON,
32,768 tokens per branch including its output, 262,144 total submitted input
tokens, 16 simultaneous evaluations, and 64 simultaneous backend calls.
Invalid requests return 422 before inference; oversized bodies return 413;
overload returns 529 with Retry-After. Backend timeouts return 504. Failed or
cancelled evaluations cancel sibling requests and attempt to abort them in SGLang.
All settings can be provided as OPENJEV_* environment variables; see
src/openjev/config.py. Common settings:
| Variable | Default |
|---|---|
OPENJEV_MODEL |
nvidia/Qwen3.6-35B-A3B-NVFP4 |
OPENJEV_SERVED_MODEL_NAME |
Qwen/Qwen3.6-35B-A3B (profile-specific public name) |
OPENJEV_REVISION |
Pinned NVIDIA checkpoint revision for the default model |
OPENJEV_FRONTEND |
rust (python is an explicit fallback) |
OPENJEV_MAX_INPUT_TOKENS |
32768 |
OPENJEV_MAX_TOTAL_INPUT_TOKENS |
262144 |
OPENJEV_MAX_CONCURRENT_REQUESTS |
16 |
OPENJEV_MAX_CONCURRENT_BRANCHES |
64 |
OPENJEV_REQUEST_TIMEOUT |
120 seconds |
OPENJEV_TEMPERATURE |
1.0, applied during label normalization |
OPENJEV_API_KEY |
Unset; optional Bearer authentication for the API |
OPENJEV_BACKEND_API_KEY |
Unset; optional separate SGLang Bearer key |
The Modal launch script forwards OPENJEV_PROFILE, OPENJEV_FRONTEND, and
OPENJEV_SERVED_MODEL_NAME from the local environment. To customize other remote settings,
add them to image.env(...) or
use a Modal Secret for keys. The default Modal endpoint intentionally has no auth.
OpenJev defines
confidence = 1 - H(probabilities) / log(number_of_options), clamped to [0, 1].
This is zero for a uniform distribution and one for a point mass. Probabilities
are conditioned on the supplied options, depend on prompt and label ordering,
and are not calibrated estimates of correctness.
uv sync uv run pytest # offline unit + API tests uv run pytest -m integration # real tokenizer, small HF download, no GPU uv run ruff check . uv run openjev schema # no GPU or model download # Connect to an existing backend; it must have matching model/tokenizer revision, # selected-token logprobs, radix cache, and a sufficient context length: uv run openjev serve --connect http://127.0.0.1:30000 # On a B200 host/container with SGLang 0.5.19 installed in another environment: uv run openjev serve --sglang-python /path/to/sglang/bin/python
English · 简体中文
Investigations grounded in evidence. Responses checked against outcomes.
CyberGuard provides investigation and response-governance infrastructure for Agents. Your existing Agent submits materials through a Skill; AgentTeams plans native tasks, investigates and independently reviews the findings, then returns a report with source quotations. Security teams can also use proposal-bound approvals, execution and outcome probes to distinguish a successful command from a resolved incident.
New walkthrough video · Finals walkthrough · Capabilities and evidence · Deploy and connect · Documentation
Latest runs: the same synthetic cryptomining case completed twice on the same configuration: Skill → native AgentTeams tasks → investigation → independent review → report delivery, in 5m 7s / 3m 52s. Both reports identified that stopping the process did not establish clearance, and limited the later successful checks to the observed window. Original reports, workflow records and review →
| Your task | CyberGuard provides | Start here |
|---|---|---|
| Delegate an investigation from your existing Agent | Connection instructions, material submission and backend reports | Connect your Agent |
| Review a proposed response | A console for targets, reasons, approval decisions and incident history | Operations console |
| Check whether a response worked | Run-linked observations, execution records and verification results | Real process laboratory |
| Bring existing security telemetry | Suricata file ingest and configurable read-only HTTP connectors | Ingest · Connectors |
Start with the web setup wizard. On Linux / WSL2 with Git, Python 3.12+ and Docker Compose, run from the repository checkout:
sudo python3 deploy/onboarding/bootstrap.py
Open http://127.0.0.1:18120/setup. Create an administrator, enter your model endpoint, model name and API key, test the connection, initialize AgentTeams and enable investigations. Existing administrators can open Settings → Deployment wizard. Credentials remain on the deployment host. Setup guide (Chinese) · Chinese documentation
For manually managed deployments, follow the native backend installation guide. The following commands start the base console separately.
For analysts and teams who need an incident queue, approval screens, roles, API keys and an audit history.
With Git, Python 3.12+ and Docker Compose, run the following in Linux / WSL2 Bash:
git clone https://github.com/elsechord/CyberGuard.git cd CyberGuard python deploy/init_secrets.py docker network inspect agentteams-net >/dev/null 2>&1 || docker network create agentteams-net CYBERGUARD_COOKIE_SECURE=false docker compose up -d --build
This command uses local HTTP sessions. For public deployment, keep secure cookies enabled and configure HTTPS as described in the deployment guide.
| Local entry | Purpose | Next step |
|---|---|---|
http://127.0.0.1:18120 |
Multi-user operations console | Create the first administrator and configure HTTPS/sessions |
http://127.0.0.1:18100/console |
Read-only evidence audit view | Collect the first fixture evidence |
The base deployment includes the console and a simulation response backend. Configure the administrator and HTTPS sessions, then follow the fresh-install guide to connect AgentTeams and your model for live investigations.
Then connect your Agent: open /connect, generate Skill v0.2.0 instructions and an investigation API key, and save the key in a private file on the Agent host. The Agent installs the Skill and checks its connection; you can then ask it to submit materials, follow tasks and retrieve reports. Connection guide →
You can also submit text or multi-source JSON directly in /investigations. Logs, financial records, audit reports and judicial documents share one material envelope, supporting plain text, JSON, CSV and Markdown. Source text and submitter interpretation stay separate. Material formats and domain adaptation →
The SQLite queue and stage checkpoints keep the investigation moving after the calling Agent disconnects. The Console shows waiting, running, failure and delivery states. Runtime configuration and task controls →
Keep your current Agent as the interaction entry point. The Skill's main role is backend delegation: submit authorized materials, inspect task status and retrieve the report. It requires file access and Python 3.10+, while the deployment supplies the AgentTeams runtime and model configuration.
Copy this into your coding Agent:
Set up CyberGuard Skill v0.2.0 in my current project and verify the configured
console connection. Obtain the source from https://github.com/elsechord/CyberGuard
in a separate directory, record the checked-out commit, and read
docs/EXTERNAL_AGENT_SKILL.md and integrations/agent-skills/cyberguard/SKILL.md.
Inspect scripts/install-agent-skill.py before using it. Choose --agent codex,
--agent claude or --agent generic for this host and --project for this project.
Reuse a compatible installation; do not overwrite existing files.
Use the console origin and private key-file path supplied by /connect.
Run check --investigations and report the actual result. If configuration is
missing, explain what is needed. This setup request does not authorize uploading
materials, starting an investigation, deploying services or taking response actions.
Or install from a checkout:
python scripts/install-agent-skill.py --agent codex --project /absolute/path/to/project
# For Claude Code, use --agent claude. The project must already exist.No console yet? Try the synthetic offline exercise, then connect the online backend for native tasks. Existing incident packages remain available through check / fetch.
One live case now runs end to end: fresh process evidence enters native AgentTeams investigation; the report is converted into a bounded proposal. After approval and execution, independent probes detect recurrence. The new evidence triggers another native investigation, proposal, approval and verification. One 11m 18s run passed 20 checks, with separate investigation and review Workers in each round. Original records and how to run it →
The Linux process lab demonstrates why an execution receipt is not an outcome check:
| Step | What happens |
|---|---|
| Observe | Collect a harmless experiment process and its persistence configuration. |
| Propose and approve | Bind approval to a specific process target. |
| Execute | The executor terminates that process successfully. |
| Verify | The supervisor restarts it. Independent observations return failed. |
| Propose again | A new proposal targets the persistence configuration and receives a new approval. |
| Verify again | The experiment process and persistence are absent during the observation window; the control workload continues. Result: verified. |
The demonstration operates harmless processes and real files in an isolated Linux environment. Published validation uses test approval; operators can choose interactive approval for a live demonstration. Run the dynamic proposal demo →
Earlier exercises and component checksStart with the interface that matches your workflow:
| Integration | Input → output | Available today |
|---|---|---|
| Existing Agent / internal application | Authorized materials + objective → durable task → backend report | Skill v0.2.0 · Task API |
| Existing incident package | Incident JSON / read-only API → evidence for local review | Legacy check / fetch |
| IDS export | Suricata EVE JSON/JSONL → normalized evidence | File ingest adapter |
| SIEM / EDR / NDR / CMDB | Configured upstream HTTP response → incident evidence | Server-owned connector configuration; vendor mappings need adaptation |
| Internal application | Scoped API request → incident, evidence and proposal data | Console API v1 |
| SOAR / device execution | Approved proposal → response action → outcome observation | Executor extension work; production vendor-specific mutating integrations are not included |
For example, read incidents from a deployed console using a key with incidents:read scope:
# Bash. Configure these environment variables locally; do not put keys in source. curl --fail --silent --show-error \ -H "Authorization: Bearer ${CYBERGUARD_CONSOLE_API_KEY}" \ "${CYBERGUARD_CONSOLE_URL}/api/v1/incidents?limit=5"
List responses use data, has_more and next_cursor. Read one incident at /api/v1/incidents/{incident_id}. API key scopes control endpoint permissions. Endpoints, authentication and errors →
flowchart TD
S[Source text / external Agent] --> K[Skill or console submission]
K --> T[Investigation task service / SQLite checkpoints]
T --> P[AgentTeams Leader / Project DAG]
P --> I[Investigation Task / Worker]
I --> V[Independent verification Task / Worker]
V --> R[Report with material references]
R --> C[Console / calling Agent]
L[Fresh security observations] --> K
R --> O[Bounded proposal converter]
O --> H[Specific proposal approval]
H --> E[Response executor]
E --> Q[Independent outcome probes / audit]
Q -->|Failed: new evidence| K
Q -->|Verified: observed window| C
AgentTeams handles native investigation and independent review. A bounded converter maps the report and fresh targets to allowed executor proposals. Approval precedes execution; new outcome observations determine whether investigation should continue. The recorded case uses harmless processes in an isolated Linux environment and test approvals, with interactive operator approval available. Complete workflow →
Adaptive collaboration: Workers can create temporary specialists, exchange directed questions and clean them up through native WorkerFlow. Research · Validation
Evidence: normalization, source metadata, identifiers and citation checks. Observation model · Contracts
Controlled actions: allowlisted dispatch, proposal-bound approvals, idempotency and action audit records. Executor · Threat model
Outcome checks: account-access and process-state probes, run correlation and evidence export. Run example
Agent integration: ten AgentTeams role Skills, the portable Skill, and local AgentTeams setup.
Optional model admission guard: per-run budget reservations and role-bound routes. Guard documentation
Security is the first application domain. Financial, legal and other text can reuse the intake and review pipeline, with specialized interpretation, policies and outcome checks supplied through domain adaptation.
| Goal | Guide |
|---|---|
| Install or troubleshoot | Quickstart · Console deployment |
| Delegate from your Agent | Skill setup and connection · Tasks and materials · Backend configuration |
| Run a multi-Agent task | AgentTeams bootstrap · Local setup |
| Inspect the trust boundaries | Threat model · Live connectors |
| Reproduce and evaluate | Judge service checks · Investigation evaluation |
| Find releases and competition material | Releases · Competition guide |
Useful contributions include sanitized integration examples, connector mappings, independent outcome probes and reproducible failure cases. Start with an issue describing the input, expected result and reproduction steps. Do not include credentials or private telemetry.
For local tests, follow the development instructions. Run each test file in its own interpreter: several services use the same Python package name. CI is linked above. Upstream integration work includes the AgentTeams Worker console-binding PR #1287.
Apache-2.0. Third-party assets retain their respective licenses; typography notices for the README artwork are in brand assets.
A curated awesome list of public projects and practices built on Jev, TypeSafe AI's System One model for typed decisions.
This README is the homepage aggregate of the current category files, so the latest accepted entries are visible here without drilling into subpages.
Jev is not a chat model. It takes unstructured state plus a typed question and returns a typed decision — a choice, a score, or a boolean, each with a confidence. That makes it a drop-in decision layer for software: classification, routing, rubric scoring, verification, and agent guardrails. This list tracks who is actually building with it, and which patterns transfer across industries.
The repository treats all categories equally — each entry lives in exactly one category, chosen by its direct Jev application domain. A dedicated Related Practices / Discussions category captures credible public practice signals — X threads, Reddit discussions, and interviews — that describe real Jev usage even when no strong standalone case page exists yet.
Warning
A listing is not an endorsement. This project applies inclusion rules only — public, citable, genuinely uses Jev for a typed decision, one-sentence summary. It does not review code quality, security, maturity, or whether a project runs at all.
Treat same-day bulk submissions with particular care. Several repositories published together by one author, sharing a scaffold and a thin commit history, can satisfy every inclusion rule and still be unproven. Volume is not evidence of quality. See Curation is not endorsement for a checklist to run before adopting anything here.
Most Jev discussion is scattered across launch threads, model-gateway listings, and one-off prototypes. This list answers two practical questions quickly:
This is not a comprehensive database. It is a high-signal, fast-scanning field guide.
An entry should meet all of the following:
Jev/jev, cites TypeSafe AI's System One models, or shows a typed-decision loop (typed question → typed answer with confidence → accept/reject/escalate).We do not include:
Inclusion means one thing: the entry satisfies the inclusion rules above. It is not a quality review, a security audit, or a recommendation. We do not verify that a project compiles, that its tests pass, that its published numbers reproduce, or that its license permits your use.
This matters most for projects that arrive in bulk. When one author releases several repositories on the same day, they commonly share a single scaffold — the same AGENTS.md, CLAUDE.md, STATE.md, and CHANGELOG.md — land in one or two commits each, and may ship considerably more prose than code. Such projects can be entirely legitimate; they are simply unproven. Treat them as leads, not as validated tools.
Before adopting an entry, check it yourself:
| Check | Why it matters |
|---|---|
| Does the code actually call the Jev API? | An entry can read well on a README alone. Look for a real request carrying typed questions, and a parsed answer coming back. |
| Is there a runnable check? | A test, an example with expected output, or a public demo. No check means no evidence that it works. |
| Do the numbers have a source? | Any accuracy, latency, cost, or volume figure should be traceable to the linked page. We strip claims we cannot verify, but the project page itself may still carry them. |
| How much of the repository is code? | Some projects are mostly prompt documents. That can be legitimate — just know which one you are getting. |
| Is there a license? | A few entries have none, which limits reuse and redistribution. |
Found something wrong? Open an issue or a pull request — removal is as valid a contribution as addition. Rules for AI-assisted work, project depth, and submission rate live in CONTRIBUTING.md.
Each entry lives in exactly one category. When a project could fit multiple categories, we choose the one closest to its direct application domain.
Optional tags on an entry name the coding agent it targets and the kind of integration it is. Most entries carry none — they are added only when the source itself supports the classification.
Source file: categories/classification-routing.md
Continued at the source.
English · Русский
Made with AI. Made by you.
A local, GPU-powered music generation and production studio — one interface for ACE-Step 1.5 and YuE2-3B, with a built-in multitrack DAW.
🚧 Actively in development — expect breaking changes, bugs, and rough edges. Not a stable release yet.
Why · What's inside · ACE-Step · YuE2 · LoRA · DAW · Desktop app · Built with · License · Installation
ACE-Step and YuE2 are two independent music generation engines, each with its own web UI, its own result-storage format, and its own process that has to be started and stopped by hand. They typically cannot run simultaneously on a single consumer GPU. Remiqora solves this with a single layer on top:
| Module | What it does |
|---|---|
| ACE-Step 1.5 | Fast generation from text/style tags, covers, section repainting, extracting/adding parts on top of a reference track. |
| YuE2-3B | Full-length track generation with CoT score planning (a symbolic ABC plan before the audio). |
| SheetSage2 | Extracts melody and harmony from a reference track into ABC notation — used as YuE2's input. |
| LoRA training | Dataset → auto-labeling → preprocessing → training → export — the whole ACE-Step fine-tuning pipeline for your own voice/style, in the browser. |
| Demucs | Splits any track into 4 stems: vocals, drums, bass, other. |
| MuScriptor | Transcribes audio (the full mix or a single stem) into MIDI notes. |
| Built-in DAW | A multitrack timeline editor for assembling tracks/stems into a final mix: an effects rack on every channel, auto-BPM and time-stretch, WAV/MP3 export. |
The interface is fully bilingual (Russian/English). It starts in your system language, and the switcher in the header overrides it.
Two input modes: “Simple” — a single text description the model uses to infer both style and lyrics on its own; and “Custom” — style tags with autocomplete plus lyrics with structure markup ([Verse]/[Chorus]/[Bridge]) and performance annotations ((whisper), (falsetto)), or an “Instrumental” checkbox.
Attaching a reference track unlocks 5 remix scenarios:
Plus: 10–300 s duration, batch of 1/2/4 variants, mp3/wav/flac formats, advanced parameters (BPM, key, time signature, vocal language, inference steps, guidance scale, seed), LoRA adapter support with adjustable strength, local presets, and a "Stop all" button for bulk job cancellation.
Three CoT (Chain-of-Thought) modes: off — straight to audio; melody — the arrangement is built around a given melody (ABC); full — the model first builds a symbolic plan (melody + chords), then generates the audio.
SheetSage2 lets you upload a reference track and pull its melody into ABC notation, right in the form, with one click — editable by hand afterwards. Beyond that: q8_0/q4_0 precision, batch of 1–4, a full set of sampling parameters for audio generation and the ABC planner separately, local presets, and viewing/reusing the ABC score of an already-generated track.
The full ACE-Step fine-tuning pipeline on your own dataset, no console required:
One click splits any saved track into 4 isolated stems (Demucs htdemucs), with a progress bar, a separate player and download per stem, and the option to redo or delete. Runs alongside the active generation model (without stopping it), sharing a GPU lock. The "Open in editor" button allows you to instantly send all 4 stems into a new built-in DAW project for further mixdown.
Transcribes the full mix, or any already-separated stem, into MIDI. Technically this isn't a separate process — it's a model loaded into the already-running YuE2 server, so transcription requires YuE2 to be the active model. Result: a built-in Web Audio synth player, a mini piano roll, a note count and BPM readout, and .mid download.
Any number of tracks, onto which you can add anything from the shared library (a full mix, a single stem, a file uploaded from disk) — via a picker dialog or by dragging a file straight onto a track. The quickest way in is through stems: the "Open in editor" button on the stems panel creates a ready-made four-track project (vocals, drums, bass, other).
S), duplicate (Ctrl+D), delete (Delete).Ctrl+Z / Ctrl+Y) — up to 30 steps of history. Zoom with Ctrl+wheel or the slider and Fit button; pan the timeline with Shift+drag or the middle mouse button.The "?" button in the toolbar opens built-in help: a list of hotkeys, mouse controls, and short tips on Loop and Magnet.
backend/ — FastAPI (Python). app/orchestrator/ manages the models' process lifecycle (start/stop/health-poll) and enforces their mutual exclusion on a single GPU. app/api/routes_proxy.py reverse-proxies /api/ace/* → ACE-Step's REST API (port 8001) and /api/yue2/* → YuE2's native server (audiocpp_server.exe, port 8080). app/db.py + routes_tracks.py are the shared SQLite database and files, organized per model, regardless of how a track was created (generation, upload, or assembled in the editor).frontend/ — Vue 3 + TypeScript + Tailwind v4 + Pinia + vue-router + vue-i18n. A fully native implementation (not an iframe) on top of the models' original APIs — src/audio/ contains its own Web Audio engine (mixer, timeline, effects, a MIDI parser and synth, WAV/MP3 encoders).desktop/ — an optional Electron shell and installer: first-run setup, server lifecycle and packaging. It runs the same backend/ and frontend/; see desktop/README.md.acestep-api and audiocpp_server.exe) runs from their original code — everything else (UI, proxying, storage, file upload/transcoding) is written in this repository. YuE2's own web UI (web-ui/server.py) is no longer used — the one useful part of it (transcoding non-WAV uploads via ffmpeg) has been ported to backend/app/api/routes_yue2_upload.py.Remiqora is a UI and orchestrator on top of third-party inference engines. Their code isn't vendored into this repository — only small functional patches (external/patches/) on top of the originals:
| Project | What's used | License |
|---|---|---|
| ACE-Step-1.5 | Text/style-driven music generation engine, LoRA training | MIT |
audio.cpp (dev branch) |
YuE2 (generation), SheetSage2 (melody extraction), MuScriptor (MIDI transcription) | Apache-2.0 |
| Demucs | Stem separation (htdemucs) |
MIT |
Patch details and exact base commits are in external/patches/README.md.
The desktop app additionally uses Electron (MIT), electron-builder (MIT), uv (MIT or Apache-2.0) and static FFmpeg builds (GPL) that it downloads on first launch instead of redistributing.
Remiqora's own code (this repository) is MIT-licensed. That covers the UI and orchestrator only — it is a separate thing from the license of a track you generate with it. Remiqora is an orchestrator, not a generator with its own model — all audio is produced by third-party engines (ACE-Step 1.5, YuE2-3B, and the SheetSage2/MuScriptor tools built on top of them). Because of that:
There are two ways to install Remiqora: the desktop app (experimental, described first) or the scripts (Steps 0–2 below).
For anyone who would rather not use a terminal, Remiqora also comes as a desktop app for Windows (NVIDIA RTX 20-series or newer, driver 580 or newer) and macOS (Apple Silicon). It opens in its own window and sets everything up on the first launch, so there is no Git, Python, CUDA Toolkit or compiler to install. The Windows installer installs per user and needs no administrator rights.
Download (v0.2.1, pre-release): Windows installer (.exe) · macOS installer (.dmg, Apple Silicon) · all files and SHA-256 sums
Status. Experimental. The installers are not signed yet, so Windows shows a SmartScreen warning ("More info" → "Run anyway") and macOS may ask you to allow the app ("Open Anyway" in System Settings → Privacy & Security). The SHA-256 sum of every file is in SHA256SUMS.txt on the release page. To build an installer yourself instead:
cd frontend && npm ci && cd ../desktop && npm ci npm run dist # Windows: dist/Remiqora-Setup-<version>.exe · macOS (run it on a Mac): dist/Remiqora-<version>-arm64.dmg
desktop/README.md covers what the first run installs, the test switches and the known gaps.
What it is built with. An Electron shell around the same web UI and FastAPI backend, packaged with electron-builder (an NSIS installer on Windows, a DMG on macOS). The first launch uses uv for the Python environments, the audio.cpp release binaries and static FFmpeg builds. Licenses are unchanged; in particular the YuE2-3B weights stay CC BY-NC 4.0.
The steps below install from scripts instead: Git, a terminal and, on Windows, the build tools.
setup_prereqs.bat
Via winget (built into Windows 10/11), installs Git, Python, uv, Node.js,
CMake, ffmpeg, plus Visual Studio Build Tools (C++ workload) and the CUDA
Toolkit — those are large, need admin rights, and can take a while.
setup_prereqs.bat -SkipHeavy installs only the small, fast tools, leaving
Build Tools/CUDA for you to install manually from links the script prints.
The NVIDIA GPU driver is deliberately left out — install it by hand from nvidia.com/drivers for your card: silently swapping a video driver on someone else's machine is risky (it can blank the screen and usually needs a reboot on your schedule, not the script's).
After installing, close the terminal and open a new one so PATH picks up the freshly installed tools.
On macOS (Apple Silicon):
./setup_prereqs.sh
Via Homebrew, installs Git, Python, uv, Node.js, CMake,
ffmpeg and Ninja. No separate GPU driver step: Metal is built into macOS.
CMake/Ninja are only actually used by the --from-source build path below —
the default YuE2 setup needs no compiler at all.
setup_models.bat
The script:
ace-step/ACE-Step-1.5 (MIT) and 0xShug0/audio.cpp (Apache-2.0,
dev branch — YuE2 support is dev-only for now) into external/.external/patches/README.md) — without the upstream
custom web-uis, which aren't needed.uv sync for ACE-Step and builds audiocpp_server (CUDA release,
yue2,sheetsage2,muscriptor models) for audio.cpp.tools/model_manager_v2.py.demucs uv project in external/Demucs for stem separation,
routed at PyTorch's cu128 wheel index so it gets a CUDA build (a plain
uv add demucs would silently resolve a CPU-only torch wheel instead).backend/.env with paths to the freshly cloned repositories,
including FFMPEG_BIN_DIR — auto-detected from ffmpeg's winget install
(setup_prereqs.bat), even right after installing it in the same
terminal, before a new one would pick it up on PATH.ACE-Step's own weights don't need a separate download — acestep-api pulls
them from HuggingFace/ModelScope on first request, the same way its Gradio
UI does.
The script is idempotent — safe to re-run (the -SkipBuild / -SkipWeights
flags skip the corresponding steps). It expects git,
uv, Python 3,
CMake, the CUDA Toolkit and Visual Studio Build Tools (C++ workload) to
already be installed — if any is missing, that step is simply skipped with a
hint on what to install.
After that, the only manual step left is checking CUDA_BIN_DIR in
backend/.env (FFMPEG_BIN_DIR is filled in automatically — unless ffmpeg
wasn't found at all, in which case the script says so and it needs setting
by hand).
Hard machine requirements the script can't remove: Windows, a CUDA-capable NVIDIA GPU (tested on an RTX 4080 16 GB), and an installed video driver.
On macOS (Apple Silicon):
./setup_models.sh
Adapted for macOS, with one difference from the Windows steps above: by
default, audiocpp_server is installed from audio.cpp's own prebuilt
macOS/Metal release (a pinned tag, sha256-verified before extracting) —
no compiler needed at all, unlike the Windows path, which always builds
from source since there's no prebuilt CUDA release. The Demucs uv project
also isn't routed at a CUDA wheel index — a plain torch dependency
already resolves an MPS-capable wheel on darwin/arm64, same as
ACE-Step-1.5's own pyproject.toml does. The written backend/.env has no
CUDA_BIN_DIR — there's no CUDA toolkit on this path.
Pass --from-source to build audio.cpp from the same pinned dev commit
Windows uses instead of downloading the release (useful if the release lags
behind a dev-only fix, or on Intel Macs, which the prebuilt asset doesn't
cover) — that path needs full Xcode.app (not just the Command Line
Tools) for its Metal shader compiler; setup_prereqs.sh prints exact steps
if it's missing. --skip-build / --skip-weights mirror -SkipBuild /
-SkipWeights. Otherwise it expects git, uv and Python 3 to already be
installed (cmake too, for --from-source).
Continued at the source.
Jev gives your software a typed judgment. Your code stays in charge.
A community field guide to TypeSafe's Jev: see one documented call, try live projects, copy a starter, and inspect independent tests.
Try a live build · Shape a decision · Explore projects · Add your project · Star on GitHub
One call, three typed answers. TypeSafe's documented support-ticket example shows the saved
jev-1.13.0response below. This is a published example, not a live model call. Application code still decides when to route or escalate.
| Input or answer | Documented value |
|---|---|
| State | Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. I'm losing sales. Please help ASAP. |
| Choice | technical · 0.85 selected probability |
| Score | 1 on a 0–2 frustration rubric (Frustrated but civil) |
| Noul | 1.0 urgency probability |
Open a live build from a preview, or read its listing first.
Three additions from 22 September 2026. These are places to explore, not a ranking or endorsement; the full listings include limitations and source links.
| I want to… | Go here |
|---|---|
| Understand the idea in 2 minutes | See where Jev fits, try the policy threshold, then read the introduction and the three primitives |
| Make my first typed call | Copy the runnable example, shape your own question, then explore the official SDKs |
| See it work live | Explore the featured builds, then browse more applications |
| Test the claims | Read what independent tests found, inspect JevBench's cross-model results, then browse independent evaluations and TypeSafe's own evals |
Download the JSON directory · Use with a coding agent · Suggest a resource · Follow updates · Join the builder community
Independent community project. This repository is not affiliated with or endorsed by TypeSafe AI. Community entries are labeled by section; inclusion is not a claim that TypeSafe has reviewed or approved them.
Last updated: 2026-09-23. Links and project descriptions change; please report a stale entry.
In a support workflow, separate the work before choosing a model. This is a practical design rule based on the TypeSafe introduction linked above, not a performance claim:
| What the step needs | Use | Example |
|---|---|---|
| Apply an explicit rule to known fields | Code | Check an account flag or enforce a routing threshold. |
| Judge messy context with a bounded answer | Jev | Choose billing, technical, or other for a ticket, with probabilities. |
| Produce prose or work through an open-ended task | Text LLM | Draft the reply after the route is chosen. |
Code still validates the answer and owns the action. Measure Jev's error and abstention rates on your own cases before automating a consequential step.
One state can answer several focused questions in the same request. Pick the answer shape your code can use directly:
| Question shape | Use it for | What comes back |
|---|---|---|
| Noul | A clear yes/no claim, such as “Does this message request a refund?” | A number from 0 to 1: the probability of yes. |
| Choice | Selecting from named options, such as billing, technical, or sales. | The selected option, a probability for every option, and confidence. |
| Score | An ordered rubric, such as calm, concerned, or angry. | A position on your rubric, probabilities over its levels, and confidence. |
Ask independent questions together. Set thresholds, fallback behavior, and side effects in application code.
The typed decision is the common idea; the client, model name, authentication, and billing depend on the route. Start with the direct API below if you want TypeSafe's documented systemOne contract, or follow the platform guide for an app already running there.
| Route | Documented way in | Check before using it |
|---|---|---|
| TypeSafe direct | Official JavaScript or Python SDK with a TypeSafe API key. | The runnable example below uses this route and sends its state to TypeSafe. |
| Cloudflare Workers AI | Run typesafe/jev with a Workers AI binding or Cloudflare API call. |
Use Cloudflare's request shape and credentials; its model page labels Jev as a third-party model. |
| Netlify AI Gateway | Use the official TypeSafe JavaScript SDK from a Netlify Function or Edge Function; the gateway supplies its environment configuration when enabled. | Follow Netlify's plan and key-override rules. This is a server-side path, not a browser key. |
| Vercel AI Gateway | AI SDK's experimental evaluate with typesafe-ai/jev. |
Its Boolean question maps to Jev's Noul; the AI SDK interface differs from systemOne. |
| OpenRouter | OpenRouter's decisions API with typesafe/jev-1.13 or its latest-model route. |
Use an OpenRouter key and its decisions request shape; do not send these questions to a chat-completions API. |
These are documented access paths, not equivalent SDKs or claims about price, latency, or reliability. Check the linked provider page before deploying because availability and terms change.
Pick JavaScript or Python. Both examples send a synthetic support ticket to TypeSafe's API and return a Choice and a Noul. Jev returns typed answers; the 0.9 routing rule is ordinary application code. It is an illustrative threshold, not a measured or recommended operating point.
Install the official JavaScript SDK with npm install @typesafe-ai/sdk (Node.js 20+), set TYPESAFE_API_KEY in your environment, save this as first-decision.mjs, then run node first-decision.mjs:
import { choice, noul, TypeSafeClient } from '@typesafe-ai/sdk'; const { answers } = await new TypeSafeClient().systemOne({ state: { ticket: 'I was charged twice. Please refund the extra payment.' }, questions: { team: choice('Which team should handle this ticket?', { billing: 'Payments and refunds', technical: 'Bugs and integrations', other: 'None of the above', }), refund: noul('Does the customer explicitly request a refund?'), }, }); const team = answers.team.choice; const probability = answers.team.probabilities[team]; const action = team !== 'other' && probability >= 0.9 ? `route to ${team}` : 'send to review'; console.log({ team, probability, refundProbability: answers.refund.noul, action });
Install the official Python SDK with uv add typesafe-sdk (Python 3.10+), set TYPESAFE_API_KEY in your environment, save this as first_decision.py, then run python3 first_decision.py:
from typesafe_sdk import Choice, Noul, TypeSafeClient with TypeSafeClient() as client: result = client.system_one( state={"ticket": "I was charged twice. Please refund the extra payment."}, questions={ "team": Choice( instructions="Which team should handle this ticket?", criteria={ "billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above", }, ), "refund": Noul(instructions="Does the customer explicitly request a refund?"), }, ) team = result.choices["team"].choice probability = result.choices["team"].probabilities[team] action = f"route to {team}" if team != "other" and probability >= 0.9 else "send to review" print({ "team": team, "probability": probability, "refund_probability": result.nouls["refund"].noul, "action": action, })
Start with one state and a question whose answer your code can use. This synthetic support report can be asked as a Choice, Noul, or Score. On the live site, edit the fields and copy a JavaScript SDK call. The designer runs in your browser without making a model request; running the copied code later sends the state to TypeSafe.
| Design input | Synthetic example |
|---|---|
| State text | The PDF upload fails with a 500 error. I need it before today's deadline. |
| Choice question | Which team should handle this report? |
| Choice options | technical=Failures and integrations; support=Account and usage help; other=Neither team |
| Noul question | Does the message explicitly mention a deadline? |
| Score question | How much does the reported issue block the user's work? |
| Score levels | Cosmetic; Workaround available; Blocks the task |
Keep the state short, describe the options so they do not overlap, and include a no-match option when the task allows it. Choose thresholds and actions only after measuring your own labelled cases.
The documented support-ticket example above selects technical with probability 0.85. In this illustrative policy, a ticket routes automatically only when the selected probability reaches the application's threshold. At 0.90, it goes to review; at 0.80, it routes to technical. The model answer stays the same. These thresholds are teaching examples, not measured operating points or safety guarantees.
| Policy input | Example value |
|---|---|
| Selected team | technical |
| Selected probability | 0.85 |
| Starting threshold | 0.90 |
On the live site, move the threshold to see which action the application takes. A real threshold needs evaluation on your own labelled cases, with a review path for uncertainty.
Independent studies make five failure modes concrete. Each result below belongs to the cited task, dataset, and model run; use it to design a test for your own workflow.
| Decision you want to make | What was measured | What to test before shipping |
|---|---|---|
| Answer or abstain? | In a KoBBQ audit, Jev chose “unknown” for 95% of 300 ambiguous items when that option was available. With that gold answer removed from the options, accuracy on those items was necessarily 0%; 79% of answers picked the dataset's stereotype. | Add an explicit no-match or review option where evidence can be missing. Measure wrong forced answers and needless abstentions on your own ambiguous cases. |
| Route to a fallback? | Janus tested 500 items each from Banking77 and Web of Science. Its tuned Jev-to-DeepSeek cascade improved Banking77 accuracy over either model alone, but on Web of Science matched Jev alone at 47% higher cost. | Label representative cases, price both legs, and choose a threshold on a held-out split. Confirm that the fallback actually fixes errors where Jev is uncertain. |
| Certify a routing threshold? | In jev-certify's CLINC150 study, a 5% bound on silently misrouted incoming queries held on 400 in-scope examples: 84.75% were auto-routed with 2.25% loss per incoming query. A separate scope gate missed its 5% target by 3.6× when out-of-scope prevalence rose. | Calibrate on traffic that represents deployment, monitor the mix, and distinguish loss per incoming query from error among routed queries. The bound does not cover a shifted population. |
| Sort by probability? | An ordering study passed six ranking gates on 360 topic-membership rows, then failed four of six on 306 human-graded shopping pairs. On the first corpus, 53 rows tied at 0.99; batching 40 rows changed a passing ranking gate into a failure. | Measure pairwise order, ties at the cutoff, and the exact request shape on your relevance labels. A good classifier is not automatically a good sort key. |
| Approve an agent action? | In a 111-case action-gate study, Jev matched 100 case labels and Claude matched 102; each had one unsafe allow. Contract and policy mapping was the largest single source of wrong decisions for both. | Test the answer-to-action mapping as well as the model. Escalate consequential tool families with deterministic policy even when a semantic answer seems confident. |
These are independent, study-specific observations, not a leaderboard or a guarantee for another task. Read the linked protocols, labels, and limitations before carrying a number into a decision policy.
For a comparison across decision models, JevBench's method publishes its scoring code, frozen tasks, adapters, and result artifacts. Its composite score combines accuracy, calibration, speed, and cost; some latency and hosting costs are estimates, and a held-out set is still sent to the evaluated services. Read the per-task outcomes and assumptions before treating a rank as evidence for your workflow.
Continued at the source.
Save Astra for the decisions that need it. Let DeepSeek V4.1 Flash do the volume.
A personal Codex skill designed to preserve Astra usage without giving up Astra's judgment. Astra stays responsible for planning, architecture, high-stakes decisions and final review. DeepSeek V4.1 Flash takes the high-volume work: repository discovery, implementation, testing, debugging and routine verification.
Bring an existing plan or start with a feature request. The workflow turns it into coherent implementation bundles, sends those bundles to Flash, then returns the completed patch and evidence to Astra for one focused acceptance pass.
Status: early release. Offline installation tests pass, and the workflow has completed a measured local field build. Results below describe that run, not guaranteed savings. A new installation still needs runtime routing verification on its first authorized task. Installation never runs paid inference.
In one substantial field build, Astra Flash Orchestrator used 98.9% less Astra input per 1,000 implementation and test lines than the all-Astra baseline. It did that by moving the implementation loop—not the important decisions—to Flash. Total API-equivalent compute per 1,000 lines was 97.0–97.7% lower, while the measured phase produced 39% more implementation and test lines.
| Workflow | Astra input per 1K implementation lines | Total compute per 1K lines |
|---|---|---|
| All Astra | 8.56M | $11.32 |
| Astra + DeepSeek V4.1 Flash | 95.9K | $0.26–$0.34 |
The per-token price difference explains why delegating implementation has so much leverage:
| Cost per 1M tokens | Astra estimator | DeepSeek V4.1 Flash | Astra premium |
|---|---|---|---|
| Uncached input | $10.00 | $0.15–$0.30 | 33–67× |
| Cached input | $1.00 | $0.003–$0.006 | 167–333× |
| Output | $50.00 | $0.60–$1.20 | 42–83× |
Astra does not have a public API SKU; its values above are API-equivalent estimates, not ChatGPT or Codex subscription charges. Flash values use published off-peak and peak API rates. See the benchmark methodology for sources, exact measurements and limitations.
Astra → scope + design + task brief
Flash → implement + test + report
Astra → review + verify + accept or request fixes
→ integrate + checkpoint + next task
astra_flash_builder role, not a separate agent CLI.This is workflow guidance, not a deterministic scheduler, a security sandbox, or a guarantee of model quality or cost savings. It is independent of OpenAI, DeepSeek and Codex Router.
There is no mode setting or mode-switch command. The package always uses the usage-saving Astra → Flash → Astra workflow for substantial implementation.
Three routing outcomes remain intentionally different:
Those are scope and safety decisions, not user-selectable performance modes.
Before installing, you need:
$CODEX_HOME/agents/.multi_agent_version: "v2".| Provider | Worker route |
|---|---|
| DeepSeek API (default) | deepseek/deepseek-v4.1-flash |
| OpenRouter | openrouter/deepseek-v4.1-flash |
| opencode Go | opencode-go/deepseek-v4.1-flash |
| Command Code | commandcode/deepseek-v4.1-flash |
| Nous Research | nousresearch/deepseek-v4.1-flash |
| Ollama Cloud | ollama-cloud/deepseek-v4.1-flash |
Provider credentials are entered by you through Codex Router's private local prompt before installing this package. Never paste an API key into an assistant chat. This installer never asks for, reads, stores or validates provider keys.
Do not spend API credit during installation. Installing this package does not authorize an assistant to run
subagents certify,test-model --live, a Router smoke test or any other paid inference probe. If the selected route is absent or is not already advertised asv2, the installer stops and reports the prerequisite. Decide separately whether to certify a route yourself.
Do not add or change [agents].default_subagent_model for this package. The
installer creates a named astra_flash_builder role that pins its own route and
catalog-supported effort, so unrelated subagents keep their existing defaults.
The installer does not install the Router, add credentials, select your root
model, or rewrite config.toml. Direct DeepSeek remains the default. Any other
provider requires an explicit --worker-route; if that route is unavailable,
installation stops instead of silently choosing another provider.
The installer supports loopback Router URLs using /v1 or /_codex-router/<capability>/v1. It rejects remote hosts, embedded credentials, queries, fragments and unexpected paths. Client/project/UI overrides still need checking in your actual session. Router subagent selection enables discovery; it does not prove successful inference. Some Router enable commands automatically launch paid verification, so inspect the installed version before changing selection. This installer never enables routes or runs those probes.
Download this repository as a ZIP and extract it, or clone it:
git clone https://github.com/ethanplusai/astra-flash-orchestrator.git
cd astra-flash-orchestratorRun the following commands from that repository folder.
The installer performs its own prerequisite checks before writing. Preview the exact destinations, then apply:
python3 -B install.py python3 -B install.py --apply
That is the normal installation path. The first command changes nothing. The second repeats preflight, installs atomically, backs up existing instructions and prints a guarded undo receipt. It does not change your root model, Router, credentials, permissions or reasoning effort.
To use an already-configured alternate provider, pass its exact route to both commands. For OpenRouter:
python3 -B install.py --worker-route openrouter/deepseek-v4.1-flash python3 -B install.py --worker-route openrouter/deepseek-v4.1-flash --apply
The option selects an existing catalog route; it does not configure the provider, collect a key, certify the model or make an inference request.
Ask Codex:
Read INSTALL-IN-CODEX.md in this folder and install the package following it.
Preserve my root model, reasoning effort, Router, config and authentication.
Do not launch workers or run paid inference during installation.
Release archives are tested before publication. If you also want to run the offline suite yourself:
python3 -B -m unittest discover -s tests -v
For a nondefault profile, pass --profile PROFILE to the dry run, apply and doctor consistently. --home and --codex-home are available for explicit location overrides. Use the same locations for undo.
| Location | Installed content |
|---|---|
~/.agents/skills/astra-flash-orchestrator/ |
Skill, references, templates, doctor, plan validator and routing binding |
$CODEX_HOME/agents/astra_flash_builder.toml |
Native builder pinned to Flash; nested agents disabled |
$CODEX_HOME/AGENTS.md |
A marked, scoped workflow policy block |
$CODEX_HOME/astra-flash-install-backups/ |
Original files and an undo receipt |
CODEX_HOME defaults to ~/.codex. An existing nonempty AGENTS.override.md receives the policy instead of AGENTS.md. Other instructions are preserved. The policy keeps trivial work single-agent and honors explicit no-delegation requests, repository restrictions and managed policies. Use --no-policy for a skill/role-only installation.
Root model/effort, provider configuration, authentication and existing permissions stay unchanged. Installation does not start services, workers or model requests, and does not commit, push or deploy anything.
Fully quit and reopen the host app (ChatGPT or Codex), then start an Astra session. A new chat alone may reuse a cached model catalog. Use:
$astra-flash-orchestrator Use the existing plan in docs/plan.md to implement
this feature. Keep Astra focused on planning and final review. Use one installed
Flash builder for a coherent implementation and verification bundle. Do not poll
the worker; review its completed patch and evidence in one batched pass.
Replace the example plan path with your actual plan or describe the feature. Your first authorized useful task should verify the child model and provider using host/router request metadata. A worker saying its model name is not proof.
If the session does not expose the custom role or exact worker model, do not substitute another model or launch a second CLI. Check client support and session configuration first.
From the repository folder:
python3 -B skill/astra-flash-orchestrator/scripts/doctor.py python3 -B skill/astra-flash-orchestrator/scripts/doctor.py --check-local-router
An installed copy reads its generated routing.json, so doctor checks the same
route automatically. Pass --worker-route only when running doctor from a fresh
source checkout or intentionally checking a different reviewed route.
The first checks local configuration/catalog data. The optional second command makes only a local /models GET, with proxies and redirects disabled. It does not read authentication files or attach credentials; an authenticated Router may reject it even when normal Codex requests work. Do not disable Router authentication to make this check pass.
Neither check proves paid inference works. See troubleshooting and validation evidence.
For an update, download the new source, run its tests, and preview python3 -B install.py --replace. Review the differences before applying with --replace --apply. Existing package-owned files are backed up; unrelated files are not deleted. An existing valid routing.json preserves the installed provider when --worker-route is omitted. Pass the option explicitly only to change providers, and review that replacement before applying it. Do not edit generated routing.json or the agent model to force a different provider through preflight.
Preview undo using the exact receipt printed during installation:
python3 -B install.py --undo /path/to/receipt.json
Add --apply to restore. Undo refuses if a managed file changed afterward, protecting later edits. Backups remain available. Keep a copy of the installer and receipt; receipts may contain private paths and original instructions and should never be published.
To validate the synthetic plan example:
python3 -B skill/astra-flash-orchestrator/scripts/validate_plan.py examples/invoice-filter/plan.json
The example is a planning fixture, not a runnable application. Markdown plans work without the optional manifest validator.
Continuous software-quality review for AI coding agents, powered by Jev.
Jev Review runs as a local MCP server and gives Claude Code, Codex, Cursor, and OpenCode structured quality scores while they work. Your coding agent remains responsible for diagnosing weaknesses and changing the code; Jev supplies a fast scalar signal across correctness, complexity, changeability, modularity, tests, security, and other independent quality dimensions.
Important
Your API key stays on your machine. Jev Review has no hosted backend, database, telemetry service, or author-operated proxy. The only remote request is sent directly to the configured Jev API.
jev-review-demo.mp4
| Purpose | Continuous, structured software-quality evaluation |
| Supported clients | Claude Code, Codex, Cursor, OpenCode |
| Distribution | This GitHub repository—no npm publication |
| Runtime | Local Node.js process over MCP stdio |
| Remote access | Direct requests to Jev using your API key |
| MCP tools | One focused tool: jev_review |
| Code changes | Always performed by the primary coding agent |
Requirements:
Set your API key before starting the coding agent:
export JEV_API_KEY="your-key"
Install Jev Review directly from GitHub—no npm publication is required:
npx plugins add NiazMorshed2007/jev-review
Choose your coding client when prompted, restart it, and ask the agent to use jev-review while implementing a nontrivial change.
flowchart LR
A[Agent implements] --> B[Focused diff and context]
B --> C[Jev Review MCP]
C --> D[Jev evaluation]
D --> E[Structured quality signals]
E --> F[Agent improves the code]
F -. review again .-> B
Jev Review is intended for frequent, focused checkpoints: after a coherent implementation slice, after a score-driven improvement, and before final handoff. The first call establishes a baseline. The agent then inspects its own implementation, forms a hypothesis about weak dimensions, improves the code, validates it, and rescores.
Jev returns typed Score, Choice, and Noul decisions rather than a free-form review essay. It does not generate a prose explanation of why a score is low. Jev Review validates and converts those decisions into metric scores, confidence levels, coarse rubric hints, and comparisons with a previous evaluation. The coding agent—not Jev—must determine the actual cause and appropriate code change.
There is deliberately no synthetic “82/100” overall score. Dimension changes such as Readability 6.3 → 8.1 and Security 8.2 → 8.2 are more useful than a blended percentage.
| Client | Plugin installation | Manual MCP available |
|---|---|---|
| Claude Code | npx plugins add NiazMorshed2007/jev-review --target claude-code |
Yes |
| Codex | npx plugins add NiazMorshed2007/jev-review --target codex |
Yes |
| Cursor | npx plugins add NiazMorshed2007/jev-review --target cursor |
Yes |
| OpenCode | Manual configuration below | Yes |
Every client starts the same bundled dist/server.js process locally over stdio.
npx plugins add NiazMorshed2007/jev-review --target claude-code
Restart Claude Code and run /mcp to confirm that jev-review is connected.
To load a local clone while developing:
claude --plugin-dir /absolute/path/to/jev-review
Manual MCP-only setup:
claude mcp add --scope user jev-review -- node /absolute/path/to/jev-review/dist/server.js
npx plugins add NiazMorshed2007/jev-review --target codex
Restart Codex and run /mcp to verify the connection.
Manual setup in ~/.codex/config.toml:
[mcp_servers.jev-review] command = "node" args = ["/absolute/path/to/jev-review/dist/server.js"] env_vars = ["JEV_API_KEY"]
npx plugins add NiazMorshed2007/jev-review --target cursor
Restart Cursor and check Settings → MCP. The bundled skill is named jev-review; invoke it with /jev-review or leave it on Agent Decides.
Manual setup in ~/.cursor/mcp.json:
{
"mcpServers": {
"jev-review": {
"type": "stdio",
"command": "node",
"args": ["/absolute/path/to/jev-review/dist/server.js"],
"env": {
"JEV_API_KEY": "${env:JEV_API_KEY}"
}
}
}
}If Cursor is launched from the macOS Dock, it may not inherit variables from your shell profile. Make the already-exported key available to GUI applications before starting Cursor:
launchctl setenv JEV_API_KEY "$JEV_API_KEY"Verify without printing the key:
test -n "$(launchctl getenv JEV_API_KEY)" && echo "JEV_API_KEY is configured"
OpenCode does not currently appear in the portable plugins installer targets. Point it at the same bundled server instead:
git clone https://github.com/NiazMorshed2007/jev-review.git cd jev-review opencode mcp add jev-review --global -- node "$PWD/dist/server.js"
For the full skill and MCP setup, add this to ~/.config/opencode/opencode.json, replacing the absolute path:
{
"$schema": "https://opencode.ai/config.json",
"skills": ["/absolute/path/to/jev-review/skills"],
"mcp": {
"servers": {
"jev-review": {
"type": "local",
"command": ["node", "/absolute/path/to/jev-review/dist/server.js"],
"environment": {
"JEV_API_KEY": "{env:JEV_API_KEY}"
}
}
}
}
}Run opencode mcp list to verify the connection. OpenCode may display the tool as jev-review_jev_review; the underlying MCP tool is still jev_review.
Jev Review intentionally starts with one tool: jev_review.
{ task?: string; diff?: string; files?: Array<{ path: string; content: string; }>; repositoryContext?: string; previousEvaluation?: Evaluation; }
At least one current-context field is required. Callers should normally send the task and focused diff, adding complete files only when the surrounding implementation is necessary to understand the change. Jev Review never reads the repository automatically.
Jev Review does not impose an additional character, token, or file-count limit. The Jev API currently enforces its own token ceiling: live jev-latest behavior indicates roughly 32,768 tokens for the submitted state, although this number is not published in the API documentation or OpenAPI schema and may change. When Jev returns max_tokens_exceeded, the server asks the agent to reduce unrelated context or split the change into coherent review slices.
The response contains:
{ "applicable": false } for dimensions unsupported by the supplied contextpreviousEvaluation is suppliedAlways evaluated when the supplied context is sufficient:
Evaluated only when relevant evidence is present:
The evaluator judges consequences in context. It does not assume short functions, small files, zero duplication, more layers, more comments, or more tests are automatically better.
The included jev-review skill teaches agents to treat Jev as a repeated scalar feedback loop:
jev_review with focused context to establish a baseline.previousEvaluation, then inspect improvements and regressions.Correctness and the user's requirements always outrank score improvement. A higher score never justifies speculative architecture, unnecessary abstraction, scope expansion, breaking behavior, meaningless tests, or needless rewrites.
jev-review/
├── plugin.json # Portable Agent Plugin manifest
├── mcp.json # Portable stdio MCP definition
├── .claude-plugin/
│ └── plugin.json # Claude Code adapter
├── .codex-plugin/
│ └── plugin.json # Codex metadata
├── skills/
│ └── jev-review/
│ └── SKILL.md # Agent review workflow
├── src/
│ ├── config/ # Environment handling
│ ├── evaluation/ # Metrics, scoring, and comparisons
│ ├── jev/ # Direct Jev client and validation
│ └── mcp/ # MCP tool boundary
├── dist/
│ └── server.js # Committed standalone server bundle
├── public/
│ └── jev-review-demo.mp4 # Product demonstration
└── test/ # Unit and MCP protocol tests
plugin.json and mcp.json are the portable Agent Plugins 1.0 package. .claude-plugin/plugin.json and .mcp.json provide Claude Code compatibility, while .codex-plugin/plugin.json supplies Codex metadata. These are small packaging adapters around one MCP implementation.
git clone https://github.com/NiazMorshed2007/jev-review.git
cd jev-review
npm install
npm run validateUseful commands:
npm run check npm test npm run build npx plugins discover . claude plugin validate . --strict
npm run build creates the committed dist/server.js bundle. Unit and MCP protocol tests use local fakes and do not consume Jev API quota; a live Jev call requires JEV_API_KEY.
The local MCP process reads JEV_API_KEY and uses it only in the TLS Authorization header sent directly to https://api.typesafe.ai/v1/systemone. Jev Review never stores or logs the key.
Only the task, diff, files, and repositoryContext explicitly supplied to jev_review are sent to Jev. previousEvaluation is compared locally and is not included in the current code context. No repository files are discovered or uploaded automatically.
Review context does leave your machine for TypeSafe's Jev API. Do not supply secrets or unrelated proprietary content, and review TypeSafe's privacy policy for the remote service's handling terms. Jev Review complements rather than replaces dedicated security tooling.
Give your AI agent answers it can act on: typed judgments with real probabilities, instead of prose it has to parse.
evaluate connects Claude Code, Claude Desktop, Codex and pi to TypeSafe's Jev model. Your agent asks a question like "is this urgent?" or "which team owns this?" and gets back a number or an option it can use in an if statement.
"Help! My payouts have been ┌──────────┐ is_urgent 0.95
failing for 3 days." ───▶ │ Jev │ ───▶ department billing (86%)
└──────────┘ technical (14%)
is it urgent? which team? sales (0%)
The problem: An agent that needs a quick judgment call usually asks an LLM, reads a paragraph back, and guesses what it meant. "This seems fairly urgent" gives the agent nothing to branch on, and it can't tell a confident answer from a coin flip.
The fix: Jev is a model built for judgments rather than text generation. You name the question and the possible answers, and Jev returns a probability for each answer in a fixed format. Your agent gets data it can compare against a threshold, and never has to parse prose.
1. Install (macOS and Linux):
curl -fsSL https://raw.githubusercontent.com/itsmostafa/typesafe-mcp/main/install.sh | sh2. Connect your agents (get a key):
TYPESAFE_API_KEY=your-key evaluate setup mcp
This finds Claude Code, Claude Desktop and Codex and registers evaluate with each one. If you already have an OpenRouter account, set OPENROUTER_API_KEY instead. To run an open model such as Laya locally, point TYPESAFE_BASE_URL at your server and keep TYPESAFE_API_KEY set, since it selects that route; any non-empty value works if your server doesn't check keys, e.g. TYPESAFE_API_KEY=local TYPESAFE_BASE_URL=http://127.0.0.1:8787 evaluate setup mcp (see custom hosts). For pi, run evaluate setup pi.
3. Ask a question:
"Use evaluate to decide whether this ticket is urgent and which team should own it: Help! My payouts have been failing for 3 days."
Your agent sends:
{
"state": "Help! My payouts have been failing for 3 days.",
"questions": {
"is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"},
"department": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"billing": "Payments, refunds", "technical": "Bugs, outages", "sales": "Pricing"}}
}
}And gets back:
{
"answers": {
"is_urgent": {"type": "noul", "noul": 0.95},
"department": {"type": "choice", "choice": "billing", "confidence": 0.79,
"probabilities": {"billing": 0.86, "technical": 0.14, "sales": 0.0}}
}
}noul), pick one option (choice), and rate on a scale (score). Each answer comes with probabilities.items and ask the same questions of each one. If one record fails, the rest still complete.evaluate setup mcp configures every supported client it finds. Run it again to update.evaluate update upgrades it in place.items, limits and errors.TypeSafe builds System One models: small units of AI judgment that you use like programming primitives. Instead of generating text, they turn natural language and application state into typed answers and probabilities that code can combine. Jev is the first of them.
Website · Docs · API reference · Console
Issues and pull requests are welcome. See docs/development.md to get started.
If evaluate spares your agent some prose-parsing, a ⭐ helps others find it.
Do not start new projects with
typesafe_sdk.TypeSafeSDK
0.4.1is the final release of this package line.New development should use:
Current release:
0.5.0
system_one_sdkis the provider-neutral successor to this project and carries the semantic question API, prepared evaluations, typed answers, batching, telemetry, runtime controls, OTP integration, testing support, and decision tooling developed here.
The functionality previously collected in TypeSafeSDK is now separated by responsibility:
system_one_sdk
The SDK application developers should use. It owns the provider-neutral
System One programming model and higher-level Elixir/BEAM APIs.
typesafe_api_sdk
The TypeSafe-specific provider/API SDK. It owns TypeSafe authentication,
endpoint and model configuration, generated wire operations, and TypeSafe API
response handling.
system_one_sdk includes a built-in TypeSafe provider backed by
typesafe_api_sdk, so normal application code does not need to assemble these
layers manually.
TypeSafeSDK 0.4.1 remains available as the final release for existing users,
but this repository is no longer the destination for new SDK development.
For migration and continued development, go to:
Historical implementation details, guides, examples, and release history remain in this repository for reference.
MIT License
Copyright (c) 2026 nshkrdotcom