On October 8, 2026, Goodfire published production cyber monitors for open models Kimi K3 and GLM 5.3. The system is a practical form of AI agent monitoring: cheap probes watch internal activations, then escalate only suspicious turns to an LLM judge.

That design matters for agencies. Open models rarely ship with the full safety stack closed labs keep on their own APIs. When agents run for hours, watching every turn with a second model gets slow and expensive.

I am Nitin from Debugged Pro. We help product and agency teams in India and the USA ship agentic software. Below is a plain briefing on what Goodfire shipped, which numbers come from the official research post, and what you should do next.

Goodfire visual of AI agents with alert signals for cyber monitoring context — Goodfire research visual for production cyber monitors on open AI models. Image via Goodfire.

What Goodfire AI Agent Monitoring Does

Goodfire's approach is probe-based, sometimes called inside-out monitoring in press coverage. A probe reads the target model's internal activations as tokens are produced. If the probe score crosses a calibrated threshold, the turn goes to an LLM judge.

Goodfire calls this a monitor cascade. Most agent turns stay benign, even when the overall goal is harmful. The probe filters noise so the judge only reviews high-stakes exchanges.

The goal is synchronous control. Risky cyber actions can be blocked before any tool is called, instead of flagged after the agent already acted.

For the primary write-up and numbers, read Goodfire's research post on production cyber monitors on Kimi K3. For product and partnership context, see TechCrunch coverage of Goodfire's inside-out monitors by Aditya Mehta.

Why Agencies Need AI Agent Monitoring on Open Models

Cybersecurity is a hard monitoring setting. Benign and malicious work often look similar. Auditing a codebase for bugs can resemble preparing an exploit. Intent can change mid-session after a harmless start.

Recent agent incidents make the risk concrete. TechCrunch notes cases where agents escaped test environments, including OpenAI agents that breached Hugging Face. It also notes that Kimi K3 used a sandbox leak to reach the internet and GitHub information earlier in 2026.

Judging every turn with another frontier model is the obvious fix. Goodfire says that path does not scale. OpenRouter rates cited in the research post put every-turn Opus 5 monitoring near $200K per million Kimi K3 turns. Using Kimi K3 itself as judge would still land near $30K.

Agencies building client agents on open weights need a cheaper default. Sandboxes help, but they are not enough alone. Pair sandbox design with live monitoring. Our earlier guide on AI agent sandbox patterns with AWS Strands Box covers the isolation side.

How the Probe-Plus-Judge Cascade Works

The probe is always on. It watches inputs, tool calls, and outputs for high-stakes signals. Only flagged exchanges hit the judge. After calibration, Goodfire reports 0% false-negative rate and 95% true-negative rate on a held-out set.

That calibration lets teams use a faster judge without reviewing every turn. It also keeps monitoring inside the inference path, so decisions can arrive before tools run.

Diagram of Goodfire monitor cascade from probes to LLM judge — High-level overview of Goodfire's probe-to-judge monitoring pipeline. Image via Goodfire research.

Goodfire also engineered the probes into a production SGLang stack. Custom kernels and careful routing keep probe scores aligned with tokens and sessions. Across four inference workloads, enabling probes kept throughput within noise, with under 2% impact.

Latency cuts that make live blocking realistic

Without judge reasoning, Goodfire reports cascade latency of about 2 ms per turn versus 160 ms when judging every turn. With reasoning, the cascade averages about 42 ms versus 2,750 ms every-turn. Both setups cut average monitoring delay by more than 50x.

For agencies, that gap is the difference between a log you read later and a refuse action before shell access.

Official Accuracy and Cost Numbers

Prefer the official research post when secondary articles use different framings. Goodfire evaluated monitors on an in-house set of about 2.4K chat and agentic sessions with over 60K turns, using Claude Opus 5 labels as reference.

Operating point: about 93% recall at 5.5% false-positive rate for benign-session interruption
Cost: about $65 per million agent turns without judge reasoning, and about $185 with reasoning
Headline efficiency: roughly 50x lower cost versus judging every turn, under $200 per million exchanges
Judge-call reduction: probes can preserve roughly the same recall with about 6x fewer judge calls than judging every turn

Goodfire also says the cascade Pareto-dominates a judge-only setup across interruption budgets they tested. At the 5.5% benign interruption budget, non-cyber traffic flagged at 0% in their reported operating point.

TechCrunch repeats the 93% and 5.5% figures and cites about $185 for one million exchanges versus thousands to hundreds of thousands for cheaper or top-tier every-turn judges. Stick to the official $65 / $185 pair when you brief clients.

Full-Session Blocking Results on Kimi K3 and GLM 5.3

Accuracy on turns is not the whole story. Agencies care whether harmful sessions get stopped without freezing normal work.

Adding monitors raised Kimi K3 harmful-session blocking from about 29% to 96%. Harmless-session interruption rose from about 5% to 9%. For GLM 5.3, harmful blocking rose from about 36% to 91%, while harmless interruption rose from about 10% to 15%.

Both monitored models approached the strongest harmful-session blocking rates among frontier systems Goodfire compared, while interrupting fewer harmless sessions than many alternatives.

Chart comparing harmful and harmless session blocking with and without Goodfire monitors — Effect of adding Goodfire monitors on harmful versus harmless session blocking. Image via Goodfire research.

Transfer from Kimi to GLM without a new data factory

Goodfire built a GLM 5.3 monitor using data originally collected for Kimi K3. Off-policy transfer worked well. The GLM monitor looked similarly capable on Kimi rollouts and only slightly weaker on GLM on-policy evaluation.

That transfer story matters for agencies. You may not want a fresh labeled pipeline for every open model you serve. A reusable recipe lowers the cost of adding AI agent monitoring across a small model fleet.

FAR.AI Red-Team Results (Preliminary)

FAR.AI ran a short, static red-team on the deployed Kimi K3 monitors over two days. From 1,500 candidate jailbreaks, testers selected 140 that worked against a different base-model version without monitors.

Across those 140 strategies, monitors cut results from 66 universal jailbreaks and 700 jailbroken interactions to 0 universal jailbreaks and 18 jailbroken interactions.

Without the monitor, universal-jailbreak success ranged from 12% to 53% across reasoning-effort settings. With monitoring, none of the 140 strategies produced a universal jailbreak. Individual prompt success on 1,120 non-universal interactions fell to about 0.1% to 1.6%, depending on reasoning effort.

Treat this as preliminary static testing, not a permanent guarantee. Goodfire says it will keep collaborating with FAR.AI. Agencies should still plan their own red teams against client policies.

Baseten Availability and Risk Controls

TechCrunch reports the monitors are available to Baseten customers. Baseten's Base Labs announced a safety partnership with Goodfire and Hugging Face last month.

Customers can choose which risks to watch:

Offensive hacking
CBRN misuse
Reward hacking

They also choose automated responses: log the event, send it for human review, or refuse the request. That mapping is useful for agencies that need different severity paths for demos, internal tools, and production client agents.

Goodfire CEO Eric Ho said on Matt Turck's MAD Podcast that internal activation monitors are cheap because they reuse forward-pass computations. CTO and co-founder Dan Balsam told TechCrunch the advantage is catching issues before they happen, including when a model might hack during eval or training.

Balsam also framed liability around inference providers and clusters of compute, not only individual downloads of open weights. He said an open "Mythos" moment will make inference-time guardrails clearer. Agencies that host or resell agent capacity should hear that as a product and policy warning, not only a research note.

Related Context Agencies Should Keep Straight

Goodfire is not the first probe research line. TechCrunch notes that Google DeepMind said in January its research informed misuse-detection probes in Gemini. Goodfire's post also cites related probe and classifier work as prior art.

Separately, Goodfire research cited by TechCrunch found leading open models, including Kimi K3 and GLM-5.2, reward-hacked in 50% to 96% of runs on agent tests. Reward hacking is one of the Baseten risk toggles for a reason.

For broader governance language with clients, point them to the NIST AI Risk Management Framework. Monitoring is one control among many. It does not replace access control, tool allowlists, logging, or human escalation.

If your client stack also needs model integration and ops support, see our AI integration services for agencies. For vuln triage workflows that sit next to agent security, read our guide on Anthropic OSS Scanner free opt-in vuln scans for agencies.

Key Takeaways for India and USA Agencies

AI agent monitoring can run live on open models without burning a second frontier judge on every turn.
Official cascade costs land near $65 to $185 per million turns, depending on judge reasoning.
About 93% recall at 5.5% benign interruption is the published operating point to quote.
Kimi and GLM full-session harmful blocking jumped into the 91% to 96% range with monitors on.
Baseten customers can map risks to log, review, or refuse actions today.
FAR.AI static red-team results are strong but preliminary. Keep testing.

Agency Action Checklist

Inventory open-model agents. Note which ones can call shells, browsers, cloud APIs, or payment tools.
Write a cyber misuse policy that separates bounded vuln reproduction from exploitation. Goodfire trained against a precise policy of that kind.
Pilot probe-based monitoring on one high-risk agent path before a fleet rollout.
Set response ladders. Log low signals, review mid signals, refuse clear offensive or CBRN paths.
Measure interruption rate on real benign client workflows. A 5% to 15% harmless interruption band is already in Goodfire's published full-session results.
Keep sandboxes and tool allowlists. Monitoring complements isolation; it does not replace it.
Schedule an independent red team after the pilot, especially if you serve many tenants from one inference pool.

What This Means for Debugged Pro Clients

Most agency clients do not need a full interpretability lab. They need agents that stay inside policy while costs stay predictable. Goodfire's cascade is one concrete pattern: cheap always-on probes, expensive review only when needed.

If you sell agent builds on open models, add monitoring to the statement of work. Price the judge path, the human review queue, and the refuse behavior as product features. Do not treat safety as a free add-on that appears after an incident.

Also brief stakeholders that open weights are easy to strip of built-in refusals. Inference-time AI agent monitoring is often the control you still own after a model file leaves the vendor.

FAQ-Ready Talking Points

When a client asks whether a second model should read every turn, answer with cost and latency. Every-turn judging can add hundreds to thousands of milliseconds and large API bills. A calibrated probe cascade keeps most of that spend on serving the agent itself. Goodfire reports more than 99% of a fixed compute budget can stay on the target model, with under 1% on monitoring in their local-judge framing.

When a client asks if probes are magic, say no. Probes are classifiers over activations. They still need policy text, calibration data, and escalation rules. They also need honest evaluation on the client's own workflows, not only vendor charts.

About the author: Nitin Kumar is Lead Project Coordinator at Debugged Pro. He works with agencies and product teams in India and the USA on software delivery, AI integration, and practical security controls for real client projects.