AI vulnerability management sounds simple on paper. You point a model at your code, it finds the bugs and you fix them. Anyone who has tried it knows the real problem. The model finds too much, and some of what it finds is not there at all.

On October 7, 2026, three AWS security engineers shared how they deal with that problem. Nidhi Ramakant, Ievgeniia Ieromenko and Justin Kontny wrote a two-part guide, starting with Building your AI vulnerability harness. It is one of the most practical write-ups I have read on using AI for security triage. Below, I break it down in plain words for agencies in India and the USA.

Why AI Vulnerability Management Needs a New Approach

Security scanners now produce findings faster than people can read them. AWS puts it plainly in Part 1. Developers ship more code with more dependencies, so the pile of possible issues keeps growing.

Most of those findings will never be exploited. The few that matter need to reach an engineer quickly, with enough proof to act on. The hard part is separating signal from noise.

The volume is not slowing down. In a Security Boulevard report on AWS's Linux kernel team, Steven J. Vaughan-Nichols notes some striking numbers from a talk at the Linux Plumbers Conference in Prague:

There were 7,178 Linux kernel CVE reports so far in 2026.
526 of them were rated Critical and 2,754 were rated High.
Disclosures jumped from about two per week before 2024 to 16 per day afterward.
On August 15, 2026 alone, 848 kernel CVEs were recorded.

No team can read all of that by hand. That is why AI is moving into the triage step. The question is how to use it without drowning in false alarms.

Title card for the AWS Security Blog post Building your AI vulnerability harness, Part 1 — AWS shared its AI vulnerability harness in a two-part series on October 7, 2026. Image: AWS Security Blog.

The Three Layers of the AWS Harness

AWS describes a test harness as a framework that feeds a system controlled inputs and checks the results. Here, each candidate finding gets a test, the test runs in a safe setting and the evidence is recorded.

One idea stood out to me. AWS does not "run a model against the codebase." The code is context handed to a model with a specific prompt. That shift makes you think hard about what the model sees.

The pipeline then filters findings through three layers. Each layer asks for a different kind of proof.

Layer 1: Multi-scanner agreement

Run several scanners on the same code, independently. When two or more flag the same issue, confidence goes up. When only one tool reports it, the finding gets extra scrutiny.

This step is cheap and uses tools you likely already have.

Layer 2: Structural verification

AI scanners explain how they think a bug works. Data comes in here, passes through there and causes harm at the end. Those stories can be wrong.

So the pipeline checks each claim against the real code. Does the file exist? Does the function exist? Can data really flow along that path? If not, the finding is rejected. AWS uses the code's abstract syntax tree, a structured map of functions and data flows, to run these checks.

Layer 3: Deployment context

A bug does not live alone. It sits inside an app that may already have protection, such as WAF rules, network isolation, login checks and input validation.

This layer reads your infrastructure as code, such as CloudFormation or Terraform files, along with the app code. It then asks whether an attacker could get past the controls in place. AWS treats control strength as a scale, not a yes or no box.

Four Principles Worth Copying

AWS says four lessons came out of building this pipeline. They apply to any team, on any cloud.

Filter in stages with stronger proof each time. Plausible first, then structurally real, then exploitable in context. Three focused layers catch more errors than one big pass.
Infrastructure changes priority. A critical bug behind strong controls may matter less than a medium bug exposed to the internet.
Agreement beats self-confidence. No single scanner or model catches everything. Independent tools agreeing is a stronger signal than one tool's score.
Verify AI claims before a human sees them. Confident but wrong findings wear down trust fast.

The Steering File: Where AI Vulnerability Management Gets Consistent

Part 2 of the series, Configuring your AI vulnerability harness, covers the instructions that drive the model. AWS calls this a steering file.

A steering file is a set of rules an AI coding assistant loads at the start of every session. The name changes by tool:

Kiro reads files from a .kiro/steering/ folder.
Claude Code uses CLAUDE.md.
Cursor uses .cursorrules.
GitHub Copilot uses .github/copilot-instructions.md.

The authors are honest about how they got here. Early versions produced findings that sounded sure of themselves but fell apart on review. The model reported a SQL injection in a function that did not exist. It also ignored a WAF that was blocking the attack. Their summary is blunt: the fix was not a better model, it was better steering.

Rule one: never trust the model's own confidence

The most important instruction tells the model to check four things before reporting. Do the files exist? Do the functions exist? Does the dataflow back the claim? Is there a call path?

This rule made a big difference. About 30% of unsteered findings pointed to code that did not exist in the target repository. With the rule in place, AWS saw zero made-up paths in its testing.

Bar chart showing 34 percent of high-confidence AI findings were exploitable versus 89 percent of findings with confirmed dataflow and no sanitization — Findings backed by structural evidence were far more likely to be real. Data: AWS Security Blog. Graphic: Debugged Pro.

Rule two: score evidence, not feelings

The steering file swaps the model's gut feeling for a formula built from yes or no checks. Confirmed dataflow carries the most weight at 0.30. Missing sanitization adds 0.25, a public entry point 0.20, a confirmed call graph 0.15 and a matching sink type 0.10.

The results back this up. In AWS testing, findings with confirmed dataflow and no sanitization were exploitable 89% of the time. Findings the model rated HIGH without structural proof were exploitable only 34% of the time. AWS says your own weights should come from your own test data.

Rule three: count the controls you already have

Controls are mapped to specific bug types. For example, a WAF with SQL injection rules in blocking mode counts as a strong control against SQL injection. It does nothing against a deserialization attack.

Controls stack, but there is a floor of 0.15, so no mix of controls drops the risk to zero. AWS also learned that login checks are different from attack blocking. A command injection behind IAM sign-in is still a command injection. Fewer people can reach it, but the damage is the same.

Rule four: add threat intelligence

The file boosts a finding when real-world signals show active risk. It uses the CISA Known Exploited Vulnerabilities catalog and the Exploit Prediction Scoring System (EPSS):

Signal	Score boost
Actively exploited (CISA KEV)	+0.30
Linked to ransomware (CISA KEV)	+0.15
EPSS score of 0.5 or more	+0.20
Public proof of concept exists	+0.10

The total boost is capped at 0.50. AWS keeps urgency and severity apart on purpose. A medium bug that ransomware groups are using today needs a faster fix than a critical bug nobody can easily reach.

Rule five: turn scores into clear actions

Developers need a decision, not a number. The file sorts findings into four tiers:

P0: score of 0.8 or more with no effective blocking control. Fix now.
P1: score of 0.6 or more, or active threat intel. Fix this sprint.
P2: score of 0.4 or more with partial evidence. Look into it when there is time.
P3: score under 0.4 or fully mitigated. Accept the risk or add it to the backlog.

AWS calls one rule its most controversial: do not file P3 findings as security issues. I agree with it. Ten confirmed tickets get more attention than two hundred maybes.

What AWS's Own Tests Showed

AWS tried the steering file on a test app with ten known bugs. Eight were exploitable and two were blocked by infrastructure controls.

Without the steering file, the model found all ten and ranked the two mitigated ones lower. But it set severity by gut feel and did not verify any of its claims.
With the steering file, it found nine of ten. The one it skipped was a hardcoded credential in unit test code that was never called. It moved the two mitigated bugs to P3 with reasons, showed its math and checked every dataflow claim.

This was a small, purpose-built test, so treat it as a signal, not a benchmark.

Title card for the AWS Security Blog post Configuring your AI vulnerability harness, Part 2: The steering file — Part 2 explains the steering file AWS released on GitHub. Image: AWS Security Blog.

The Same Idea at Linux Kernel Scale

The Security Boulevard piece shows a separate AWS team using a similar pattern. The engineers behind AWS EC2's custom kernel, Dylan Johnson and Justinien Bouron, explained it in Prague.

First, a plain automated filter checks each CVE record against their kernel versions and compiled files. That removes about 70% of CVEs in their setup. An AI agent then reviews the remaining 30% in about 3.4 seconds per CVE, down from minutes. People review every result.

"We're really using it as a very fancy classifier," Johnson said during the Q&A. Bouron informally put the rate of problem decisions at roughly 5% to 10%. That is exactly why humans stay in the loop.

What This Means for Agencies in India and the USA

At Debugged Pro, we see the same pressure on client projects every week. Teams ship faster with AI help, so security debt builds up faster too. Clients in both countries now ask how we check code before release.

This method gives agencies a clear, explainable answer. A few practical gains:

Less alert fatigue. Developers see a short list of proven issues instead of a wall of warnings.
Better client reports. Each finding comes with evidence and a reason for its priority.
Repeatable results. The same rules apply on every project, whoever runs the scan.
Lower cost. AWS says the steering file needs no new services or procurement process.

Speed matters here too. Indian teams must report certain cyber incidents to CERT-In within six hours. US clients often have strict patch timelines in their contracts. Knowing which bug to fix first saves real time in both cases. Our guide on patch management lessons from the FBI and Accenture breach covers the fixing side in more detail.

A Simple AI Vulnerability Management Rollout Plan

AWS suggests building the harness in stages, starting with what pays back fastest. Here is how I would adapt it for a small agency:

Map your code first. Build or use an AST index that lists entry points, sinks and login settings. No AI is needed yet.
Triage what you already scan. Take Semgrep or CodeQL output and run it through the structural check plus an AI exploitability review.
Add the steering file. Grab it from the aws-samples repository on GitHub and adjust the weights to your stack.
Feed in infrastructure. Parse your Terraform or CloudFormation so the model knows which controls protect each service.
Test live last. Run proofs of concept only on pre-production, and only for findings above your threshold.
Tune it monthly. Check results against known true and false positives, then update the weights.

Know the limits before you start. The harness does not fix code. It cannot see a WAF rule someone switched off last week, and it cannot query live threat feeds on its own. Recent incidents like the Atlassian vulnerability CVE-2026-21589 show why live threat data still matters.

Need help setting up a review process like this? Our software development consulting team helps businesses in India and the USA ship secure code.

Key Takeaways

AI vulnerability management works best when every finding must prove itself with evidence.
AWS's harness filters findings through scanner agreement, code structure checks and deployment context.
About 30% of unsteered AI findings pointed to code that did not exist.
Findings with confirmed dataflow were exploitable 89% of the time, against 34% for unverified HIGH ratings.
The steering file is public on GitHub and works beyond AWS tools.
Human review stays essential, and AWS's kernel team calls it the new bottleneck.

Frequently Asked Questions

What is AI vulnerability management?

It is the use of AI models to help find, check and rank security flaws in code and systems. The AI reads scanner output and code, then helps decide which findings are real and which to fix first. People still make the final call.

What did AWS release in October 2026?

On October 7, 2026, the AWS Security Blog published a two-part guide to an AI vulnerability harness. Part 1 explains a three-layer pipeline. Part 2 explains a steering file, which AWS shared in a public GitHub repository for teams to use and adapt.

Do I need AWS or Kiro to use the steering file?

No. AWS says the method is not tied to its tools. The repository also has a CLAUDE.md version for Claude Code. The same ideas work in instruction files for Cursor, GitHub Copilot and others.

Can AI replace a human security reviewer?

Not today. AWS says even its top priority findings still need an engineer's review before code changes. Its own Linux kernel team keeps humans in the loop and says review is now the main bottleneck.

How can a small agency start?

Start with the scanners you already run, such as Semgrep or CodeQL. Add a check that every AI finding points to real files and functions. Then rank what is left using your infrastructure and threat data like the CISA KEV list.

About the Author

Nitin Kumar is the Lead Project Coordinator at Debugged Pro. He works with development teams that build software for clients in India and the USA. He follows the AI and security tools those teams use every day. He writes clear, practical guides so teams know what changed and what to do next.