---
title: "How Autonomous Pentest Tools Avoid False Positives"
description: "Autonomous pentest tools avoid false positives by reporting only what they can prove. How Nythera validates SQLi, XSS, command injection and more."
url: https://www.shreyanshkumarsingh.com/notes/ai-pentest-agent-false-positives
author: Shreyansh Kumar Singh
updated: 2026-10-08
---

# How Autonomous Pentest Tools Avoid False Positives

By Shreyansh Kumar Singh · 2026-10-08

**How do autonomous penetration-testing tools avoid reporting false positives?**

By refusing to report anything they can't prove. A finding should only reach the report when the scanner has captured physical evidence of exploitation — a database error signature, an unencoded reflection of a unique payload, real command output, a known file signature, or a template expression that actually evaluated — and anything without that evidence should be discarded, not downgraded to 'possible'.

## Why 'possible' findings are the real problem

A report with forty 'possible' issues trains a team to ignore the report. The one real vulnerability gets ignored with the rest. So the useful design goal for an automated scanner isn't maximum coverage of suspicions — it's a report where every line is something a developer can reproduce.

Nythera, an open-source scanner Shreyansh built for in-house web applications, is designed around that rule: every active check carries its own proof criterion, and a check that can't meet it produces nothing.

## What counts as evidence, per vulnerability class

- SQL injection — a database error signature in the response, or boolean inference: a true condition and a false condition produce consistently different responses.
- Reflected XSS — a unique payload comes back unencoded in the response body, so it would execute.
- OS command injection — the output of an injected command appears in the response.
- Path traversal / LFI — the response contains the signature of a known operating-system file.
- Server-side template injection — a template expression is evaluated (a maths expression comes back as its result), then confirmed with a second expression.
- Open redirect — the Location header actually points at the attacker-supplied host.
- Broken access control — the same request as an authenticated and an anonymous user returns the protected data both times.

## Discard, don't downgrade

The tempting compromise is to keep unconfirmed hits as 'low confidence'. That quietly reintroduces the noise. If a payload was reflected but encoded, there's no XSS; if an error appeared but isn't a database error, there's no SQL injection. The finding is dropped.

This also makes the CI integration trustworthy: Nythera's headless mode exits with a failure code only when a finding at or above a chosen severity exists, so a red build always means a proven issue.

## Where an LLM agent fits

An LLM agent can reason about a target and suggest further probes, and Nythera includes one as an optional phase. But it sits on top of the validated checks, not in place of them: anything the agent proposes still has to produce evidence before it's reported.

## Should internal-network scanning be off by default?

Yes. Scanning localhost and private address ranges is what makes a tool useful for in-house testing — and what makes it dangerous when it's pointed at the wrong place. In Nythera internal mode is off by default and has to be enabled explicitly; requests stay locked to the target host with include/exclude rules and a rate limiter; private hosts are refused unless internal mode is on; and every scan requires confirming authorization first.


---

Shreyansh Kumar Singh — AI & Full-Stack Engineer, Pune, India. Contact: shreyanshkumarsingh208@gmail.com or https://www.shreyanshkumarsingh.com/contact (call and WhatsApp buttons). More: https://www.shreyanshkumarsingh.com/llms.txt · https://www.shreyanshkumarsingh.com/sitemap.xml