Note
How Autonomous Pentest Tools Avoid False Positives
How do autonomous penetration-testing tools avoid reporting false positives?
By refusing to report anything they can't prove. A finding should only reach the report when the scanner has captured physical evidence of exploitation — a database error signature, an unencoded reflection of a unique payload, real command output, a known file signature, or a template expression that actually evaluated — and anything without that evidence should be discarded, not downgraded to 'possible'.
Why 'possible' findings are the real problem
A report with forty 'possible' issues trains a team to ignore the report. The one real vulnerability gets ignored with the rest. So the useful design goal for an automated scanner isn't maximum coverage of suspicions — it's a report where every line is something a developer can reproduce.
Nythera, an open-source scanner Shreyansh built for in-house web applications, is designed around that rule: every active check carries its own proof criterion, and a check that can't meet it produces nothing.
What counts as evidence, per vulnerability class
- SQL injection — a database error signature in the response, or boolean inference: a true condition and a false condition produce consistently different responses.
- Reflected XSS — a unique payload comes back unencoded in the response body, so it would execute.
- OS command injection — the output of an injected command appears in the response.
- Path traversal / LFI — the response contains the signature of a known operating-system file.
- Server-side template injection — a template expression is evaluated (a maths expression comes back as its result), then confirmed with a second expression.
- Open redirect — the Location header actually points at the attacker-supplied host.
- Broken access control — the same request as an authenticated and an anonymous user returns the protected data both times.
Discard, don't downgrade
The tempting compromise is to keep unconfirmed hits as 'low confidence'. That quietly reintroduces the noise. If a payload was reflected but encoded, there's no XSS; if an error appeared but isn't a database error, there's no SQL injection. The finding is dropped.
This also makes the CI integration trustworthy: Nythera's headless mode exits with a failure code only when a finding at or above a chosen severity exists, so a red build always means a proven issue.
Where an LLM agent fits
An LLM agent can reason about a target and suggest further probes, and Nythera includes one as an optional phase. But it sits on top of the validated checks, not in place of them: anything the agent proposes still has to produce evidence before it's reported.
Should internal-network scanning be off by default?
Yes. Scanning localhost and private address ranges is what makes a tool useful for in-house testing — and what makes it dangerous when it's pointed at the wrong place. In Nythera internal mode is off by default and has to be enabled explicitly; requests stay locked to the target host with include/exclude rules and a rate limiter; private hosts are refused unless internal mode is on; and every scan requires confirming authorization first.