Case study · VERIFICATION · 2607.VER.08
HallogenAI: verifying bug fixes by evidence, not by commit
How can an AI agent verify that a bug is actually fixed rather than trusting the commit log?
HallogenAI re-tests each reported bug against the running application and marks it fixed only when the observed behavior, captured as evidence, says so. Eight specialized agents split the work — taking in the report, understanding it, automating the reproduction, collecting evidence and writing the verdict — and the governing rule is simple: never guess.
- Specialized agents
- 8
- Pipeline
- Intake → Understand → Automate → Evidence → Report
- Automation platforms
- 2
- Stack
- Python · FastAPI · Playwright · Pydantic v2 · SQLAlchemy · Groq
01The problem it solves
In most teams a bug is 'fixed' when someone pushes a commit and closes the ticket. Whether the behavior actually changed in the QA environment is assumed, not checked — and regressions slip through on exactly that assumption.
02What he built
A five-stage verification pipeline
Intake → Understand → Automate → Evidence → Report. Eight specialized agents divide the stages so each one does a narrow job well, instead of one general agent improvising the whole thing.
Real browsers, typed data
Playwright drives the application the way a tester would; FastAPI serves the system; Pydantic v2 and SQLAlchemy keep every report, step and verdict typed and stored; Groq provides the models. It works across two automation platforms.
03 — THE HARDEST DECISIONEvidence outranks the commit log
The tempting shortcut is to let an agent read the diff and decide the fix 'looks right'. HallogenAI deliberately ignores that signal. A bug is marked fixed only when its behavior in the QA environment — reproduced, observed and captured — shows the defect is gone. If the evidence can't be produced, the verdict isn't 'probably fixed'; it's not verified.
04The result
Bug status that reflects what the application actually does, with the evidence attached — the same 'prove it' standard Nythera applies to security findings, applied to QA.
RESEARCHResearch findings from this project
R-10Evidence outranks the commit log
A bug is never marked fixed because code changed or a commit exists — only because its actual behavior in the QA environment, captured as evidence, says so.
NOTESQuestions this project answers
STACKTech stack
- Python
- FastAPI
- Playwright
- Pydantic v2
- SQLAlchemy
- Groq