An AI-Generated Bug Report Lands in Your Inbox. Now What?
Open source maintainers are being swamped with AI-generated security reports that mostly turn out to be nothing. The same thing is starting to land in business inboxes too, and most people have no way to tell a real issue from noise.
Open source maintainers have spent the last few months wading through a flood of vulnerability reports generated by AI tools. Most of it has turned out to be nothing: hallucinated bugs, misread code, findings that describe a problem that doesn't actually exist in the version being scanned. One well-known case ran for seven months before the maintainers worked out that almost none of the submitted "findings" held up. That's a lot of unpaid, unwanted work for people who were just trying to keep a project running.
If you run a business with a website, a customer portal or any bespoke software, the same thing is starting to happen to you. Bug bounty hunters have always existed, but now anyone can point an AI tool at your login page, your API, or your public GitHub repo and generate a report in minutes. Some of those reports will be genuine. A lot will read convincingly but be wrong. The problem is telling which is which when you're not the one who can read the code.
What these reports actually look like
A typical email will claim something like "SQL injection in your search endpoint" or "exposed admin panel", often with a screenshot, a proof-of-concept URL, and a request for a bounty or "responsible disclosure payment" before they'll share more detail. Some are entirely legitimate security researchers doing exactly what they say. Others are running an automated scanner, pasting the output through an AI tool to make it sound professional, and sending it to every business they can find a contact address for.
The language in both cases can be nearly identical. That's the whole problem. A well-formatted report with technical-sounding terms doesn't tell you whether there's a real hole in your system or whether the scanner flagged a field that was never vulnerable in the first place.
Why you can't just ignore them, and can't just pay them either
Ignoring every report is risky, because occasionally one of them is real and sitting on it is how a small issue turns into a breach. But paying out or scrambling to patch every claim is just as bad. It trains every opportunist who finds your contact page that you'll react to pressure, and it burns time chasing phantom bugs instead of running your business.
The honest answer is that you need someone to actually look at the claim against your real code and your real data, not the description in the email. That's not something a general IT helpdesk or a support ticket queue is well placed to do quickly, because they usually don't know the system well enough to check a specific claim in context.
What I'd actually check first
- Does the claim match a real endpoint? A lot of automated reports reference URLs or parameters that don't exist in the live system, or that were removed months ago.
- Can it be reproduced? A genuine vulnerability can usually be demonstrated cleanly. A hallucinated one tends to fall apart the moment you ask for a step-by-step.
- What's the actual exposure? Even a real finding might be low severity: a verbose error message, say, rather than a path to customer data. Severity decides whether this is urgent or a backlog item.
- Who's asking, and how? A researcher who shares detail upfront and asks for a disclosure window behaves very differently to someone demanding payment before they'll explain anything.
None of this takes long if you, or someone who built the system, can read the code directly. It takes a lot longer, and a lot more worry, if the first step is trying to find anyone who understands the system at all.
The real cost isn't the fix, it's the noise
Most businesses that come to me after one of these emails don't need a rebuild. They need someone to spend an hour checking whether the claim is real, and then either patch something small or write back explaining why it isn't an issue. The expensive part isn't the technical work. It's not knowing who to ask, panicking, and either overpaying a stranger or ignoring something that mattered.
This is one of the quieter arguments for having someone who actually knows your codebase on call, rather than treating security as a thing you deal with once a year during an audit. If I built your system, or I've worked in it before, I can usually tell you within the hour whether a report is worth acting on. That's a very different position to starting from scratch every time something lands in your inbox.
What to do if one turns up today
Don't click anything in the email and don't pay anything upfront. Ask the sender for a clear, reproducible description of the issue, including exactly what was tested and what the expected versus actual behaviour was. A genuine researcher will give you this without much pushback. Then get it checked against the actual code, not the summary in the email, before deciding whether it's urgent, low priority, or nothing at all.
If you don't currently have anyone who can do that check quickly, it's worth sorting out before the next one arrives rather than during it. You can read more about how I work with businesses on an ongoing basis, including the kind of quick turnaround checks that a report like this needs.