Late at night, a vulnerability scanner triggers a CVSS 9.8 alert, prompting an urgent call to a vendor. The next morning, the vendor replies that "the reported function does not exist in the code for that version"—this scenario is now genuinely happening.
The response itself ends as a false alarm. The problem lies in the time wasted before realizing it was a false alarm, and the trust lost with staff woken in the middle of the night. Moreover, these wild goose chases will only increase.
Six "critical vulnerabilities" that were entirely fabricated
Security firm JFrog investigated six reported critical vulnerabilities concerning SQLite and published findings showing that every single one was non-existent (SQLite Critical CVEs or LLM Slop? — JFrog Security Research).
All were flagged as severe memory-related flaws, assigned scores as high as 9.8. Yet investigation revealed that the functions alleged to contain issues did not exist in the code of the specified versions. Running the attached proof-of-concept code produced no crashes. While the formatting of the reports was immaculate, the substance failed to align.
Further investigation revealed that a single newly created GitHub account had posted 55 vulnerability reports in a matter of days. Fifty-four of them were concluded to be fabrications. When submitted to generative AI detection tools, the text was classified as machine-generated.
Once published in public databases, treated identically to real ones
If these were merely posts on GitHub, ignoring them would suffice. What makes this genuinely troublesome is that they reached the NVD, the official US vulnerability database.
They were recorded as critical in the NVD and even enriched with information by CISA teams. And once listed there, scanners and defenders across the globe treat them as authentic. This is the pathway through which organizational vulnerability management tools receive alerts.
From the perspective of those receiving the alerts, the verification of authenticity appears to have already concluded. Few organizations operate under workflows built on the assumption that "critical vulnerabilities published in official public databases" should be second-guessed.
Four checks to verify authenticity
Fortunately, distinguishing real from fake does not demand specialized techniques. The methods JFrog used to uncover the fabrications can be applied directly in everyday operations.
| What to verify | If authentic | Common signs of a fake |
|---|---|---|
| Does the relevant function or code actually exist? | Exists in the source code of the specified version | Function name not found, or exists only in other versions |
| Does the proof-of-concept reproduce the issue? | The expected anomaly occurs | Completes execution without any issues |
| Reporter track record | Past reports and affiliations are traceable | A new account posting in bulk over a short timeframe |
| Upstream project reaction | Maintainers are verifying and fixing | No trace of discussion, or refuted |
Of these, the first two can be answered in dozens of minutes simply by asking your vendor. Just two questions: "Does the function in question exist?" and "Does the PoC reproduce?" Inserting this step before initiating emergency response stops the vast majority of false alarms.
The fourth check is easily overlooked, but highly effective. For a genuinely critical vulnerability, activity will inevitably show up in issues, commits, or mailing lists for that project. A situation where a vulnerability appears in public databases while the upstream repository remains completely quiet is itself a major red flag.
What to establish in maintenance contracts
As this category of alert multiplies, companies outsourcing maintenance face middle-of-the-night escalations while accountability for who evaluates the alert first remains unclear. Three points should be agreed upon:
Who performs initial triage? Does emergency response begin upon receiving the alert, or only after verifying factual existence? Without defining this, everyone scrambles whenever a scanner rings
How is the trigger for emergency response defined? Contracts triggered by "CVSS 9.0 or higher" will now fire even on fabricated reports. Updating terms to trigger on confirmed impact rather than score alone is a sensible revision
Handling false alarms. Investigating non-existent vulnerabilities still constitutes labor. Whether that is included in maintenance scope or billed as ad-hoc work must be clarified, or billing disputes will follow. Defining contractual scope was addressed in The era of AI finding and fixing vulnerabilities — What clients should establish in maintenance contracts
Having AI agents fix code is where danger peaks
There is one more operational reality that packs a punch: when remediation is delegated to AI agents.
Given a vulnerability in a non-existent function, an agent will dutifully attempt to resolve it. It searches for the relevant location, guesses a similar function when nothing matches, and applies a patch. Real code gets rewritten to fix a non-existent problem. Where a human engineer would stop and say "no such function exists," an agent does not stop.
If automated remediation is integrated, you must insert an existence check at the ingestion point for vulnerability data. Simply programmatically verifying whether target code actually exists before passing tasks to agents prevents this class of incident. The question of who reviews AI-written code was explored in Who inspects the security of AI-written code?.
The idea of waiting a set period rather than rushing dependency updates was introduced in Dependabot's default three-day pause, and this incident provides supporting rationale for that judgment. An operational model that reacts fastest to every alert also reacts fastest to fake alerts.
What to verify right now
Trace where an alert goes after your vulnerability management system generates it. If that "someone" in scanner → notification → someone is undefined, or if automated response workflows are triggered directly, that is your entry point into this problem.
Neglected vulnerabilities remain genuinely dangerous. As noted in Six months after React2Shell: CVSS 10.0 still active on abandoned sites, the risks of non-response are very real. What has changed is the necessity of a step to verify the alert itself.
If you want to structure vulnerability triage workflows into maintenance contracts or design verification stages prior to introducing automated fixes, GleamHub offers consultations on development, AI, and automation. Because the optimal setup varies depending on requirements, we provide customized quotes. Please reach out via Contact Us.
