On May 30, 2026, EY Canada published a cybersecurity report and most citations were hallucinated rose to the top of Hacker News. An investigation by GPTZero revealed that more than half of the citations (27 out of 51) in the annual cybersecurity report released by EY Canada (a Big Four firm) were AI hallucinations (non-existent research papers, URLs, and statistics). While EY initially responded by deleting the report, the issue was widely covered across business media as brand damage for an audit firm.
From the perspective of supporting AI adoption, content generation, and proposal materials for mid-sized enterprises through custom development, this signifies that organizations solely pursuing "writing faster with AI" have entered an era where lacking governance—namely, "publishing AI outputs without running them through an internal verification workflow"—causes catastrophic brand damage. Connecting this with the input-side governance in OpenAI Privacy Filter & Trusted Access AI Governance (GH Media), the test automation in Automating QA with Playwright × AI (GH Media), and the prompt architecture in Custom Anthropic XML Prompt Structure Development (GH Media), we organize the key considerations when assembling "AI deliverable QA governance" for custom development engagements. Note that "provenance and digital watermarking (SynthID / C2PA) to prove whether content is AI-generated" was addressed in Custom Development: Protecting AI Content Trust with SynthID and C2PA (GH Media); this article focuses on its counterpart: "how to verify the factual accuracy of outputs."
Why “AI deliverable QA governance is a turning point”
| Dimension | Conventional QA (centered on human writing) | QA in the AI era (centered on AI generation) |
|---|---|---|
| Verification targets | Text / numbers / logic | + Citations / URLs / statistics / regulations |
| Types of errors | Typos / transcription errors / logical inconsistencies | + Hallucinations (non-existent sources) |
| Detection methods | Visual inspection by proofreaders | + LLM × secondary LLM × URL existence verification |
| Locus of responsibility | Authors | + Prompt designers + review infrastructure |
| Approval flows | Editorial → Legal | + AI fact-checking → Editorial → Legal |
| Post-publication auditing | Correction notices | + Audit logs + prompt reproducibility |
| Brand risk | Minor | Severe (reverberates across the industry) |
| Target deliverables | Articles / reports | + Proposals / legal documents / IR / audit materials |
In short, AI deliverable QA demands a structural shift from “proofreading human writing” to “protecting AI-generated output through facts and auditability.”
Three structural changes beneficial to custom development projects
Structure 1: From “manual proofreading” to “LLM × verification LLM × external APIs”
Even if marketing, PR, and sales planning teams in mid-market companies have adopted workflows to write faster using ChatGPT, fact-checking still relies on manual proofreading. In our custom engagements, we incorporate a secondary verification LLM distinct from the drafting LLM, URL existence checks, and cross-referencing against academic databases and official statistical APIs directly into the pre-publication pipeline. This represents the content counterpart to the automated QA using AI covered in Automating QA with Playwright and AI (GH Media).
Structure 2: From “author responsibility” to “prompt + audit log accountability”
The EY incident illustrated that accountability can no longer rest solely on “the author used AI.” In our custom solutions, we ensure that prompts, responses, verification results, and approvers are encrypted and retained for set retention periods, making outputs fully reproducible when issues arise. This is the output audit counterpart to the input governance discussed in OpenAI Privacy Filter Governance (GH Media).
Structure 3: From “marketing and PR only” to “deliverables across all departments”
QA governance is essential not only for articles, but also for proposals, estimates, legal documents, and IR materials. In our engagements, we establish variable verification rigors based on departmental risk levels, applying maximum rigor to legal, IR, and audit materials, while maintaining lightweight workflows for internal study materials. This represents the approval flow counterpart to the structured prompts covered in Anthropic XML Prompt Structure for Clients (GH Media).
The 5 phases of assembling “AI deliverable QA governance”
Phase 1: Current state assessment (2 weeks)
- Inventory of AI-utilizing departments and tools
- Classification of public and externally submitted deliverables
- History of past near-misses and corrections
- Existing approval flows / legal reviews
- Brand risk scoring
- Departmental verification rigor matrix
Phase 2: Governance design (2–3 weeks)
- Deliverable risk levels (Red / Yellow / Green)
- Verification workflows (automated / semi-automated / manual)
- Mandatory checklist items (citations / statistics / regulations / proper nouns)
- Audit log retention + access controls
- Approvers and exception handling procedures
- Governance KPIs
Phase 3: Technical implementation (3–5 weeks)
- Verification LLM pipeline (OpenAI / Anthropic / Bedrock)
- URL existence checks (HEAD requests + Wayback)
- Academic / statistical API integrations (Semantic Scholar / Crossref / e-Stat)
- Legal database integrations (e-Gov / Westlaw / LexisNexis, etc.)
- Workflow foundations (Notion / Asana / Jira / in-house)
- Dashboards (Grafana / Datadog / Looker)
Phase 4: Pilot rollout (2–3 weeks)
- Commencing operations in 1–2 departments
- Verification pass rate measurement
- False positive and false negative triage
- Training + runbook distribution
- KPI measurement + refinement
Phase 5: Monthly operational reviews (ongoing)
- Verification pass / rejection statistics
- Addition of novel error patterns
- Evaluation of LLM vendor transitions
- Brand risk incident reviews
- Semi-annual standards revision
Standard technology stack set for custom development
| Layer | Recommended technology | Alternative |
|---|---|---|
| Drafting LLM | GPT-5.5 / Claude Opus 4.X / Gemini 3.5 | DeepSeek / Llama |
| Verification LLM | Higher-tier model from an alternative vendor | Different generation from the same vendor |
| URL verification | In-house + Wayback Machine API | LinkChecker |
| Academic / statistics | Semantic Scholar / Crossref / e-Stat | OpenAlex |
| Legal databases | e-Gov / Daiichi Hoki / Westlaw | LexisNexis |
| Workflow | Notion / Asana / Linear / in-house | Jira |
| Audit Logging | OpenTelemetry + S3 / GCS + encryption | Datadog Logs |
| SIEM | Microsoft Sentinel / Splunk | Sumo Logic |
Which projects need this and which do not
| Projects requiring this | Projects not requiring this |
|---|---|
| Generating externally published content with AI | Internal memos only |
| Using AI for proposals / IR / audit materials | Chit-chat bot |
| Audit requirements (ISO 27001 / SOC 2 / J-SOX) | Not subject to auditing |
| Industries with significant brand damage risks | Minimal impact scope |
| Drafting legal / regulatory compliance documents with AI | Unregulated areas |
Six clauses to include in client contracts
| Clause | Details | What the client should verify |
|---|---|---|
| Target deliverables | Department-by-department coverage and exclusions | Handling out-of-scope items |
| Risk level | Red / yellow / green judgment criteria | Business-specific factors |
| Mandatory verification items | Citations / statistics / laws and regulations / proper nouns | Industry-specific checks |
| Audit log retention | Retention period + encryption + access control | Regulatory requirements |
| Handover Upon Project Completion | Prompts / verification rules / logs | Internal operational continuity |
| Incident operations | Post-publication retractions and PR coordination | Escalation thresholds |
Client ROI estimate (assuming 400 monthly deliverables across 6 departments)
| Item | Current state (manual proofreading only) | After implementing QA governance | Difference |
|---|---|---|---|
| Monthly hallucination oversight count | 12 incidents | 1 incident | -11 incidents |
| Post-publication corrections / retractions per year | 8 cases | 1 incident | -7 incidents |
| Proofreading workload (monthly) | 200 hours | 60 hours | -140 hours |
| Audit support workload (annual) | 240 hours | 80 hours | -160 hours |
| Brand damage risk (annual) | Projected 1 major + 3 moderate incidents | Projected 1 moderate incident | -80% risk |
| Annual benefit | — | — | Workload reduction equivalent to approximately 18 million yen + brand protection |
Equivalent to an annual workload reduction of 14 million yen + correction handling cost savings at an hourly rate of 8,000 yen. When evaluating return on investment, be sure to include brand risk avoidance (preventing corrections, retractions, and PR responses) as the greatest benefit alongside these workload cost savings in your assessment criteria.
Five common pitfalls
Pitfall 1: Using the same LLM for drafting and verification
Having the same model self-verify frequently leads to cases where the same hallucination is judged as "correct." As a golden rule, verify using a higher-tier model from a different vendor.
Pitfall 2: Checking only HTTP 200 status for URLs
A URL existing and it substantiating the stated content are entirely different matters. Delve deeper with Wayback Machine / academic databases / content snippet matching.
Pitfall 3: Overcomplicating workflows
Applying the verification pipeline at maximum rigor to all deliverables leads to on-site teams creating workarounds. A design that adjusts rigor based on risk levels is essential.
Pitfall 4: Failing to capture audit logs
If you cannot reproduce "which prompt produced which response" when an issue occurs, root cause investigation and recurrence prevention become impossible. Operate with encryption + time-limited retention.
Pitfall 5: Failing to involve legal, IR, and PR
QA governance is not merely a technical challenge; it encompasses operations up to post-publication retractions, correction announcements, and stakeholder notifications. Involving legal, IR, and PR during the initial setup is mandatory.
90-day action plan
| Week | Action |
|---|---|
| Week 1〜2 | Inventory + risk-level design + near-miss collection |
| Week 3〜5 | Governance design + KPIs + legal / IR / PR alignment |
| Week 6〜10 | Verification pipeline construction + API integration + workflows |
| Week 11〜12 | Pilot department rollout + training + KPI measurement |
| Week 12 | Company-wide rollout + runbook preparation |
| Week 13 | Initial monthly review + ROI dashboard |
Summary — Moving from "writing faster with AI" to "governance that protects AI deliverables"
EY Canada's AI hallucination report symbolized an era where even a Big Four firm suffered catastrophic brand damage by neglecting AI deliverable governance. From the perspective of supporting AI adoption for mid-sized enterprises through custom development, "AI deliverable QA governance"—integrating drafting LLMs, verification LLMs, external APIs, workflows, and audit logs—will become an indispensable discussion going forward.
If you have concerns such as "citations in our AI-written proposal were fabricated," "we almost published hallucinated statistics in a press release," or "our pre-publication verification workflow for external content is inadequate," please feel free to reach out via our contact form. Because the scope of verification pipeline implementation and targeted departments vary greatly by enterprise, we provide individualized estimates after reviewing your current AI usage.
