"Sales uses this AI to summarize meeting minutes, development pastes code into a different AI, and marketing feeds customer lists into yet another tool. As a company, we have no grasp of who handed what to which AI." In companies where AI adoption has permeated frontline operations, shadow AI is quietly spreading. Frontline teams start using tools on their own initiative simply because they are convenient. That in itself is not a bad thing. The real problem is that the company cannot see what is happening. Confidential information is handed over to external AIs, usage costs swell across departments, and when an issue finally arises, the data paths cannot be traced.
To address this challenge, InfoQ's Governing AI in the Cloud: A Practical Guide for Architects organizes practical measures such as shadow AI discovery, data classification, IAM-based access control, and policy as code. The core takeaway is straightforward: govern AI by making it visible, not by banning it. If you prohibit it, shadow AI will simply be driven further underground.
The starting point of governance is discovery, not prohibition
When people hear "governance," they tend to think of usage restrictions, but that reverses the proper order. Unless you first visualize what is currently being used, you can neither restrict nor evaluate it. When designing AI governance in custom development projects, we build it up in the following order:
| Stage | Scope | Objective |
|---|---|---|
| Discovery | Take inventory of AI usage from network logs, billing data, and SaaS integrations | Map out shadow AI |
| Category | Classify which data flows to which AI based on sensitivity | Determine priorities for what must be protected |
| Control | Control who can use what via IAM and gateways | Tighten only high-risk pathways |
| Operations | Continuously monitor usage status and costs | Detect runaways and policy deviations early |
If you skip the initial discovery step and start straight with control, you will merely provoke pushback from frontline teams, and workarounds will ultimately be found. You can safely consider that creating a map of the current state accounts for 80% of governance. The structural reasons why AI implementations fail to deliver results were also examined in The GenAI Divide: The Trap Behind 95% of Failures (GH Media). Visualization is effective not only for security, but equally in terms of return on investment.
Gateway-based control to unify the entry point
Once discovery and classification are complete, the next step is control. An effective approach here is an architecture that consolidates all outbound traffic from within the company to external AI into a single gateway. Instead of each application and department calling AI services directly, placing a gateway in between makes it possible to achieve the following in one place:
- Visibility: Gain an overall view of which team used which model and how much
- Cost control: Set usage caps per department or application to halt runaway billing
- Access control: Apply additional authentication, logging, and masking exclusively to pathways handling confidential data
- Model swapping: Switch models in the future simply by updating entry-point configurations
Cost controls can also be introduced at the individual tool level. Specific methods for stopping runaway AI costs using usage limits are covered in Configuring Usage Limits with Cloudflare AI Gateway (GH Media). The advantage of the gateway pattern is that it lets you capture both security and cost governance at the very same entry point.
Turning policy into code rather than documentation
A classic way governance becomes a hollow formality is when rules are simply written down in PDFs or spreadsheets and gradually drift away from actual operations. What prevents this is policy as code—the concept of enforcing rules like "confidential data must not be sent to external AI" and "only this specific model can be invoked from production environments" systematically through automated mechanisms rather than relying on manual operation.
At an actual client of ours (whose name we will keep confidential), masking and audit logging were made mandatory only for AI pathways handling internal company data, and those conditions were integrated as checks within the deployment pipeline. Adhering to the rules ceased to be a matter of individual discretion; instead, deployments simply cannot proceed unless the checks pass, making it far harder for someone to later realize they were inadvertently sharing confidential data. This architecture of safely exposing internal AI with access controls also shares principles with Protecting Custom MCP Servers with OAuth 2.1 (GH Media).
Starting by mapping the current state of AI usage
AI governance does not start by writing corporate regulations; it begins by making visible who is handing what to which AI right now. Once that is clear, you can distinguish which pathways need to be tightened and which can be left alone, avoiding bringing frontline operations to a halt through heavy-handed prohibitions. In our custom development work, we provide step-by-step support ranging from this initial discovery and classification to gateway consolidation and systematic policy enforcement.
