An AI planning meeting ends abruptly when a stakeholder notes, "That data cannot leave our premises." The discussion moves to the next agenda item, never to be reopened. Six months later, the exact same deliberation restarts from square one.
What makes this kind of stall troublesome is that no one is at fault. The decision that data cannot leave is often correct, and no single person bears the responsibility of proposing alternatives. Without reaching a conclusion, the initiative itself simply fades away.
Three distinct meanings of "cannot leave"
Categorizing these upfront changes what actions you can take next.
The first category is data bound by contract or regulation. Non-disclosure agreements with partners may forbid third-party sharing, or industry regulations may impose restrictions. This is not something to bypass with technology; it requires assessing whether conditions can be met.
The second category is data governed by internal rules. Company policies such as "Confidentiality Level A data must not be sent to external services." Because your organization established the rule, it can be revisited under the right conditions.
The third category is data kept inside out of vague apprehension. It appears in neither policies nor contracts, yet teams hold back due to a lack of precedent. In practice, this third case is the most common. And this third case cannot be resolved through technical discussions.
Verify first which category applies to your stalled project. Bringing a technical solution to the third category will not sway decision-makers.
There used to be only two choices
When contracts or policies pose obstacles but business necessity remains, organizations previously had effectively only two avenues.
One was sending data to the cloud under compliant terms: restricting data usage via contracts and ensuring settings prevent data from being used for training before transmission. The other was running workloads locally or on-premises: avoiding external data transfer by providing computing resources in-house.
While the latter has become substantially more feasible recently, the requirement for infrastructure and operations remains unchanged. As outlined in The Reality of On-Device AI regarding hardware demands, a significant distance separates "functioning" from "viable in daily business."
HEIR as a third path
In August 2026, Google open-sourced HEIR. It is a compiler and toolchain that compiles trained AI model Python code to enable direct inference input from encrypted data without prior decryption.
Underpinning it is fully homomorphic encryption (FHE). Computations occur while data remains encrypted, and the output is returned in an encrypted state as well. The receiving party cannot inspect the contents of the data being processed at any point.
| Approach | Data location | Can processing party view contents? |
|---|---|---|
| Send to cloud | External service | Yes (contractually restricted) |
| Run locally / on-premises | In-house | In-house only |
| Compute while encrypted | Can be external | No |
When the third row becomes viable, the entire premise of "cannot leave" changes. What leaves the premises is ciphertext, not plaintext.
Not yet ready for current projects
Let us establish expectations upfront: this is not a technology to put into current-quarter proposals.
Fully homomorphic encryption is known to demand drastically higher computational resources compared to performing the same operations on plaintext as the price of computing while encrypted. While GPU acceleration research is advancing and bringing it closer to practical utility, running general-purpose generative AI via this approach is far from commonplace.
HEIR seeks to lower the barrier preceding that. Previously, entering this field required specialized cryptography expertise; using the compilation of trained model code as an entry point broadens who can work with it. Combined with its open-source release, this represents a shift where "more people can experiment," not that it is "ready for production."
Consequently, its current role is less of an immediate option and more material to inform today's investment decisions while factoring in choices a few years away. If your organization is on the verge of substantial investments in on-premises infrastructure, verifying progress in this direction is well worth the time.
Who holds the key changes everything
Separate from technological maturity, this paradigm requires resolving a core question from the outset: who holds the decryption key?
Computing while encrypted is meaningful only when the processing party does not possess the key. If the service provider handling the workload also holds custody of the key, they can decrypt at will, making it functionally equivalent to handing over plaintext. Explaining that data is "sent encrypted" alone cannot prove whether meaningful protection exists.
Conversely, your company must retain custody of the key. This introduces ongoing key management duties: secure storage, backups, and handoffs during personnel changes. Losing the key renders even your organization unable to read the encrypted data. This risk must be evaluated separately from data leaks as an availability concern.
When evaluating this approach in the future, whether your organization can bear this key management burden will serve as a decision criterion well before technical feasibility. While you need not answer this today, recognizing it as an issue provides a framework for evaluating vendor proposals.
What you can still do today
Whichever path you eventually choose, one task will be universally required: classifying which internal datasets truly cannot leave the premises.
Referring back to the three initial categories, only data matching the first and second categories requires technical intervention. In many workplaces, skipping this boundary leads teams to treat "all company data as confidential," halting every initiative as a result. The mechanism discussed in AI Classification and Labels in Google Drive serves as a practical starting point for classification.
This classification will carry over directly when cryptographic methods become practical. Without it, discussions will continue stalling on "cannot leave" regardless of incoming technologies. This is the work you can accomplish while waiting.
What to do next
First, recall one recent AI initiative that stalled and verify which category explained why data could not leave. If it was the third (lack of precedent), what you need is not cryptography, but an arrangement to run a small trial and establish precedent.
Next, begin classifying the data that cannot leave. There is no need to tackle the entire company at once; focusing only on the data needed for that stalled initiative is enough. Establishing boundaries for a single project lets you restart future deliberations from that point forward.
At GleamHub, we assist with designing AI architectures handling confidential data, organizing data classifications, and comparing on-premises versus cloud options through our development, AI, and automation consulting services. Because viable architectures depend on data characteristics and constraints, feel free to consult with us individually via Contact Us.
