In May 2026, InfoQ released Presentation: Leadership in AI-Assisted Engineering, where Justin Reock addressed the GenAI Divide — the phenomenon where 95% of pilots fail — and how to overcome it using the DORA, SPACE, and Core 4 frameworks. His core argument emphasizes moving beyond anecdotal claims by measuring AI ROI using empirical data from DORA and developer experience (DX) research.
In our custom development practice, we receive frequent inquiries along the lines of “We want to introduce AI, but cannot quantify the benefits.” This article outlines practical implementations for deploying these three frameworks in client engagements, designing contract-backed KPIs, and avoiding common pitfalls.
Why do 95% of pilots fail?
The core of the GenAI Divide lies in the gap where “tool adoption succeeds, but organizational transformation fails.” Typical failure patterns in custom development engagements include:
| Pitfall | Manifestation in client projects |
|---|---|
| Lack of measurement | Stopping at “some people are using it” |
| Focusing solely on output volume | Tracking only commit counts and lines of code, ignoring quality regressions |
| Measuring only individual productivity | Failing to improve end-to-end lead times for the team as a whole |
| Zero investment in training | Leaving adoption to “they'll get used to it,” with nobody sharing effective patterns |
| Lack of executive buy-in | Middle management failing to translate gains into business value, leading to canceled budgets |
These mirror the root dynamics of the “gap between winners and losers” analyzed in AI Agent Enterprise Polarization 2026, where the primary failure is omitting organizational engineering on top of tool adoption.
Translating the three frameworks to custom development
Here is how the three frameworks recommended by Reock can be applied in custom development engagements.
DORA — metrics for delivery speed and stability
DORA evaluates “software delivery speed and stability” across four core metrics.
| DORA metric | Details | Use cases in client projects |
|---|---|---|
| Deployment frequency | Number of production deployments per time window | Top-line metric for monthly client reports |
| Lead time for changes | Time elapsed from commit to production | Tracked on a per-sprint basis |
| Mean time to recovery (MTTR) | Time from incident occurrence to full recovery | Aligned with service level agreements (SLAs) |
| Change failure rate | Percentage of changes causing production failures | Before-and-after comparison of automation |
Because DORA metrics reflect “values that clients feel directly,” they are particularly straightforward to incorporate into monthly contractual reporting.
SPACE — measuring productivity across five dimensions
The SPACE framework operates on the premise that “productivity cannot be measured by a single metric.”
| Dimension | Application in custom development |
|---|---|
| Satisfaction | Satisfaction surveys of developers and client project managers |
| Performance | Contribution to client business KPIs |
| Activity | Counts of commits, PRs, and reviews |
| Communication | Review response times and pair-programming duration |
| Efficiency | Number of interruptions and context switches |
Crucially, the core intent of SPACE is to avoid judging performance by activity alone, making it possible to accurately evaluate scenarios where “commit volume decreased, but satisfaction improved.”
Core 4 — four metrics for executive communication
Core 4 bridges the gap between DORA and SPACE as a metric suite specifically tailored for “executive reporting.”
| Core 4 metric | Details |
|---|---|
| Speed | Lead time and cycle time |
| Quality | Bug density and production incidents |
| Impact | Contribution to client KPIs (revenue, churn rate, etc.) |
| Engagement | Developer satisfaction and retention rates |
Core 4 serves as the ultimate business-outcome metric group used to “persuade executive leadership to sustain AI investments.”
Building four layers of AI ROI measurement in custom development
When introducing AI to custom development projects, organizing your measurement system across the following four layers simplifies implementation. Lower layers have lower implementation costs, while higher layers carry greater persuasive power with executive stakeholders.
[Layer 1: ツールメトリクス]
├ AI 利用回数・トークン消費・受諾率
├ ベンダーが提供するダッシュボード
└ 導入負荷: 最小（既存ダッシュボードを見るだけ）
[Layer 2: DORA メトリクス]
├ デプロイ頻度・リードタイム・MTTR・変更失敗率
├ GitHub Actions + DORA エクスポーター
└ 導入負荷: 小〜中（CI/CD の計測基盤を整備）
[Layer 3: SPACE メトリクス]
├ 5 次元のうち最低 3 次元（S, A, E）を測定
├ 月次サーベイ + 自動収集
└ 導入負荷: 中（サーベイ設計と回収オペレーションが必要）
[Layer 4: Core 4 ダッシュボード]
├ 経営層向け四半期レポート
├ 顧客ビジネス KPI との突合
└ 導入負荷: 大（顧客側の KPI データ連携が前提）In particular, Layer 3 (SPACE) serves as a major differentiator for custom software providers, making it possible to illuminate cases where “metrics look positive on paper, but teams are exhausted.” This aligns with the concept of “balancing cost efficiency with human workload” examined in Optimizing Claude Code Operational Costs in 2026.
“KPI reporting clauses” to include in client contracts
In contracts for custom development projects incorporating AI, explicitly stipulate KPI reporting clauses as follows:
| Clause | Details | What the client should verify |
|---|---|---|
| Measurement scope | Metrics to be measured across DORA, SPACE, and Core 4 | Alignment with industry standards |
| Reporting frequency | Monthly, quarterly, or at project completion | Executive reporting cadence |
| Baseline collection | Measurement during the 4 weeks preceding rollout | Validity of pre-improvement benchmarks |
| Improvement targets | Percentage improvement targets for DORA metrics after 6 months | Feasibility of targets |
| Actions upon non-achievement | Action plan if targets are not reached | Conditions for contract continuation |
| Data handling | Storage and deletion terms for telemetry data | Handling of personal data and sensitive information |
In particular, establishing "remedies for unmet targets" upfront avoids the nightmare scenario where "six months have passed with no measurable results, yet the contract continues."
Five common pitfalls
Here are common pitfalls when implementing ROI measurement in custom development.
Pitfall 1: Over-investing in measurement tooling
Attempting to deploy the entire Core 4 framework from day one will consume excessive engineering bandwidth on telemetry, defeating the original purpose. Start with Layers 1 and 2, then expand incrementally.
Pitfall 2: Launching without establishing a baseline
Without pre-rollout data, evaluations inevitably end with “it feels like things improved.” Make capturing at least four weeks of baseline metrics a contractual requirement.
Pitfall 3: Evaluating success solely through activity metrics
Relying exclusively on commit counts and lines of code as KPIs leads to situations where “AI churns out voluminous, redundant code, inflating numbers on paper.” Always combine activity metrics with the Satisfaction pillar of SPACE.
Pitfall 4: Neglecting alignment with client business KPIs
Core 4 Impact metrics are meaningless unless mapped to “client business KPIs like revenue, churn, and NPS.” Agree upon the specific client KPIs to cross-reference during early contract stages.
Pitfall 5: Treating engineers purely as measurement subjects
Measurement without qualitative surveys engenders developer distrust and skews the underlying data. Incorporate monthly surveys and feedback loops into the contract.
Conclusion — moving from “we adopted AI” to “demonstrating real ROI”
The key to overcoming the 95% failure rate of the GenAI Divide lies in “measurement and organizational design built on top of tooling.” Creating an environment in custom development where teams can articulate how AI drove improvement using the client's own business terminology directly increases contract renewals and billable rates.
How deeply you should integrate DORA, SPACE, and Core 4 varies substantially based on team size, existing CI/CD infrastructure, and how clients track their business KPIs. If you are struggling because “AI was adopted but the benefits cannot be explained” or need to “persuade executive leadership to sustain investments,” we provide tailored roadmaps and quotes based on your current telemetry setup. Please feel free to reach out via our contact form.
