What 90 Benchmarks Across Claude Code, Codex CLI, and Copilot CLI Taught Us About Selecting CLIs for Client Projects

In May 2026, Zenn published Comparing Claude Code, Codex CLI, and Copilot CLI Across QCD (Adult Independent Research #16). Measuring quality (Q), cost (C), and delivery speed (D) across 90 tasks revealed nearly identical quality among all three options. The most significant takeaway was the “divergence in code review behavior,” leading to the conclusion that teams should select the underlying model rather than obsess over the CLI itself.

In custom development engagements, our firm must decide on every project “which CLI to use, during which phases, and by whom.” Drawing on the implications of this QCD benchmark, this article presents a practical CLI selection matrix tailored for professional engineering environments.

What the benchmark's “virtual tie” really signifies

The fact that quality converged across all three tools over a substantial sample of 90 trials provides reassuring proof for custom development teams that “you can achieve high quality regardless of which one you choose.” In the past, teams often searched for the “definitive CLI” through head-to-head articles like Cursor 3 vs Claude Code; by mid-2026, the industry has entered a phase of “purpose-driven segregation.”

The characteristics highlighted by the benchmark are summarized below.

In particular, differences in "code review behavior" separate real-world operational quality in custom development in ways benchmarks do not capture. In designs where you "delegate PR reviews to a CLI," the behavioral divergence where Claude "asks clarifying questions when uncertain" while Codex "simply churns out a suggestion for now" translates directly into revision counts and final code quality.

Selection matrix for custom development — deciding across four pillars

Evaluating CLI selection across the following four pillars provides clear decision criteria for individual client projects.

Pillar 1: Project scale

Pillar 2: Regulated industries

Pillar 3: Budget (monthly tokens)

For details, refer to the “100,000 JPY per person-month baseline” discussed in Optimizing Claude Code Operational Costs in 2026 for easier project estimation.

Pillar 4: Team proficiency

Phase-by-phase CLI deployment in custom development

Deploying CLIs across specific project phases results in the following allocation.

[要件定義]
  └ Claude Code（Plan モード）— 仕様書ドラフト

[設計]
  └ Claude Code（サブエージェント）— ADR / シーケンス図

[実装]
  ├ Claude Code — 新規機能
  ├ Codex CLI — リファクタ・移行系
  └ Copilot CLI — IDE 内の Lint 修正・命名

[コードレビュー]
  └ Claude Code（質問挙動が強み）

[テスト追加]
  └ Codex CLI（提案多めが好相性）

[ドキュメント]
  └ Copilot CLI（短く簡潔）

[本番障害対応]
  └ Claude Code（Plan モード + 影響範囲分析）

In particular, for "code reviews" and "production incident response," we recommend Claude Code because its behavior of "asking clarifying questions when uncertain" defaults to the safer side. This aligns with the insight highlighted in Approval Gates in Claude Code Auto Mode that "an AI that asks questions causes fewer incidents."

“CLI usage clauses” to include in client contracts

When utilizing CLIs in client engagements, the following clauses must be explicitly defined in the agreement.

In particular, because enterprise contract terms for “training data exclusion” differ by vendor, always verify the latest contractual terms.

Five common pitfalls

Here are common pitfalls when using CLIs in custom development.

Pitfall 1: Defaulting to Claude Code for every project

Deploying the higher-cost Claude Code across all projects will erode profit margins on spot engagements. Draw clear boundaries based on project scale and budget from day one.

Pitfall 2: Taking benchmark numbers at face value

Benchmarks reflect performance on specific tasks. “Build an in-house benchmark using five representative tasks from your own projects” and re-evaluate tools semi-annually.

Pitfall 3: Fully delegating code reviews to AI

A CLI code review serves as “an aid to human reviewers, never a full replacement.” Always require human sign-off for final approvals and avoid treating CLI review findings as primary metrics.

Pitfall 4: Forgetting to lock model versions

If models change unannounced during an ongoing project, you will encounter scenarios where “prompts that worked last week suddenly fail.” Specify a “30-day advance notice for model changes” in your contracts.

Pitfall 5: Failing to archive logs

If you do not archive CLI prompt logs, you will not be able to reproduce issues during incidents. Mandate at least 90 days of prompt and output log retention in client agreements.

Conclusion — from “which CLI is best” to “how to use each tool strategically”

The “virtual tie across all three tools” demonstrated by the 90-trial benchmark signals to engineering service providers “the end of agonizing over tool selection.” Moving forward, relying on a selection matrix that “allocates tools strategically based on scale, regulation, budget, and team makeup” will become standard practice.

Whether you are wondering “which CLI is the right fit for your team” or find that “Claude Code was adopted but is underutilized,” we offer tailored consultations and estimates based on project scale and regulatory requirements. Feel free to contact us via our contact form.

Sources

• Comparing Claude Code, Codex CLI, and Copilot CLI Across QCD (Zenn)
• Cursor 3 vs Claude Code（GH Media）
• Claude Code Operational Cost Optimization 2026 (GH Media)
• Safely Operating Claude Code Auto Mode in Custom Development (GH Media)