Gemma 4 vs. Qwen 3.5 vs. Granite 4.0: comparison [spring 2026 for SMBs]

The dawn of the mid-sized open model "Three Kingdoms"

Between March and April 2026, three notable releases were announced back-to-back across the open-model landscape.

• Google Gemma 4 (April 2) — four sizes (E2B, E4B, 26B MoE, 31B Dense), multimodal support, transitioned to the Apache 2.0 license
• Alibaba Qwen 3.5 (late March) — a lineup spanning seven sizes from 2B to 397B-A17B
• IBM Granite 4.0 3B Vision (March 31) — a LoRA-adapter vision language model specialized in enterprise document extraction

While all of these models compete in the category of "cloud-independent mid-sized open models," the defining characteristic of spring 2026 is that their design philosophies and core strengths are completely distinct. Rather than a standalone guide, this article organizes the models across four dimensions (licensing, benchmarks, core strengths, and operational costs) to help implementation teams choose according to their use case.

For local deployment steps for Gemma 4 itself, see What can Gemma 4 do? A hands-on guide to running Google's latest open model locally; for agent control design in general, please refer to Introduction to harness engineering.

1. Licensing and commercial use

The first factor to check when choosing a model is licensing. In enterprise deployments, whether redistribution is permitted and whether derived models face restrictions can be critical issues.

Most notable is that Gemma 4 has finally transitioned to Apache 2.0. Proprietary clauses previously found in Gemma's license, such as mandatory adherence to usage policies, have been eliminated, putting it on an equal footing with Qwen and Granite. Regarding this license change, VentureBeat remarked that "this may ultimately prove more important than the benchmark numbers."

2. Real-world performance through benchmarks

Comparison in the 31B/35B class

Reviewing public benchmarks available as of spring 2026 reveals a landscape where Gemma 4 31B holds a slight lead in overall capabilities, while Qwen 3.5 remains consistently strong in coding and agent tasks.

While Gemma 4 outperforms Qwen 3.5 in mathematical reasoning and general knowledge, Qwen 3.5 holds a clear edge in practical coding (LiveCodeBench, Codeforces). Japanese benchmark articles trending on Zenn also reflect a consensus that competition in the same parameter class is neck and neck, with performance depending heavily on the workload.

Multilingual and translation tasks

Global developer communities report widespread findings that Gemma 4 outperforms Qwen 3.5 in German, Arabic, Vietnamese, and French, demonstrating Gemma 4's advantage in multilingual tasks. A similar trend is observed in Japanese tasks, with users noting that Gemma 4 produces more natural translations and summaries.

Enterprise document extraction

Bringing Granite 4.0 Vision into the comparison distinctly changes the positioning. Granite is designed as a compact 3B model paired with LoRA adapters. Rather than being measured on general benchmark leaderboards, it should be evaluated for its specialization in document extraction (chart-to-code, table-to-HTML, invoice OCR). It holds a proven record, ranking third on the VAREX leaderboard for the 2–4B parameter class.

3. Strengths and optimal choices by use case

Organizing the three models by "what they should be used for" makes selection straightforward.

4. Operational costs and hardware requirements

Minimum operational requirements

An important community caveat is that Gemma 4's MoE model (26B A4B) has been reported to have slower inference throughput than comparable Qwen 3.5 MoE models. Be sure to run benchmarks using llama.cpp or vLLM before deploying to production.

How to evaluate costs

While open models carry "zero licensing fees," actual operating costs are determined by GPU time × inference throughput. There have been reported cases where choosing Gemma 4 26B A4B in anticipation of MoE efficiency resulted in lower-than-expected throughput and higher overall costs. Therefore, it is essential to conduct actual measurements on identical workloads during the PoC phase.

Model selection flowchart

When making your selection, following this decision sequence will keep your evaluation focused:

1. Is document extraction the primary objective? → If yes, Granite 4.0 Vision; if no, proceed
2. Are coding and development support the primary objectives? → If yes, Qwen 3.5 35B-A3B; if no, proceed
3. Will it run on edge devices (smartphones, embedded systems)? → If yes, Gemma 4 E2B/E4B; if no, proceed
4. Is multilingual performance a priority? → If yes, Gemma 4 31B; if no, Qwen 3.5 35B-A3B
5. Is inference throughput a priority? → Dense models over MoE (Gemma 4 31B Dense / Qwen 3.5 Dense series)

Conclusion

In spring 2026, the mid-sized open model space is no longer about finding a single "ultimate model," but rather leveraging distinct models according to their respective strengths.

• Gemma 4 — The all-rounder for multilingual, multimodal, and edge inference
• Qwen 3.5 — The proven performer for coding and agent workflows
• Granite 4.0 Vision — The focused specialist for enterprise document extraction

Because all three models are released under the Apache 2.0 license with no commercial restrictions, evaluating multiple models in parallel is the fastest route to selection. We strongly recommend running a one-week PoC to make decisions based on benchmark results from your own workloads.

For agent control design, refer to Introduction to harness engineering; for pairing models with MCP, see the Complete MCP guide.

References

• Google releases Gemma 4 under Apache 2.0 — VentureBeat
• Gemma 4 vs Qwen 3.5 — Interconnects
• Qwen 3.5 vs Gemma 4: the benchmark-by-size comparison — Maniac
• IBM Releases Granite 4.0 3B Vision — MarkTechPost
• Granite 4.0 3B Vision — Hugging Face Blog