Gemini 4 Carbon Tops Intelligence Index, Limits Access

By Priya Raman

The independent evaluator Artificial Analysis scored Google’s Gemini 4 Carbon 59 on version 5.0 of its Intelligence Index, one point ahead of Opus 5.5 at 58.

On October 28, Google unveiled Carbon as its “strongest model yet,” initially directing it toward cyber defenders searching for software flaws. The release came four weeks after Gemini 4 Argon surfaced, extending a restricted rollout rather than opening the model to general users.

The launch followed a schedule reset that began at Google’s May I/O conference. Google had promised Gemini 3.5 Pro within a month, but that model slipped repeatedly and was apparently scrapped after Bloomberg reported that its coding performance missed internal goals. Cheaper Flash releases filled the gap.

Gemini 3.5 Flash appeared at I/O, and 3.8 Flash followed on September 2. Gemini 3.1 Pro Preview remained Google’s newest Pro model until Argon arrived on September 30.

What does the benchmark lead establish?

Carbon produced the 59-point result at its high reasoning setting on Artificial Analysis version 5.0. The same index placed GPT-6 Astra at 57 and GPT-6.1 Sol at 55 in their listed maximum-reasoning configurations. Opus 5.5 scored 58. Its nine evaluations span coding, scientific reasoning, knowledge and other tasks.

Among the nine components are:

• Terminal-Bench 4.0
• FrontierCode 2.0
• HLE-Diamond
• Terminal-Bench-Science 0.1

Argon scored 50 on version 5.0, nine points behind Carbon. Artificial Analysis had assigned Argon 53 on its earlier v4.3.2 index, but the evaluator says scores carry over only within a version, so that earlier number cannot serve as a like-for-like comparison. The result establishes a lead on this composite, not across every research or production workload.

What do the cyber results show?

Google reports Carbon at 88.2% on SEC-bench Pro for vulnerability discovery, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching. Collinear AI supplies CWE-bench, a defensive cybersecurity and AI evaluation benchmark.

The announcement does not disclose test-set sizes, run counts or confidence intervals for these percentages. Google ran the external benchmarks with guardrails disabled, a condition that limits direct extrapolation to the safeguarded product. Researchers should treat the figures as benchmark point estimates rather than expected production success rates.

In a separate internal comparison, Google’s Chrome team found Carbon “more than 10X better” at producing high-quality patches than an unnamed large frontier model from another laboratory. Google also reports Carbon in fourth place on WebDev Arena at 1,768 Elo, an 89-point gain, without disclosing the comparison set or scoring window in the announcement.

Google’s internal secure-code testing found Carbon more than doubled Argon’s success rate. The announcement gives no sample count, trial design, uncertainty range or external funding for that comparison. For procurement and quality teams, the result warrants a controlled patch review rather than an automatic replacement of existing review processes.

On Gray Swan’s indirect prompt-injection benchmark, Google reports a 0.0% attack success rate for Carbon. Argon had led that benchmark at 0.7%. The zero result is a company-reported benchmark outcome, not evidence that enabled production guardrails eliminate injection risk, and the announcement supplies no run count.

Carbon was designed for long-horizon engineering work. Google says internal teams have run it for days, persisting through dead ends on multistep tasks. For lab groups, the relevant claim is sustained task execution, although the announcement does not define a task, duration limit or success metric.

What does Carbon cost to run?

Artificial Analysis recorded 200 million output tokens across Carbon’s index evaluation, about three times the 64 million median for comparable models. Argon generated 110 million output tokens on the same index, 1.7 times the median, confirming that the new model did not reduce token consumption.

Using standard pricing, Artificial Analysis calculated a weighted average cost of $7.76 per task for Carbon. That figure sat below Opus 5.5 at maximum reasoning, $10.04, but above GPT-6 Astra at $6.24 and GPT-6.1 Sol at $1.27.

Latency adds an operational constraint: Carbon waited 97.5 seconds on average before its first output token on Google’s API, compared with a 3.6-second median. For teams structuring API timeouts and human review, the measured wait matters alongside task cost, although the announcement does not state hardware configuration or concurrency conditions.

Google plans introductory rates of $2 per million input tokens and $10 per million output tokens, unchanged from Argon, before rising to $4 and $20 after the promotion. Cached inputs carry a 95% discount. Teams can therefore lower repeated-context expense, but the quoted $7.76 benchmark cost does not show how much each workload benefited from caching.

Who gets access, and when?

Carbon extends the restricted cyber-defender rollout that accompanied Argon. Google opened that access through the Fairwind Program, launched September 2 for government authorities, critical infrastructure operators and software maintainers, and now plans to give selected defenders Carbon access during the announcement week.

Google will expand Carbon through Fairwind to more trusted partners. Broader developer, business and consumer access remains a future phase that the company says will follow defender feedback and further work on guardrails. Google also participates in the U.S. government’s voluntary process for pre-release model testing.

For R&D leaders, Carbon’s 59-point lead, reported patch results and days-long runs make it a candidate for controlled vulnerability work, but the 200 million-token footprint, 97.5-second startup wait and unquantified internal comparisons require budget controls and task-level validation before wider deployment.

The next test is whether Google can retain those measured gains with guardrails enabled while converting selected-defender feedback into a broader release.