Frontier AI Models Struggle With Real-World Professional Tasks

The gap between AI hype and AI capability has been laid bare by a striking new benchmark. SemiAnalysis, an analytics firm known for deep technical research, has released GDPx — a benchmark that places AI agents inside simulated professional work environments and tasks them with completing complex, multi-hour assignments across eleven specialized domains. The result: no model scored above 12 percent.

Anthropic’s Claude Opus 5.5 led the leaderboard at 12.0 percent, followed by Google’s Gemini 4 Carbon at 10.9 percent and OpenAI’s GPT-6 Astra at 9.6 percent. While these are the most advanced AI systems available today, the scores reveal that reliable, long-horizon professional work remains largely beyond their reach.

What Makes GDPx Different

Most AI benchmarks test what a model knows. GDPx tests what a model can do. Instead of presenting self-contained questions with all context in the prompt, the benchmark drops AI agents into realistic working environments where they must explore, discover relevant information, and operate professional software for up to eight hours per task.

This is a deliberate departure from OpenAI’s GDPval benchmark, released in September 2025, which SemiAnalysis describes as inadequate for evaluating frontier models because every task is precisely specified and non-interactive. Real professional work requires agents to figure out what the task actually demands by navigating complex environments — something GDPx forces them to do.

The 206 tasks were built with 14 specialist companies that create reinforcement-learning training environments for AI labs. Each task runs inside a working simulation that mirrors what a professional would face:

• Finance — Agents search private-equity data rooms containing roughly 4,000 anonymized files to find what they need
• Chip design — Agents work with tools including Siemens NX, SolidWorks, Fusion 360, KiCad, and AMD Vivado
• Materials science — Agents operate simulated lab equipment including atomic force microscopes and transmission electron microscopes
• Cybersecurity — Agents navigate multi-service enterprise environments seeded with vulnerabilities and access controls
• Biology — Agents work with single-cell transcriptomics, genomics, drug discovery, and biosecurity technologies

The Leaderboard: How Each Model Performed

The public leaderboard reports scores for 13 models. The spread from first to last place is just over nine percentage points, but even the leader falls far short of competent professional work:

• Claude Opus 5.5 (Anthropic) — 12.0%
• Gemini 4 Carbon (Google) — 10.9%
• GPT-6 Astra (OpenAI) — 9.6%
• Muse Wonder Full (Meta) — 9.0%
• Claude Fable 5.1 (Anthropic) — 8.5%
• GPT-6.1 Sol (OpenAI) — 8.4%
• Claude Sonnet 5.5 (Anthropic) — 8.2%
• Gemini 4 Argon (Google) — 7.3%
• Muse Wonder (Meta) — 6.8%
• DeepSeek V4.1 Flash 1017 (DeepSeek) — 5.4%
• GPT-6 Luna (OpenAI) — 5.1%
• Grok 4.7 (xAI) — 3.9%
• GLM 5.3 (Zhipu AI) — 2.7%

One notable finding is the cost-performance tradeoff. Claude Opus 5.5 costs approximately $51.41 per task, while Gemini 4 Carbon delivers its second-place score at just $16.29 per task — roughly a third of the price. GPT-6 Astra sits at $53.99 per task. For organizations deploying AI agents at scale, the cost difference between a 12 percent and 10.9 percent success rate could be substantial.

Where AI Completely Fails

Finance and legal tasks proved nearly impossible for every model tested. The tasks are so demanding that even partial-credit grading — which counts the share of individual requirements met rather than requiring a perfect solution — yielded scores near zero. SemiAnalysis notes that in finance, a single wrong value in a spreadsheet makes the entire output useless in practice, and by that strict standard no model would have completed even one task.

Domain-specific winners reveal where each model’s strengths lie:

• Chip design — Claude Opus 5.5
• Hardware design — Claude Opus 5.5
• Quantitative trading — Claude Opus 5.5
• Biology — Gemini 4 Carbon
• Software engineering — Gemini 4 Carbon
• Recursive self-improvement — GPT-6 Astra
• Materials science — GPT-6 Astra
• Cybersecurity — Muse Wonder Full at 35.2%

Safety Classifiers Create a Hidden Penalty

One of the most interesting findings is that safety classifiers actively reduced scores for several leading models. Claude Opus 5.5 and GPT-6 Astra both saw their overall scores depressed because their safety systems sometimes intervened in cybersecurity, biology, and alignment tasks. When SemiAnalysis excluded those three domains from the calculation, GPT-6 Astra jumped to first place at 11.8 percent.

This creates a paradox: the most capable models are also the most cautious, and that caution costs them on benchmarks that require operating in sensitive environments. In cybersecurity specifically, GPT-6 Astra and GPT-6.1 Sol scored zero — likely because their safety classifiers blocked them from performing attack-oriented tasks — while Meta’s Muse Wonder Full scored 35.2 percent, the highest domain-specific result in the entire benchmark.

How Scoring Works

The headline score is pass@1 averaged over three attempts per task. Every domain carries equal weight, and partners within a domain are also weighted equally. Grading is binary or partial-credit depending on the field. Because each of the 11 domains accounts for roughly 9.1 points of the total, a headline score of 12 percent is roughly equivalent to fully solving one domain and making small progress in a second.

SemiAnalysis selected domains by asking where frontier AI is most likely to generate revenue, deliberately excluding government, real estate, and insurance. The 11 domains are chip design, cybersecurity, biology, finance, legal, quantitative trading, hardware design, materials science, recursive self-improvement, alignment, and software engineering.

What This Means for the AI Industry

The GDPx results arrive at a moment of enormous investment in AI agents. Companies across industries are racing to deploy autonomous systems that can handle complex professional work. The benchmark suggests that while AI has made remarkable progress in knowledge tasks, conversation, and code generation, the leap to reliable multi-hour professional work in specialized environments remains enormous.

However, the results should be read carefully. GDPx is deliberately designed to sit at the frontier — to test the hardest work that AI might plausibly attempt. A 12 percent top score on expert-level tasks does not mean AI fails at routine work. Many business applications involve tasks far simpler than searching 4,000 files in a private-equity data room or operating a transmission electron microscope.

The benchmark also highlights how far evaluation methods have evolved. Traditional benchmarks that test knowledge in a single prompt are increasingly inadequate for measuring what frontier AI systems can do in real working environments. GDPx represents a shift toward evaluation that mirrors actual professional conditions, with all their complexity, ambiguity, and need for sustained autonomous operation.

SemiAnalysis plans to update GDPx every three to six months and intends to add broader independent scrutiny and a human comparison baseline in later editions. Until then, the benchmark stands as the most rigorous available test of whether AI agents can truly do professional work — and the answer, at least for now, is that they are only beginning to try.

---

Edited by Palawan @QUE.COM
Website: https://QUE.COM Intelligence
Sponsored by: https://MAJ.COM AI Autonomous