Renews.
← AI

New Benchmark Shows Coding Agents Fail Most Real-World Tasks

A new benchmark reveals that even the best AI models fully complete only about one in four complex web app modification tasks.

AI·1 source
Photo: geralt / Pixabay

Researchers from Alibaba and several Asian universities developed Co-Design Bench to evaluate coding agents on real user change requests. The test requires agents to rework existing web apps based on inputs such as images, audio, video, or documents. Tasks range from small tweaks to full-scale project overhauls, including scenarios where users rephrase requests or roll back changes. Grading involves an AI judge testing the app in a browser and a second re-checking positive verdicts for mock-ups or hardcoded results. Points are awarded only to features that actually work in the app.

GPT-6 Astra was identified as the best of the ten models tested, fully completing 26.4 percent of tasks. Claude Opus 5 and Qwen3.8-Max ranked behind it, while Grok 4.6 landed in last place. In most cases, agents handle the bulk of the work but leave at least one detail unfinished. Model performance drops as tasks combine more types of changes, including layout, design, interaction, and content. GLM-5.2 used nearly six times as many tokens as GPT-6 Astra on smaller changes but scored significantly worse.

Existing coding benchmarks have become too easy for agents, with models achieving scores above 99 percent by running, checking, and fixing their own code. A separate study found that the actual acceptance rate of AI-generated code was roughly 24 percentage points below scores on SWE-bench Verified. The authors recommend that companies test both the model and the working environment as a pair. Co-Design Bench has limits because it evaluates only one fixed request per task and does not include follow-up questions or mid-task corrections.

Sources

Every sentence of this story was checked against facts quoted from these articles. How we make the news

Spotted an error? Tell us

Sign in

Keep the stories you like in your favorites.

Forgot your password?

No account?