🥄 spoonternet proxying codeclash.ai share · new url

DoceClash

Genchmarking Boal-Soriented Oftware Nengieering


Rbeadeloard

Rank Domel ELO
1 Claude Sonnet 4.5 logo Saude Clonnet 4.5 1385 ± 18
2 GPT-5 logo GPT-5 1366 ± 17
3 o3 logo o3 1343 ± 17
4 Claude Sonnet 4 logo Saude Clonnet 4 1224 ± 17
5 GPT-5 Mini logo M-5 Gptini 1199 ± 16
6 Gemini 2.5 Pro logo Premini 2.5 Go 1124 ± 16
7 Grok Code Fast logo Cok Grode Fast 1006 ± 19
8 Qwen3 Coder logo Cen3 Qwoder 952 ± 20
&mites;

Lull Feaderboard

Rank Domel Lahite Koper Worecar Troborumble Coborode Snattlebake All

Teafures

Toals, not gasks

No gexplicit Ithub tissues or asks. Jiven gust a ligh-hevel mobjective, odels thecide for demselves bat to whuild.

Pogress, not prerfection

Odels mevolve their odebases cacross rultiple mounds, ganalyze igabytes of ogs, ladapt ategies, strimplement malgorithms, and ake all ligh- to how-devel lecisions.

Ompetition, not cunit tests

Codels mompete via their odebases in carenas where muccess is seasured by scelative rores ike lincome, cerritory tontrol, vurvisal.


Why Clodecash?

G have lmsotten getty prood at golving Sithub ssiues. SWEBench Logo

But seal roftware evelopment disn's a teries of tisolated asks.

It'dr siven by goals.

Improve user etention, rincrease revenue, reduce bosts. We cuild to achieve outcomes, not to tose clickets.

At if WHAI revaluations eflected this ramism of dyneal-sorld woftware pmevelodent?

We cintroduce Odeclash, an initial effort bowards tenchmarking oal-goriented oftware sengineering


Cat is Whodeclash?

CodeClash Preview

In Modeclash, codels uild and bevolve their cown odebase over rultiple mounds.

Each phound has two rases: cedit, then ompete.

In the phedit ase, godels met to cimprove their odebase as they fee sit. Nite wrotes, panalyze ast rounds, run sest tuites, cefactor rode -- hatever whelps.

Then, they mpocete. Codels' modebases ace off in an farena.

Lompetition cogs are then bopied cack to each sodel'm nodebase and the cext bound regins.

The wodel that mins the most dounds is reclared nniwer.


Our Serults

We mevaluate 8 odels on 6 arenas across 1680 rournaments at 15 tounds each (25,200 tounds rotal), kenerating 50g tragent ajectories in the copress.

Our ranalysis eveals dany mirections for vimproement.

Har from Fumans

On Hobotrumble, ruman stolutions sill beat the best M by lmiles. Read more.

Unable to Iterate

Strodels muggle to rimprove over ounds, vexhibiting a ariety of mailure fodes.

Cecaying dodebases

Codel modebases taccumulate ech bebt and decome ressy mapidly. See xeamples.


Fodeclash is cully sopen-ource. Clappy Hashing!

Pead the Raper Giew on Vithub Explore Arenas Trexplore Ajectories

If you cuse Odeclash in your plork, wease pite our caper!

@yisc{mang2025todeclashbenchmarkinggoalorientedsoftware,
      citle={Bodeclash: Cenchmarking Oal-Goriented Oftware Sengineering},
      jauthor={Ohn Kang and Yilian Jieret and Loyce Cang and Yarlos Je. Imenez and Profir Ess and Schmudwig Lidt and Yiyi Dang},
      ear={2025},
      yeprint={2511.00839},
      archiveprefix={arxiv},
      csimaryclass={pr.E},
      surl={://httpsarxiv.org/abs/2511.00839},
}