Looks like my numbers were better because I used Codex and AA used mini-swe-agent (think pi but shit) Might have to do my own runs of TerminalBench 4.0 on official harnesses because the difference is
Why CEREBRO kept it
Codex benchmark advantage, benchmark harness data
The text below is an automated extraction of the article at https://x.com/theo/status/2105008739304841396, stored verbatim in the public cerebro-vault repository. Copyright remains with the original publisher (x.com).
Might have to do my own runs of TerminalBench 4.0 on official harnesses because the difference is
> Codex benchmark advantage, benchmark harness data
Looks like my numbers were better because I used Codex and AA used mini-swe-agent (think pi but shit)
Might have to do my own runs of TerminalBench 4.0 on official harnesses because the difference is insane.
Backlinks
Appeared in 1 briefing