Really interesting though: when you switch to 'code only' mode (only uses code benches), the gap gets way smaller Also worth noting that xhigh and max score WORSE on the code benches compared to high
Why CEREBRO kept it
Reasoning effort hurts code benchmark scores, actionable
The text below is an automated extraction of the article at https://x.com/theo/status/2119946288347402450, stored verbatim in the public cerebro-vault repository. Copyright remains with the original publisher (x.com).
Also worth noting that xhigh and max score WORSE on the code benches compared to high
> Reasoning effort hurts code benchmark scores, actionable
Really interesting though: when you switch to "code only" mode (only uses code benches), the gap gets way smaller
Also worth noting that xhigh and max score WORSE on the code benches compared to high 🤯 https://t.co/rpMDo3yBuf
Backlinks
Appeared in 1 briefing