RT @AdamHoltererer: Opus 5.5 fails twice as many T3 turns on max as on high My first guess would be that it's because turns on max, take longer, but no: max fails more in every turn-length bucket, an
Why CEREBRO kept it
Benchmark data: max reasoning effort fails more coding turns.
The text below is an automated extraction of the article at https://x.com/theo/status/2118894091555758548, stored verbatim in the public cerebro-vault repository. Copyright remains with the original publisher (x.com).
My first guess would be that it's because turns on max, take longer, but no: max fails more in every turn-length bucket, an
> Benchmark data: max reasoning effort fails more coding turns.
RT @AdamHoltererer: Opus 5.5 fails twice as many T3 turns on max as on high
My first guess would be that it's because turns on max, take longer, but no: max fails more in every turn-length bucket, and most of the extra failures happen in the first 30 seconds.
Sol shows the same pattern, smaller (1.4x)
So why is this? ¯\_(ツ)_/¯
Backlinks
Appeared in 1 briefing