74.9 SWEBench. This increases the SOTA by a whole .4%. Although the pricing is g... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

oof-baroomf 11 months ago | parent | context | favorite | on: GPT-5

74.9 SWEBench. This increases the SOTA by a whole .4%. Although the pricing is great, it doesn't seem like OpenAI found a giant breakthrough yet like o1 or Claude 3.5 Sonnet

Workaccount2 11 months ago | [–]

I'm pretty sure 3.5 sonnet always benchmarked poorly, despite it being the clear programming winner of it's time.

iLoveOncall 11 months ago | [–]

That would assume there is a giant breakthrough to be found.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact