I don't think SWE Verified is an ideal benchmark, as the solutions are in the tr... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		fuddle 14 days ago \| parent \| context \| favorite \| on: GPT-5.2 I don't think SWE Verified is an ideal benchmark, as the solutions are in the training dataset.

joshuahedlund 14 days ago [–]

I would love for SWE Verified to put out a set of fresh but comparable problems and see how the top performing models do, to test against overfitting.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact