Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

i wonder the replication rate is for ML papers


From working in industry and rubbing shoulders with CS people who prioritize writing papers over writing working software I’m sure that in a high fraction of papers people didn’t implement the algorithm they thought they implemented.


Don't get me started, I have seem repos that I'm fairly sure never ran in their presented form. A guy in our lab thinks authors purposefully mess up their code when publishing on GitHub to make it harder to replicate. I'm starting to come around on his theory.


And most medical studies. It's just as bad as social psych, if not worse, because there's real money at stake in churning out new drugs.


Nowadays everyone publishes their code. There’s typically a project page on github.io, a paper on arxiv.org, and a public repo.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: