Hacker News new | past | comments | ask | show | jobs | submit login

Any time I see humaneval comparisons I feel the need to point out that humaneval is only 164 questions. 66.5 vs. 65.9% is a difference between 109 and 108 solutions, a single question. Still interesting work though



At least they used HumanEval+, which adds a bunch more test cases and fixes some errors in the original benchmark!




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: