wh
@nrehiew_
eng primarily, ml mostly, research previously
가입 October 2023
104 팔로잉 중 18.5K 팬
Super cool work! This also explains why some benches have an "artificial ceiling".
For example, on DeepSWE, Epoch found 23 false negatives out of 131 tasks. This means a ceiling of roughly 79.6%, which actually matches the current (almost) saturated high score of ~74%
더 보기
Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.
더 보기
