注册并分享邀请链接,可获得视频播放与邀请奖励。

Hamel Husain
@HamelHusain
Evals Evals Evals @OpenAI About Me:
加入 September 2012
2.7K 正在关注    66.3K 粉丝
I've been looking into public benchmarks and am astounded by the poor quality vs. how much we rely on them as an industry. @xeophon pointed me to this excellent site which is an independent third party which scrutinizes benchmarks and "looks at the data" 💪 For example, here are some popular benchmarks we all rely on. Before trusting a benchmark its worth running them and inspecting examples the grader failed carefully, to get a sense of the quality. I'm continuing to look more into this, and will write something up eventually once I've collected more data, but I think this is an important subject.
显示更多
0
135
689
74
转发到社区