注册并分享邀请链接,可获得视频播放与邀请奖励。

Isaac Flath
@isaac_flath
Documents in AI Products
加入 August 2020
11 正在关注    3.1K 粉丝
Everyone tends to be #1# on their own benchmark, but it's not a sign of trickery or financial bias. It's a sign that problem specific evals are what's important. Here's why. Whatever you think is most important is what your solutions will probably do well at, because that's what you focus on (duh). Whatever you think is most important is what your benchmark should measure (also duh). So it's only natural that your solutions will do best. They're both designed to focus on your assumptions of what's most important. But it also shows that even small differences in the primary goal changes benchmark rankings for models. Which should tell you that you really need to make your own dataset with your own tests based on what's important for your product.
显示更多