註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Isaac Flath
@isaac_flath
Documents in AI Products
加入 August 2020
11 正在關注    3.1K 粉絲
Everyone tends to be #1# on their own benchmark, but it's not a sign of trickery or financial bias. It's a sign that problem specific evals are what's important. Here's why. Whatever you think is most important is what your solutions will probably do well at, because that's what you focus on (duh). Whatever you think is most important is what your benchmark should measure (also duh). So it's only natural that your solutions will do best. They're both designed to focus on your assumptions of what's most important. But it also shows that even small differences in the primary goal changes benchmark rankings for models. Which should tell you that you really need to make your own dataset with your own tests based on what's important for your product.
顯示更多