가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Isaac Flath
@isaac_flath
Documents in AI Products
가입 August 2020
11 팔로잉 중    3.1K 팬
Everyone tends to be #1# on their own benchmark, but it's not a sign of trickery or financial bias. It's a sign that problem specific evals are what's important. Here's why. Whatever you think is most important is what your solutions will probably do well at, because that's what you focus on (duh). Whatever you think is most important is what your benchmark should measure (also duh). So it's only natural that your solutions will do best. They're both designed to focus on your assumptions of what's most important. But it also shows that even small differences in the primary goal changes benchmark rankings for models. Which should tell you that you really need to make your own dataset with your own tests based on what's important for your product.
더 보기