登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
参加 May 2026
286 フォロー中    471 ファン
TL;DR: SO-Arena is a benchmark that measures System One models, which return a decision in a single forward pass, on both capability and reliability. A more capable model is not necessarily a more stable one. SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models Key points ・📚 2,919 tasks across 11 domains, built from 40 public datasets ・⚡ Open-weight System One models answer within 0.3 s per task, tens of times faster than System Two ・📉 Even the best System One trails GPT-6 Luna (79.7) by about 10 points ・🔀 Order stability: does the answer survive reshuffling the options? ・🚫 Null discrimination: can the model tell when the correct option has been removed? ・🔍 The most reliable model, JevAny-27B, ranks 11th of 21 in capability Simple semantic judgments are fine for small models, but multi-step derivation like math benefits greatly from thinking. If you use these for routing or verification, look at stability, not just accuracy. #LLMBenchmark# #SystemOne#
もっと見る