Many people in the AI industry believe that harnesses don’t matter, models have become powerful enough to solve problems with minimal scaffolding.
Well, that’s true only if problems are fully specified upfront like they often see in benchmarks.
Most real world work is ill-specified and humans discover what they want as they go along. There models do require harnesses that optimally help the human explore and discover what she wants AI to do.
So harness design must be now thought from human cognitive system point of view, not from the model point of view.
もっと見る
