Jonas Geiping(@jonasgeiping):What else have we been up to? As models get better and work over longer and longer time horizons, how do we even evaluate how well they can act and adapt? One domain we really like there is forecasting, as a hard task that test reasoning under uncertainty. We've made a benmchmark out of this, where we simulate a whole 3 month period of news, and sanboxed let models continuously read news from those days, plan, and update their forecasts. (see the animation below, just don't be fooled by its speed, this is a slice of the larger 12m token trajectory) Many more details linked below:

Jonas Geiping

@jonasgeiping

Machine Learning Researcher in Tübingen at the ELLIS Institute & Max Planck for Intelligent Systems // Working on Safety & Efficiency of modern ML

参加 September 2021

822 フォロー中 5.8K ファン

Jonas Geiping@jonasgeiping

2026.05.15 17:49

What else have we been up to? As models get better and work over longer and longer time horizons, how do we even evaluate how well they can act and adapt? One domain we really like there is forecasting, as a hard task that test reasoning under uncertainty. We've made a benmchmark out of this, where we simulate a whole 3 month period of news, and sanboxed let models continuously read news from those days, plan, and update their forecasts. (see the animation below, just don't be fooled by its speed, this is a slice of the larger 12m token trajectory) Many more details linked below: