Wondering how VLMs can be trained to play games using only visual inputs, like Anthropic’s newly released Claude Fable 5?
Check out our recent work, Odysseus:
In Odysseus, we train VLMs to play games directly from visual inputs, using Super Mario Land as a testbed, and scale RL to improve their long-horizon decision-making capabilities. Excited to see more exploration in this direction!