How much time should robots spend thinking? Vision-Language Models are increasingly used as high-level planners for robots, and the prevailing strategy has been to scale test-time compute to boost capability. But more reasoning steps, bigger models, and longer memory all come with increased latency, tokens, and FLOPs—often with diminishing and uneven returns. So when, and where, is test-time compute actually worth its cost? 🧐 We study three dominant scaling axes and find that each unlocks a distinct capability, showing that test-time compute is not a uniform lever: - Chain-of-thought depth helps with tasks involving implicit semantic, physical, or spatial constraints, but its additional latency is not always necessary (on VLABench, a non-CoT model matches a CoT model on 44% of tasks). - Model size governs the breadth of skills a planner can reliably draw upon, but its benefits appear only when those additional skills are actually required. - Memory history improves performance on long-horizon, history-dependent tasks, but can actively hurt performance elsewhere. Across all three axes, a consistent pattern emerges: the gap between cheap and expensive configurations is large, but highly non-uniform and task-dependent. DIRECT (Dynamic Inference Router for Embodied Compute Tradeoffs) is a lightweight router that reads scene + instruction context and sends each task to the cheapest planner that can still solve it, allocating compute per task rather than committing to one fixed model. 👉 Takeaway: smart allocation of test-time compute can recover frontier-level planning at a fraction of the cost. 📄 Paper: 🔗 Website: Work led by @_jadelynn @milanganai With an outstanding team of collaborators: @ajaysridhar0 @Mozhgan_nasr @katielulula Clark Barrett @jiajunwu_cs @chelseabfinn #Robotics# #VLM# #EmbodiedAI# #MachineLearning# #TestTimeCompute#