Can we steer a robot foundation model without retraining it?
Our new paper explores this question through control-inspired notions of feature observability and controllability in Vision-Language-Action (VLA) models.
We find that state- and action-relevant information can often be linearly predicted from VLA internal representations—and that these representations can be steered at inference time to change robot behavior, without fine-tuning.
As an example, on a real robot, our approach increased the preferred handle-grasp rate from 14% to 74%, while adding only ~1% inference-time overhead.
The broader idea is that representation-level control could provide a lightweight interface for adapting robot foundation models to new preferences and constraints—without retraining.
Joint work with Hugo Buurmeijer Carmen Amo Alonso
@SwannAiden
Paper:
#
PhysicalAI# #
Robotics# #
VLA# #
RobotLearning#
@StanfordEng @StanfordAILab