5.83x Speedup: PolicyTrim Helps VLA Robots Take Fewer Detours and Finish Tasks Faster

[Introduction] VLA models can already perform tasks, but real robots are still slow! PolicyTrim is a method to optimize the execution efficiency of VLA robots without retraining. It helps robots complete tasks more directly and improve overall speed by extending reliable action sequences and reducing redundant steps.

Vision-Language-Action (VLA) models unify visual observations, language instructions, and robot actions into a single policy model, enabling robots to perform complex manipulation tasks based on natural language.

From OpenVLA, OpenVLA-OFT to π0.5, GR00T, such models are becoming an important technical route for embodied intelligence.

However, when VLA models are actually deployed on robots, a key bottleneck immediately emerges: the total time for a robot to complete a task depends not only on how fast each inference is, but also on how many inferences and real physical actions the policy needs to execute.

In multiple rollouts of the same task, the number of execution steps of the VLA model fluctuates significantly, indicating that shorter and more direct successful paths are achievable, but the current policy often only finds them by chance.

Unreliable tail of action chunks: The model predicts multiple actions at once, but the later actions are more prone to accumulated errors, forcing the system to frequently re-plan. Forcibly lengthening the execution horizon reduces success rate and increases physical steps.

Severe redundancy in physical actions: Even if the task ultimately succeeds, the trajectory may contain a large number of corrective, backtracking, and repetitive actions.

Therefore, the deployment efficiency of VLA depends not only on the inference latency per step, but also on policy efficiency: how many actions in a single prediction can be safely executed? How many real physical steps are needed to complete the task?

Existing methods mostly start from the computation side, such as visual token pruning, quantization, KV-cache reuse, distillation, or inference engine optimization. These methods can reduce single inference latency, but if the policy still requires many redundant physical steps, the actual task time remains long.

Directly fixing longer action chunks is also unreliable.

Experiments in the paper show that as the action execution length is forcibly increased, the success rate may drop, and the number of physical steps may increase instead. This indicates that low-quality tail predictions introduce errors into the physical world, requiring more corrective actions from the robot.

Key question: Can we, without sacrificing task success rate, enable existing VLA policies to automatically learn to "take fewer detours" and complete tasks with fewer physical steps through post-training?

A team led by Professor Yinjie Lei from Sichuan University proposed PolicyTrim—a two-stage reinforcement learning post-training framework for "policy efficiency." It does not change the VLA architecture or recollect expert data, but continues to optimize the robot's real execution behavior on top of the existing pre-trained policy.

Project page: https://inceptionwang.github.io/PolicyTrim/

Paper link: https://arxiv.org/abs/2606.22540

Code link: https://github.com/INCEPTIONwang/PolicyTrim

Model link: https://huggingface.co/INCEPTIONwang/PolicyTrim

The new method achieves:

• 3× action chunk utilization, reliable execution over longer horizons;
• 51.4% reduction in physical steps, compressing redundant corrective actions;
• 5.83× end-to-end speedup on LIBERO Object/π0.5;

From "Compute Faster" to "Act Faster"

The core goal of PolicyTrim is not to replace computation-side acceleration, but to address another overlooked dimension: reducing the number of forward inferences required to complete a task. It decomposes policy efficiency into two optimizable objectives: first, extend reliable action chunks, then compress redundant physical steps.

1. Reliable Action Chunk Extension

Currently, many VLA policies output action sequences in the form of action chunks, but during deployment, they often only execute the first few steps. In the first stage, PolicyTrim uses dynamic execution horizon exploration: different acceptance ratios are assigned to the same rollout group, allowing the model to explore reliable boundaries in parallel across short, medium, and long windows.

Trajectories that successfully complete the task with longer execution windows receive higher rewards, encouraging the model to make the originally unreliable tail actions of the chunk more usable.

The horizon reward is only activated when a successful trajectory appears in the group, preventing the model from blindly pursuing longer chunk lengths in failure cases.

2. Redundancy-Aware Step Reduction

After the reliable horizon is extended, the second stage further reduces the total number of physical steps. PolicyTrim introduces a step-saving reward: the fewer steps a successful trajectory uses to complete the task, the higher the reward.

The step-saving reward directly encodes "shorter, more direct successful trajectories" into the optimization objective.

Group-anchored regularization suppresses non-reproducible speculative shortcuts, preventing the model from pursuing short paths at the expense of success rate.

The two-stage sequential optimization avoids interference between the objectives of "lengthening chunks" and "shortening steps," ultimately reducing the number of forward inferences required to complete the task.

Experimental Results

The paper validates PolicyTrim on LIBERO, ManiSkill, Meta-World, and real robot tasks, covering different VLA architectures such as π0.5, OpenVLA-OFT, and GR00T. Overall results show that PolicyTrim significantly improves execution efficiency while maintaining stable task success rates.

In LIBERO, π0.5 achieves up to 5.83x end-to-end speedup while maintaining a success rate above 98%.

Cross-benchmark results show that PolicyTrim achieves up to 2.36x speedup on ManiSkill and 2.52x speedup on Meta-World.

Cross-architecture results show that the retrained OpenVLA-OFT using the full two-stage pipeline achieves 2.97x speedup; OpenVLA using only the second stage also achieves 1.41x speedup.

Qualitative Comparison: Fewer Detours, More Direct Trajectories

In randomly sampled LIBERO tasks, the baseline often exhibits redundant corrective actions and hesitation/jittering near the target; PolicyTrim's trajectories are smoother and more direct, completing the same task with fewer physical steps, typically compressing the number of physical steps to about half.

Real Robot Deployment: Efficiency Gains in Simulation Transfer to the Physical World

The paper further validates real robot tasks on an Agilex Piper robotic arm equipped with two Intel RealSense D435i cameras, including three tasks: FlipMug, HangMug, and TapeBox. The experiments cover standard settings with fixed target positions and dynamic settings with random perturbations during grasping.

PolicyTrim maintains or improves success rates under both standard and dynamic perturbation settings while significantly reducing real execution time. Under standard real robot settings, it achieves an average of 1.86x end-to-end speedup.

From the experimental results, PolicyTrim does not merely optimize benchmark metrics: on physical robots, task completion time is also reduced to about half of the baseline, and robustness is maintained in dynamic disturbance scenarios.

Why is PolicyTrim Effective?

• It improves efficiency at the policy level: it reduces redundant actions and unnecessary replanning, not just single inference latency.

• No need for training from scratch: as a post-training framework, it can be optimized based on existing VLA models, reducing data collection and training costs.

• Balances speed and success rate: reliable horizon expansion avoids blindly lengthening chunks, and stability constraints prevent short-path speculation.

• Can be combined with computational acceleration: PolicyTrim optimizes the number of forward inferences, while methods like VLA-Cache optimize the time per inference; the two approaches are orthogonal and can produce compound acceleration.

The core insight of PolicyTrim is that for VLA robots, deployment efficiency comes not only from faster model inference but also from more efficient policy behavior. Making robots "take fewer detours" can also bring significant acceleration.

Reference: https://arxiv.org/abs/2606.22540

This article is from the WeChat public account "新智元", author: 新智元; editor: LRST