ReVision: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction
Authors: Amirhossein Abaskohi, Yuhang He, Peter West et al.
Summary
ReVision tackles the token cost problem for computer-use agents that observe screenshots over time. Each screenshot encodes into many visual tokens, so trajectories quickly become expensive. The authors train a patch selector that removes redundant visual patches across consecutive screenshots while preserving spatial structure. On Qwen2.5-VL-7B processing 5 history screenshots, ReVision cuts token usage by ~46% and improves success rate by 3%. Importantly, they find that performance keeps improving with more history when redundancy is removed, suggesting that visual history saturation isn't due to limited usefulness but inefficient representations.
Main takeaways:
- Computer-use agents process screenshots that encode into huge numbers of visual tokens, limiting how much history fits in context.
- ReVision trains a patch selector to remove redundant visual patches between consecutive frames while keeping spatial structure intact.
- Reduces tokens by ~46% on average across three benchmarks while improving success rate by 3% (Qwen2.5-VL-7B, 5 history screenshots).
- Performance continues improving with more history when redundancy is removed, challenging the idea that visual history has diminishing returns.
- Suggests commonly observed saturation is due to inefficient token representations, not limited usefulness of past observations.
Relevance
Not directly related to my persona/midtraining work—this is about vision-language efficiency for UI agents. Tangentially interesting for thinking about what information is redundant vs. necessary: my marker leakage results show that similar personas pick up markers from each other, which could be framed as the model treating semantically similar system prompts as "redundant" in some representation space.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses benchmark.
Abstract
arXiv:2605.11212v1 Announce Type: new Abstract: Computer-use agents~(CUAs) rely on visual observations of graphical user interfaces, where each screenshot is encoded into a large number of visual tokens. As interaction trajectories grow, the token cost increases rapidly, limiting the amount of history that can be incorporated under fixed context and compute budgets. This has resulted in no or very limited improvement in the performance when using history unlike other domains. We address this inefficiency by introducing ReVision, which is used to train multimodal language models on trajectories where redundant visual patches are removed using a learned patch selector that compares patch representations across consecutive screenshots while preserving spatial structure required by the model. Across three benchmarks, OSWorld, WebTailBench, and AgentNetBench, when processing trajectories with 5 history screenshots using Qwen2.5-VL-7B, ReVision reduces token usage by approximately 46% on average while improving success rate by 3% over the no drop baseline. This establishes a clear efficiency gain, enabling agents to process longer trajectories with fewer tokens. With this improved efficiency, we revisit the role of history in CUAs and find that performance continues to improve as more past observations are incorporated when redundancy is removed. This suggests that the commonly observed saturation in visual history is not due to limited usefulness of past information, but rather a consequence of inefficient token representations.