Weblica: Scalable and Reproducible Training Environments for Visual Web Agents

Authors: O\u{g}uzhan Fatih Kar, Roman Bachmann, Yuanzheng Gong et al.

arXiv · PDF

Summary

The authors built Weblica, a framework for creating reproducible and scalable web navigation environments by capturing real websites at the HTTP level (so interactions replay consistently) and using LLMs to synthesize thousands of diverse navigation tasks grounded in real-world sites. They trained an RL agent on these environments, producing Weblica-8B, which outperforms similar-sized open-weight baselines on multiple web navigation benchmarks while using fewer inference steps and scaling well with test-time compute.

Main takeaways:

• Existing web-agent training is limited to offline trajectories or a handful of simulated sites; Weblica scales to thousands of diverse, reproducible environments.
• HTTP-level caching captures stable visual states and interactive behavior from real websites, enabling consistent replay for RL training.
• LLM-based synthesis generates diverse tasks grounded in real-world websites and core navigation skills (form-filling, search, multi-step goals).
• Weblica-8B outperforms open-weight baselines of similar size, uses fewer inference steps, and is competitive with API models.
• Scales favorably with additional test-time compute (more search/rollouts improve performance).

Relevance

Not directly related to my persona/midtraining work—this is about training web navigation agents via RL in scalable simulated environments. Only tangentially relevant if I ever explore how persona installation affects agent behavior in interactive domains, but that's not my current focus.

Abstract

arXiv:2605.06761v1 Announce Type: new Abstract: The web is complex, open-ended, and constantly changing, making it challenging to scale training data for visual web agents. Existing data collection attempts remain limited to offline trajectories for supervised fine-tuning or a handful of simulated environments for RL training, thus failing to capture web diversity. We propose Weblica (Web Replica), a framework for constructing reproducible and scalable web environments. Our framework leverages 1) HTTP-level caching to capture and replay stable visual states while preserving interactive behavior and 2) LLM-based environment synthesis grounded in real-world websites and core web navigation skills. Using this framework, we scale RL training to thousands of diverse environments and tasks. Our best model, Weblica-8B, outperforms open-weight baselines of similar size across multiple web navigation benchmarks while using fewer inference steps, scales favorably with additional test-time compute, and is competitive with API models.