AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites

Authors: Qinshi Zhang (University of California, San Diego), Weipeng Deng (University of Hong Kong) et al.

arXiv · PDF

Summary

The authors tackle a problem in model-based reinforcement learning: standard world models learn correlations between actions and outcomes but ignore whether an action is actually executable right now (its preconditions). They propose AGWM, which adds an explicit affordance structure—a directed acyclic graph tracking which actions become available or blocked as the agent changes the environment—so the model can track whether each action is currently executable, reducing compounding prediction errors over multi-step rollouts.

Main takeaways:

• Standard world models confuse correlation with causation: if action A and outcome B co-occur frequently in training, the model assumes A always causes B, ignoring prerequisites.
• In interactive environments, actions can change what's possible next ("structure-changing events")—e.g., unlocking a door makes "walk through door" executable.
• AGWM adds a directed acyclic graph that explicitly tracks prerequisite dependencies, so the model knows which actions are executable at each timestep.
• This reduces multi-step prediction error, improves generalization to novel environment configurations, and makes the model more interpretable.
• Tested in game-based simulated environments where action availability changes dynamically.

Relevance

Not directly connected to my persona-installation or behavioral-transfer work. This is about RL agents learning environment dynamics, not about language model conditioning or persona state.

Abstract

arXiv:2605.06841v1 Announce Type: new Abstract: In model-based learning, the agent learns behaviors by simulating trajectories based on world model predictions. Standard world models typically learn a stationary transition function that maps states and actions to next states, when an action and an outcome frequently co-occur in training data, the model tends to internalize this correlation as a general causal rule while ignoring action preconditions. In interactive environments, however, agent actions can reshape the future affordance space. At each timestep, an action may becomes executable only after its prerequisites are met, or non-executable when they are destroyed. We term such events structure-changing events (SC events). As a result, a conventional world model often fails to determine whether a given action is executable in the current state, especially in multi-step predictions. Each imagined step is conditioned on an incorrect affordance state, and therefore the prediction error compounds over the rollout horizon. In this paper, we propose AGWM (Affordance-Grounded World Model), which learns an abstract affordance structure represented as a DAG of prerequisite dependencies to explicitly track the dynamic executability of actions. Experiments on game-based simulated environments demonstrate the effectiveness of our method by achieving lower multi-step prediction error, better generalization to novel configurations, and improved interpretability.