Learning and Reusing Policy Decompositions for Hierarchical Generalized Planning with LLM Agents

Authors: Shirin Sohrabi, Haritha Ananthakrishnan, Harsha Kokel et al.

arXiv · PDF

Summary

The authors build an LLM-based agent system that learns reusable task decompositions: when the agent successfully completes a task, it automatically extracts reusable components (sub-policies) and stores them in a library for future use via semantic search. This combines classical planning ideas (hierarchical task decomposition, generalized policies) with LLM agents. On the AppWorld benchmark (tasks involving real app interactions), their approach achieves 98.2% accuracy on normal tasks and 97.8% on challenge tasks with unseen apps, improving 15.8 points over static approaches on hard cases; for open-source models, dynamic component reuse enables 62.5% success versus near-zero without it.

Main takeaways:

• The system learns parameterized policies that generalize across task instances and automatically decomposes successful executions into reusable components
• Components are stored in a library and retrieved via semantic search when facing new tasks
• Addresses three challenges: automated decomposition of successful trajectories, generalizing components to maximize reuse, and efficient retrieval
• On AppWorld benchmark with unseen applications, achieves 97.8-98.2% accuracy (15.8 point improvement over static synthesis on challenging scenarios)
• For weaker open-source models, component reuse is the difference between 62.5% success and near-zero

Relevance

Tangential—this is about learning and composing task-solving behaviors, whereas my work focuses on how persona-like behaviors are installed and conditioned during training. The notion of reusable behavioral components is conceptually related to personas, but the context (hierarchical task decomposition) is quite different from what I'm studying.

Abstract

arXiv:2605.06957v1 Announce Type: new Abstract: We present a dynamic policy-learning approach that combines generalized planning and hierarchical task decomposition for LLM-based agents. Our method, Hierarchical Component Learning for Generalized Policies (HCL-GP ), learns parameterized policies that generalize across task instances and automatically extracts reusable components from successful executions, organizing them into a component library for compositional policy generation. We address three challenges: (1) learning components through automated decomposition, (2) generalizing components to maximize reuse, and (3) efficient retrieval via semantic search. Evaluated on the AppWorld benchmark, our approach achieves 98.2% accuracy on normal tasks and 97.8% on challenge tasks with unseen applications, improving 15.8 points over static synthesis on challenging scenarios. For open-source models, dynamic reuse enables 62.5% success versus near-zero without reuse. This demonstrates that classical planning concepts can be effectively integrated with LLM agents for improved accuracy and efficiency.