loading subjects...
Ebook: Integrating Planning and Learning for Agents Acting in Unknown Environments
loading...
An Artificial Intelligence (AI) agent can perceive an environment through sensors and act in the environment through actuators. When performing tasks in a known environment, an agent knows what actions it can execute and how they affect the environment state, but when the environment is unknown, the agent needs to learn how the environment works to make good decisions for accomplishing tasks. In a real-world situation, an agent may have only low-level perceptions of the environment rather than the high-level representations required to make decisions by means of symbolic planning.
This book, Integrating Planning and Learning for Agents Acting in Unknown Environments proposes an architecture that integrates learning, planning, and acting. The author, Leonardo Lamanna, won the 2023 Marco Cadoli award, an annual award from the Italian Association for Artificial Intelligence (AIxIA) for the best doctoral thesis in the field of artificial intelligence, for this work. The approach combines data-driven learning methods for building an environment model with symbolic planning techniques for reasoning on the learned model, focusing on learning the model, either from continuous or symbolic observations. The problem of online learning the mapping between continuous perceptions and symbolic states is tackled, and symbolic planning techniques are exploited to enable an agent to autonomously gather relevant information online, which is required by learning methods to overcome some of the simplifying assumptions of symbolic planning. The effectiveness of the approach in simulated complex environments is shown experimentally and the applicability of the approach in real environments is demonstrated by conducting experiments on a real robot.
Outperforming state-of-the-art methods, the approach described in this book will be of interest to all those working in the field of AI and autonomous agents.
An Artificial Intelligence (AI) agent acting in an environment can perceive the environment through sensors and execute actions through actuators. Symbolic planning provides an agent with decision-making capabilities about the actions to execute for accomplishing tasks in the environment. For applying symbolic planning, an agent needs to know its symbolic state, and an abstract model of the environment dynamics. However, in the real world, an agent has low-level perceptions of the environment (e.g. its position given by a GPS sensor), rather than symbolic observations representing its current state. Furthermore, in many real-world scenarios, it is not feasible to provide an agent with a complete and correct model of the environment, e.g., when the environment is unknown a priori. The gap between the high-level representations, suitable for symbolic planning, and the low-level sensors and actuators, available in a real-world agent, can be bridged by integrating learning, planning, and acting.
Firstly, an agent has to map its continuous perceptions into its current symbolic state, e.g. by detecting the set of objects and their properties from an RGB image provided by an onboard camera. Afterward, the agent has to build a model of the environment by interacting with the environment and observing the effects of the executed actions. Finally, the agent has to plan on the learned environment model and execute the symbolic actions through its actuators.
We propose an architecture that integrates learning, planning, and acting. Our approach combines data-driven learning methods for building an environment model online with symbolic planning techniques for reasoning on the learned model. In particular, we focus on learning the environment model, from either continuous or symbolic observations, assuming the agent perceptual input is the complete and correct state of the environment, and the agent is able to execute symbolic actions in the environment. Afterward, we assume a partial model of the environment and the capability of mapping perceptions into noisy and incomplete symbolic states are given, and the agent has to exploit the environment model and its perception capabilities to perform tasks in unknown and partially observable environments. Then, we tackle the problem of online learning the mapping between continuous perceptions and symbolic states, assuming the agent is given a partial model of the environment and is able to execute symbolic actions in the real world.
In our approach, we take advantage of learning methods for overcoming some of the simplifying assumptions of symbolic planning, such as the full observability of the environment, or the need of having a correct environment model. Similarly, we take advantage of symbolic planning techniques to enable an agent to autonomously gather relevant information online, which is necessary for data-driven learning methods. We experimentally show the effectiveness of our approach in simulated and complex environments, outperforming state-of-the-art methods. Finally, we empirically demonstrate the applicability of our approach in real environments, by conducting experiments on a real robot.