Joint BCAM-UPV/EHU Data Science and Artificial Intelligence seminar: Self-Composing Policies for Scalable Continual Reinforcement Learning

LOCATION: UPV/EHU Donosti, Faculty of Computer Science, room 3.1 and Online

Link to the session here

Abstract
Continual reinforcement learning aims to develop agents that learn in a never-ending stream of tasks and leverage the knowledge obtained from solving previous problems to solve new ones. However, allowing such knowledge transfer while avoiding catastrophic forgetting and interference is one of the main challenges of the field. In this work, we propose a growable and modular Neural Network (NN) architecture that naturally avoids the mentioned issues by instantiating a new policy module every time a new task is introduced. Moreover, the NN architecture of each module enables selectively composing preceding policies together with its internal policy for the purpose of accelerating solving the current task. Conducted experiments show that the proposed architecture is able to transfer knowledge in sequences of continuous control problems as well as in visual control tasks, such as MuJoCo and Atari. Unlike previous growing NN approaches, we also show that the number of parameters of the proposed approach grows linearly with respect to the number of tasks, and does not sacrifice plasticity to scale. Finally, we shed some light on the possibility of using the presented method to perform knowledge transfer across different Atari games.

Comienza la navegacion principal

1. General information