Skip to main content

Reinforcement Learning via Predictive Coding

Supervisors

Suitable for

MSc in Advanced Computer Science

Abstract

Learning an internal representation of an outside world is a challenging but key task for any agent that aims to interact with it. The subfield of machine learning that tackles this task, called reinforcement learning (RL), has achieved incredible performance on a large number of benchmarks by using an internal model of the world (or dataset) to determine a specific policy. However, agents trained with classic algorithms sometimes learn and are controlled in a non human-like way, and natural behaviors, vital to reach specific goals, are often hard-coded. In parallel, different computational models of action and planning in the brain have been developed by neuroscientists. Merging progress in AI and neuroscience has the potential to give new insight and more human-like behavior to RL algorithms. In a series of our recent publications, we have, for the first time, identified the initial evidence that a neuroscience-inspired learning method, called predictive coding, is potentially more powerful than the current foundation of AI, backpropagation. The goal of this project is to use predictive coding to perform RL experiments, in place of backpropagation. Particularly, the student is required to study and analyse the difference between this method and the ones already present in the literature in a variety of tasks, both theoretically and empirically. Prerequisites: Good programming skills, experience with reinforcement learning experiments. References: [1] David Ha, Jürgen Schmidhuber, World models, NeurIPS 2018 [2] Karl Friston et al. World model learning and inference, Neural Networks 2021 [3] Rajesh Rao, Dana Ballard Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects, Nature, 1999.