Reinforcement Learning via Predictive Coding
Supervisors
Suitable for
Abstract
Learning an internal representation of an outside world is a challenging but key task for any agent that aims to interact
with it. The subfield of machine learning that tackles this task, called reinforcement learning (RL), has achieved incredible
performance on a large number of benchmarks by using an internal model of the world (or dataset) to determine a specific policy.
However, agents trained with classic algorithms sometimes learn and are controlled in a non human-like way, and natural behaviors,
vital to reach specific goals, are often hard-coded. In parallel, different computational models of action and planning in
the brain have been developed by neuroscientists. Merging progress in AI and neuroscience has the potential to give new insight
and more human-like behavior to RL algorithms. In a series of our recent publications, we have, for the first time, identified
the initial evidence that a neuroscience-inspired learning method, called predictive coding, is potentially more powerful
than the current foundation of AI, backpropagation. The goal of this project is to use predictive coding to perform RL experiments,
in place of backpropagation. Particularly, the student is required to study and analyse the difference between this method
and the ones already present in the literature in a variety of tasks, both theoretically and empirically.
Prerequisites: Good programming skills, experience with reinforcement learning experiments.
References: [1] David Ha, Jürgen Schmidhuber, World models, NeurIPS 2018 [2] Karl
Friston et al. World model learning and inference, Neural Networks
2021 [3] Rajesh Rao, Dana Ballard Predictive coding in the visual cortex: a functional interpretation of some extra-classical
receptive-field effects, Nature, 1999.