Designing and Developing Verifiably Safe AI Agents
- 10:00 4th June 2026 ( Trinity Term 2026 )Seminar Room 051
Artificial Intelligence is ubiquitous nowadays. Unfortunately, modern AI systems typically come with no guarantees about their safety, security, compliance, reliability, generalisability. To this aim, Formal Methods are increasingly applied to learning algorithms for their ability to provide strong theoretical guarantees. In this talk I will present some recent works of our lab on Formal Methods in AI about safe reinforcement learning, specifically on shielding RL agents against unsafe behaviours. Our results on this subject are not only of theoretical interest, but have also led to the development of practical safe RL algorithms, which have been implemented in the MASA library, providing a controllable tradeoff between safety and performance.