Reinforcement Learning for Healthcare

Many health settings involve making a sequence of decisions (e.g. sequencing treatments for HIV to manage viral loads now and limit cross-resistance later).  The number of decisions makes these settings challenging to explore with traditional clinical trials; it would be helpful to know what kinds of sequences are most promising to explore. Batch Reinforcement Learning (RL) aims to propose novel treatment policies based solely on existing data -- e.g. the longitudinal views of a patient via their health records.  In this talk, I will discuss batch RL algorithms that we have developed and applied in the context of hypotension management in the ICU as well as managing HIV.  I will also discuss the limitations of these methods and our work to move beyond fundamental statistical limitations by seeking and integrating different kinds of validation by domain experts.  

This work is in collaboration with Srivatsan Srinivasan, Isaac Lage, Dafna Lifshcitz, Ofra Amir, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, Xuefeng Peng, David Wihl, Yi Ding, Omer Gottesman, Liwei Lehman, Matthieu Komorowski, Aldo Faisal, David Sontag, Fredrik Johansson, Leo Celi, Aniruddh Raghu, Yao Liu, Emma Brunskill, and the CS282 2017 Course.