Explainability and Compact representation of K-MDPs

Markov Decision Processes (MDPs) are frameworks used to model decision-making in situations where outcomes are partly random and partly under the control of a decision maker. While small MDPs are inherently interpretable for people, MDPs with thousands of states are difficult to understand by humans. The K-MDP problem is the problem of finding the best MDP with, at most, K states by leveraging state abstraction approaches to aggregate states into sub-groups. The aim of this project is to measure and improve the interpretability of K-MDP approaches using state-of-the-art XAI approaches. We will instantiate and evaluate our approaches on a range of computational sustainability case studies from the domain of conservation of biodiversity, natural resource management and behavioural ecology. 

Relevant Publications: 

1. Ferrer-Mestres, J., Dietterich, T. G., Buffet, O., & Chades, I. (2020, June). Solving k-mdps. In Proceedings of the International Conference on Automated Planning and Scheduling (Vol. 30, pp. 110-118).
2. Ferrer-Mestres, Jonathan, Thomas G. Dietterich, Olivier Buffet, and Iadine Chades. "Interpretable Solutions for Stochastic Dynamic Programming." bioRxiv (2024): 2024-08.
3. Wells, Lindsay, and Tomasz Bednarz. "Explainable ai and reinforcement learning—a systematic review of current approaches and trends." Frontiers in artificial intelligence 4 (2021): 550030.