Primary supervisor
Mor VeredCo-supervisors
- Iadine Chades
Markov Decision Processes (MDPs) are frameworks used to model decision-making in situations where outcomes are partly random and partly under the control of a decision maker. While small MDPs are inherently interpretable for people, MDPs with thousands of states are difficult to understand by humans. The K-MDP problem is the problem of finding the best MDP with, at most, K states by leveraging state abstraction approaches to aggregate states into sub-groups. The aim of this project is to measure and improve the interpretability of K-MDP approaches using state-of-the-art XAI approaches. We will instantiate and evaluate our approaches on a range of computational sustainability case studies from the domain of conservation of biodiversity, natural resource management and behavioural ecology.
Student cohort
URLs/references
Relevant Publications:
- Ferrer-Mestres, J., Dietterich, T. G., Buffet, O., & Chades, I. (2020, June). Solving k-mdps. In Proceedings of the International Conference on Automated Planning and Scheduling (Vol. 30, pp. 110-118).
- Ferrer-Mestres, Jonathan, Thomas G. Dietterich, Olivier Buffet, and Iadine Chades. "Interpretable Solutions for Stochastic Dynamic Programming." bioRxiv (2024): 2024-08.
- Wells, Lindsay, and Tomasz Bednarz. "Explainable ai and reinforcement learning—a systematic review of current approaches and trends." Frontiers in artificial intelligence 4 (2021): 550030.
Required knowledge
- Strong background in computer science in general
- Familiarity and understanding of basic principles underlying automated reasoning, MDPs and RL
- C/C++ programming and knowledge
- Python programming