Skip to main content

A Computer Vision Project: Generalized Learning for Image-based Planning

Primary supervisor

Buser Say

Co-supervisors


Planning is the reasoning side of acting in Artificial Intelligence. Planning automates the selection and the organization of actions to reach desired states of the world as best as possible. For many real-world planning problems however, it is difficult to obtain a transition model that governs state evolution with complex dynamics. Fortunately as visualized in Figure 1, recent works have shown that the unknown transition models can be accurately approximated as (deep) neural networks which then can be compiled into mathematical optimization models (e.g., MILP, Weighted Partial MaxSAT, pseudo-Boolean optimization etc.) over a fixed horizon, and solved optimally using off-the-shelf solvers.

Figure 1: Visualization of the learning and planning framework introduced previously [1] where red circles represent action variables, blue circles represent state variables, gray circles represent the activation units and w's represent the weights of the neural network.
Figure 1: Visualization of the learning and planning framework where red circles represent action variables, blue circles represent state variables, gray circles represent the activation units and w's represent the weights of the neural network.

 

Student cohort

Double Semester

Aim/outline

One limitation of this learning and planning framework is that the learned transition models do not generalize over multiple problem instances. In this project, you will work on learning generalizable image-based deep neural network transition models that are amendable to automated planning. Please note that this is a deep learning/computer vision project, and can be extended to a fully-funded PhD project in the future.

Required knowledge

A successful candidate should have strong programming skills (e.g., in Python) as well as background in deep learning (ideally with applications to image-based tasks).