A Dataset and Multi-task 3D Visual Perception System for a Mobile Robot in Human Environments

Primary supervisor

Hamid Rezatofighi

To operate, interact and navigate safely in dynamic human environments, an autonomous agent, e.g. a mobile social robot, must be equipped with a reliable perception system, which is not only able to understand the static environment around it, but also perceive and predict intricate human behaviours in this environment while considering their physical and social decorum and interactions.

 

Student cohort

Double Semester

Aim/outline

Our aim is to design a multitask perception system for an autonomous agent, e.g. social robot. This framework includes different levels and modules, from basic-level perception problems to high-level perception and reasoning. This project also works on creating a large-scale dataset, used for the training and evaluation of such a multi-task perception system.

URLs/references

https://vl4ai.erc.monash.edu/research.html

https://arxiv.org/pdf/1910.11792.pdf

https://arxiv.org/pdf/2002.08397.pdf

https://jrdb.stanford.edu/

Required knowledge

  1. Good coding skills in a variety of coding languages
  2. Previous experience working with deep learning models for different tasks
  3. ​​​​​Proficient programming skills in Python and one of the main deep learning libraries (e.g., TensorFlow, PyTorch, Keras)