This project will explore the use of Mixed-Reality (MR) headset technology to support people in performing maintenance tasks in complex environments, where the nature of the work involves close inspection of and interaction with mechanical devices. Examples might include aircraft maintenance or other complex workshop environments. We term work in such situations as "physically embedded" in that the nature of the workflow and the information and data associated with the work is closely tied to the physical machinery. Such maintenance support requires providing the worker with timely and relevant information in the spatial context of the task at hand. To achieve this the computer guidance system needs to be aware of the environment through a rich digital-twin model that is kept up-to-date in the face of dynamic change (for example, new data concerning faults or context). The ultimate aim is to create an effective "human-machine" symbiosis in which the worker is able to work with the machine guidance system, informed via MR of information necessary to complete the task, but also able to supervise any machine-learning or decision-making processes.
This is an ambitious goal involving many sub-problems concerning: activity capture (incorporating computer vision and sensor information to map and identify changes in the environment); activity representation (modelling of the activity and environment to achieve a useful digital twin); and instructional design (effective interactive visualisation and information overlay techniques to guide/instruct the user). The scope of the PhD will be narrowed to address the most pressing and achievable research challenges in this topic as they become apparent.
Required knowledge
Top-up funding will be available from DST
