Skip to main content

Primary supervisor

Chunyang Chen

Although we normally use our fingers to manipulate apps in the smart phone, that operation may not apply in some special situation e.g., checking information when holding your kids, driving, people with disability, etc. In those special scenarios, controlling the app with voice will make it much more convenient. In this study, we are trying to develop a tool to automatically map your voice to operations in mobile apps. You will work with one of my PhD students and me to deliver this exciting work which can be both published in academic and applied into the practice.

Student cohort

Double Semester

Aim/outline

  • Convert speech to text by using existing service like Google Assistant.
  • Understand the text and convert it to the actions for mobile apps.
  • Send actions to mobile apps for finishing users' tasks.

URLs/references

  • https://events.google.com/io/session/92fbcb38-9597-4cc4-9662-2a8787bb2410?lng=en
  • https://developers.google.com/assistant/conversational/actions

Required knowledge

  • Great coding skills
  • Android development
  • Basic knowledge about natural-language processing