Skip to main content

Primary supervisor

Roisin McNaney

Co-supervisors


Voice assistants are software agents installed in devices such as phones, computers or tablets, or on purpose-built speakers. They are capable of interpreting human speech and, depending on the command they receive, can complete different tasks (e.g. tell the time or the weather, send and read text messages, make phone calls, set alarms, play music, and control various connected devices). Currently, one in five homes in the UK own a voice assisted speaker, a figure which is predicted to rise significantly. As many as 40% in the US own one. As such, these voice-assisted technologies are growing in popularity and are becoming pervasive. The older population (60+) make up around 20% of smart speaker ownership, with almost 60% of these consumers using the device every day. The Amazon Alexa is the market leader across all age groups. Voice assisted devices offer hands-free access and naturalistic voice interaction, which is a beneficial means of interacting with the device for those with physical disabilities or lower levels of technology literacy. As such, recent years have seen an emergence in research in the health and care space, which is exploring the role of voice assisted devices in supporting people within these demographics.

However, while developing apps (or skills) for commercial voice assistants like Alexa is relatively simple (with the open source Alexa skill SDK), capturing useful data from the technology for research purposes is more of a challenge. For example, command logs (e.g. 'Alexa, what time is it'; 'Alexa, what's the weather like today?'), which allow us to look at user's interactions with the device, can be extracted from a user through a companion app. However, we cannot yet access raw audio data, which could give us a much more nuanced understanding of the types of errors users are making (which could be linked to declining cognition in dementia, or give us an understanding of performance when providing interventions for speech and language therapy). 

This study will develop software for a companion device, to be used for research purposes, which will provide access to raw data captured by the Alexa device. It will need to consider the privacy and security concerns of users and be designed to address these concerns. 

Aim/outline

Main aim:

To design and develop a privacy sensitive companion device which can be used alongside commercial voice assistants to capture raw audio data for research purposes. 

Secondary aims: 

  1. Understand the privacy and security concerns of voice assistant users relating to the collection of their audio data in home settings 
  2. Develop software that can be run on a smart speaker which will capture raw audio data for later analysis 
  3. Conduct user studies which engage end users in exploring their data and any existing privacy and security concerns 

 

URLs/references

 

 

Required knowledge

Good basic knowledge of signal processing  

Strong programming skills (Python or C++)