Skip to main content

Ensuring Mobile App Accessibility with Deep Learning

Primary supervisor

Chunyang Chen

Research area

Software Engineering

According to Australian Network on Disability, over 4 million people (about 20% of the whole population) in Australia have some form of disability. At the same time, it is estimated that 15% Australians were aged 65 and over. For people with disabilities and older users, mobile phones and other mobile devices can provide increased freedom by allowing users to act independently while remaining in contact with friends, family, and caregivers.  However, studies of relatively small groups of mobile apps found that there still exists significant accessibility barriers. Imagine the frustrating UX experience of an elderly person with arthritis whose soft hand movements are not registered by her smartphone’s touchscreen. Or a color-blind user is unable to distinguish between the different color-coded routes in a transportation app. This suggests a continuing need for accessibility improvements.

Within this project, we will first carry out a large-scale accessibility analysis in real-world Android apps. Based on the findings from this empirical study, we will further develop algorithms and tools to enhance the accessibility of mobile apps. For development team including designers and developers, we will propose a model to proactively locate accessibility issues and recommend potential fixes to their app based on deep learning models. On the other hand, we will also propose a new approach based on the program analysis to deploy interaction proxies for runtime accessibility repair which could allow individuals or communities to quickly address accessibility failures that an app’s original developer is unable or unwilling to address.

Required knowledge

  • Strong programming background.
  • Basic understanding of Deep Learning and Human-Computer Interaction.
  • Experience of mobile app development and accessibility will be a big plus.

Learn more about minimum entry requirements.