Guarding On-device Machine Learning Models via Privacy-enhancing Techniques

 On-device machine learning (ML) is rapidly gaining popularity on mobile devices. Mobile developers can use on-device ML to enable ML features at users’ mobile devices, such as face recognition, augmented virtual reality, voice assistance, and medical diagnosis. This new paradigm is further accelerated by AI chips and ASICs embedded on mobile devices, e.g., Apple’s Bionic neural engine. Compared to cloud-based machine learning services, on-device ML is privacy-friendly, of low latency, and can work offline. User data will remain at the mobile device for ML inference.

Problems: In order to enable on-device ML, models must be deployed at the local mobile devices, thereby creating a new attack surface inevitably. Commercial ML models are now stored on mobile devices, which is completely out of the control of the model owners. They can be easily stolen or abused if not properly protected. A recent empirical study [1] has demonstrated that half of on-device ML models are stored in plaintext form at the devices, and even if encryption is used to protect ML models, those models can be extracted during dynamic analysis.

To secure on-device ML models, in this project, we aim to employ privacy-enhancing techniques to design new and efficient algorithms that can provide strong protection during the entire life cycle of ML models used on the devices. 

Research Task I: Investigate ML algorithms and optimisations which are friendly to privacy-enhancing techniques and on-device ML, including but not limited to model compression, quantisation, distillation, transfer learning, pruning, etc.

Research Task II: Apply privacy-enhancing techniques such as secure multi-party computation, homomorphic encryption, differential privacy, and trusted execution to design algorithms and protocols to secure ML models within their life cycle at mobile devices.

Research Task III: Conduct theoretical and experimental analysis for the performance of the proposed designs as well as the security against attacks on on-device ML.

Impact: The outcome of this project will directly contribute to the roadmap of securely developing on-device ML applications and systems, i.e., preventing capital loss of companies and protecting the data of data owners and end-users involved in on-device training and inference. The outcome is of excellent commercialisation value given the increasing popularity of on-device applications using ML models for decision making. 

[1] Sun et al., "Mind your weight(s): A large-scale study on insufficient machine learning model protection in mobile apps.", USENIX Security, 2021.