Security Risks in On-Device Machine Learning

The last several years have witnessed the promising growth of AI-empowered techniques in mobile devices, from the camera to smart assistants. Users can find traces of AI in almost every aspect of mobile devices. The global mobile artificial intelligence market has reached $8.56 billion in 2020 and is expected to have a growth rate of 6.44% from 2021 to 2030. As the technology keeps advancing, energy-friendly and low latency on-device AI solutions can be the entrance to the next level of development and innovation in the field like AR and autonomous driving and reducing the reliance on cloud AI operations. And using AI to contextualise user behaviours into applications will make each app session more valuable than the last. Compared with the previous ways of selling technology products, delivering compelling and personalised experiences and services has become an essential part of vendor roadmaps for upcoming years.

Based on this trend, as approaching the end of 2021, one of the most widely used deep learning frameworks Tensorflow has enabled its new feature of enabling on-device training in the Tensorflow Lite. Models can be easily finetuned locally to achieve better flexibility and personalization to users. However, on the other side, it also brings the question of the security and privacy issues of deep learning models to a new level and demand. In the past serval years, scholars have put attention on security issues for deep learning models like trojan attacks, information leakage, or adversarial attacks, and have come up with different ways of defending and preserving the robustness of deep learning models in inference time. It is also essential to get a better understanding of how reliable on current on-device training schema is and pave the way for further application of on-device AI models.