On-device Machine Learning

Co-supervisors

With the glow of digital information techniques, mobile systems are powerful ever and occupying more market shares. Just like wildly used social media sites, e.g. Facebook and Twitter, smartphone usage is up to 80% by 2020. In parallel to this trend, many companies are trying to incorporate Artificial Intelligent especially deep learning empowered applications into devices to further ease the life of people. At an early stage, the models are offloaded to the cloud and require frequent data transfer to perform inference since in the mobile end the high computational consumption and complex structure of deep learning models are not affordable. In recent years they are trying to lighten the deep learning models to enable inference and even training at mobile devices directly. Techniques like pruning, quantizing, knowledge distillation are developed to delimitate computation labour of computer vision (CV) tasks. In addition to the help of a new mobile end, deep learning framework (Tensorflow lite, PytorchMobile, CoreML) makes the market of on-device applications getting bigger and bigger due to the low latency and better privacy protection for individuals. 

 

    Student cohort

    Single Semester
    Double Semester

    Aim/outline

    In Natural Language Processing, despite the many attempts inspired by CV, compressed language models still remain gaps for deploying the application on-device. Powerful pre-trained language models are too big and complicated (Bert-base over 303M parameters and 400 M storage space) to deploy. Several early proposals have addressed the trend that on-devices for NLP are coming, there is still no real application of the on-device language model on the market.  

    In this project, we aim to  

    1. Empirically analyze what is the true difficulty that prevents the language model from a real application. 
    2. Comparative study of language model deployment in the pre-trained language model  
    3. Explore and formulate requirement of deployment NLP application on the product side to give an insight of orientations of a commercial company to work towards from both user and customer side.

    Required knowledge

    1. Knowledge of deep learning, NLP 
    2. Experience in python, TensorFlow