Automated Mobile App Development with Deep Learning

Mobile apps now have become the most popular way of accessing the Internet as well as performing daily. Different from traditional desktop applications, mobile apps are typically developed under the time-to-market pressure and facing fierce competition — over 3.8 million Android apps and 2 million iPhone apps are striving to gain users on Google Play and Apple App Store, the two primary mobile app markets. Therefore, for app developers and companies, it is crucial to accelerate the mobile app development process.

Toward that goal, we have finished a series of works in automated some app development process by leveraging deep learning including automated app functionality summarization [1], UI design generation [2], and front-end code generation [3]. We hope to explore more in this direction including functional/backend code generation and automated app testing with sophisticated machine learning models.

[1] Chen, S., Fan, L., Chen, C., Su, T., Li, W., Liu, Y. and Xu, L., 2019, May. Storydroid: Automated generation of storyboard for Android apps. In Proceedings of the 41st International Conference on Software Engineering (pp. 596-607). IEEE Press. https://arxiv.org/pdf/1902.00476.pdf

[2] Chen, C., Feng, S., Xing, Z., Liu, L., Zhao, S. and Wang, J., 2019. Gallery DC: Design Search and Knowledge Discovery through Auto-created GUI Component Gallery. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), pp.1-22. https://chunyang-chen.github.io/publication/designGallery_CSCW19.pdf

[3] Chen, C., Su, T., Meng, G., Xing, Z. and Liu, Y., 2018, May. From ui design image to gui skeleton: a neural machine translator to bootstrap mobile gui implementation. In Proceedings of the 40th International Conference on Software Engineering (pp. 665-676). ACM. https://chunyang-chen.github.io/publication/ui2code.pdf