The notion of simplicity can occur in many contexts. Sometimes simplicity can be explicitly sought so that the number of variables to be considered is manageable, and sometimes simplicity can arise as a consequence of other desiderata. In the context of machine learning, many approaches propose to learn simple models since judicious simplicity can lead to good predictions [3,1]. These approaches can be used to learn simple models (e.g., decision trees [6], decision graphs [4, 5], etc.) or improve more complex models (e.g., neural networks [2]).
Recent advancements in mathematical optimisation have unlocked the potential to solve many challenging problems in machine learning to optimality. Such approaches typically balance the optimisation of total accuracy and model complexity according to an informal interpretation of the Ockham's Razor. This project aims to explore how optimisation can be used to solve challenging machine learning problems that is consistent with formal interpretations of simplicity.
References:
[1] Dowe, D.L., S. Gardner and G.R. Oppy (2007) (Dec. 2007), "Bayes Not Bust! Why Simplicity is no problem for Bayesians", in Brit. J. Philos. Sci. (BJPS), Vol. 58, No. 4 (December 2007), pp709-754
[2] Fang Z, Dowe D L, Peiris S, Rosadi D (2021), "Minimum Message Length in Hybrid ARMA and LSTM Model Forecasting", Entropy, 23(12):1601, 2021.
[3] Needham, S.L. and D.L. Dowe (2001), "Message Length as an Effective Ockham's Razor in Decision Tree Induction", Proc. 8th International Workshop on Artificial Intelligence and Statistics (AI+STATS 2001), pp253-260, Key West, Florida, U.S.A., Jan. 2001.
[4] J J Oliver and C S Wallace (1991), “Inferring decision graphs”, IJCAI '91 Workshop 8, Sydney, 1991.
[5] P. J. Tan and D. L. Dowe (2003). "MML Inference of Decision Graphs with Multi-Way Joins and Dynamic Attributes", Proc. 16th Australian Joint Conference on Artificial Intelligence (AI'03), Perth, Australia, 3-5 Dec. 2003, Published in Lecture Notes in Artificial Intelligence (LNAI) 2903, Springer-Verlag, pp269-281
[6] C S Wallace, J D Patrick (1993), “Coding Decision Trees”, Machine Learning 11, 7-22, 1993.
Required knowledge
Required knowledge: A successful candidate should have:
- a degree in Computer Science, Information Technology Engineering, or equivalent,
- excellent mathematical and analytical skills,
- excellent skills in machine learning and mathematical optimisation,
- excellent communication skills (i.e., both written and verbal),
- the ability to work independently, and
- the ability to collaborate with a team of researchers and scientists.
