Clustering of (time series of) generalised dynamic Bayesian nets, etc.

The relationship between the information-theoretic Bayesian minimum message length (MML) principle and the notion of Solomonoff-Kolmogorov complexity from algorithmic information theory (Wallace and Dowe, 1999a) ensures that - at least in principle, given enough search time - MML can infer any underlying computable model in a data-set.

A consequence of this is that we can (e.g.)

• put latent factor models (Wallace and Freeman, 1992; Wallace, 2005, sec. 6.9) into mixture models (Wallace and Dowe, 1994b; Wallace and Dowe, 2000) to do clustering with latent factor models (Edwards and Dowe, 1998),
• put decision trees (Wallace and Patrick, 1993) into Bayesian nets to give generalised hybrid Bayesian nets (Comley and Dowe, 2003; Comley and Dowe, 2005),
• combine autoregressive time series (Fitzgibbon, Dowe and Vahid, 2004) and clustering (Wallace and Dowe, 1994b; Wallace and Dowe, 2000) to cluster time series (Molloy, Albrecht, Dowe and Ting, 2006),
• etc.

Indeed, using the relationship between MML and Solomonoff-Kolmogorov complexity (Wallace and Dowe, 1999a), it is possible to (e.g.) create a general hybrid of (none, some or) all of the above methods - and then (if we wish) to generalise that even further.

This project concerns creating suitable models - and/or general hybrid models - as yet undeveloped.

References:

 Comley, Joshua W. and D.L. Dowe (2003). General Bayesian Networks and Asymmetric Languages, Proc. 2nd Hawaii International Conference on Statistics and Related Fields, 5-8 June, 2003

 Comley, Joshua W. and D.L. Dowe (2005). ``Minimum Message Length and Generalized Bayesian Nets with Asymmetric Languages'', Chapter 11 (pp265-294) in P. Gru:nwald, I. J. Myung and M. A. Pitt (eds.), Advances in Minimum Description Length: Theory and Applications, M.I.T. Press (MIT Press), April 2005, ISBN 0-262-07262-9. [Final camera ready copy was submitted in October 2003.]

  D. L. Dowe (2008a), "Foreword re C. S. Wallace", Computer Journal, Vol. 51, No. 5 (Sept. 2008) [Christopher Stewart WALLACE (1933-2004) memorial special issue [and front cover and back cover]], pp523-560 (and here). www.doi.org: 10.1093/comjnl/bxm117

  D. L. Dowe (2011a), "MML, hybrid Bayesian network graphical models, statistical consistency, invariance and uniqueness", Handbook of the Philosophy of Science - (HPS Volume 7) Philosophy of Statistics, P.S. Bandyopadhyay and M.R. Forster (eds.), Elsevier, [ISBN: 978-0-444-51862-0 {ISBN 10: 0-444-51542-9 / ISBN 13: 978-0-444-51862-0}], pp901-982, 1/June/2011

  Dowe, D.L., S. Gardner and G.R. Oppy (2007) (Dec. 2007), "Bayes Not Bust! Why Simplicity is no problem for Bayesians", in Brit. J. Philos. Sci. (BJPS), Vol. 58, No. 4 (December 2007), pp709-754

 David L. Dowe and Nayyar A. Zaidi (2010), "Database Normalization as a By-product of Minimum Message Length Inference", Proc. 23rd Australian Joint Conference on Artificial Intelligence (AI'2010) [Springer Lecture Notes in Artificial Intelligence (LNAI), vol. 6464], Adelaide, Australia, 7-10 December 2010, Springer, pp82-91

  R T Edwards and D L Dowe (1998). "Single factor analysis in MML mixture modelling", pp96-109, 2nd Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD98), Lecture Notes in Artificial Intelligence (LNAI) 1394, Melbourne, Australia, April 1998, Springer-Verlag

  Fitzgibbon, L.J., D. L. Dowe and F. Vahid (2004). Minimum Message Length Autoregressive Model Order Selection. In M. Palanaswami, C. Chandra Sekhar, G. Kumar Venayagamoorthy, S. Mohan and M. K. Ghantasala (eds.), International Conference on Intelligent Sensing and Information Processing (ICISIP), Chennai, India, 4-7 January 2004 (ISBN: 0-7803-8243-9, IEEE Catalogue Number: 04EX783), pp439-444.

  Molloy, S., D.W. Albrecht, D. L. Dowe and K.M. Ting (2006). Model-Based Clustering of Sequential Data, Proc. 5th Annual Hawaii Intl. Conf. on Statistics, Mathematics and Related Fields, 22 pages, 16th - 18th January, 2006, Hawaii, U.S.A.

  P. J. Tan and D. L. Dowe (2003). MML Inference of Decision Graphs with Multi-Way Joins and Dynamic Attributes, Proc. 16th Australian Joint Conference on Artificial Intelligence (AI'03), Perth, Australia, 3-5 Dec. 2003, Published in Lecture Notes in Artificial Intelligence (LNAI) 2903, Springer-Verlag, pp269-281

  Wallace, C.S. (2005), ``Statistical and Inductive Inference by Minimum Message Length'', Springer  (Link to the preface [and p vi, also here])

  Wallace, C.S. and D.L. Dowe (1994b), Intrinsic classification by MML - the Snob program. Proc. 7th Australian Joint Conf. on Artificial Intelligence, UNE, Armidale, Australia, November 1994, pp37-44

  Wallace, C.S. and D.L. Dowe (1999a). Minimum Message Length and Kolmogorov Complexity, Computer Journal (special issue on Kolmogorov complexity), Vol. 42, No. 4, pp270-283

  Wallace, C.S. and D.L. Dowe (2000). MML clustering of multi-state, Poisson, von Mises circular and Gaussian distributions, Statistics and Computing, Vol. 10, No. 1, Jan. 2000, pp73-83

  Wallace, C.S. and P.R. Freeman (1987).  Estimation and inference by compact coding. J. Royal Statist. Soc. B, 49, 240–252.