Variational bayesian algorithms for generative topographic mapping and its extensions

Resumen

This thesis is the result of our general interest in the study of the Generative Topographic Mapping (GTM), a non-linear Latent Variable Model originally proposed as a probabilistic alternative to the well-known Self-Organizing Maps to visualize and cluster high dimensional data by extracting their low-dimensional hidden inherent structures, Over the last decade, it has been extended to tackle other data problems, and it has been applied in a wide variety of areas. The standard GTM (for multivariate static data) as well as the GTM Through Time (for multivariate time series) make use of the Maximum Likelihood framework through the Expectation-Maximization (EM) algorithm to compute the local-optimum values of its parameters. However, this approximation is often too drastic to handle the high-dimensional, multi-modal and strongly correlated data that can be encountered, therefore risking data overfitting. In this thesis, we first present an exhaustive set of experiments to assess the capabilities of the standard GTM Through Time model for clustering and visualization of multivariate time series. A novel index of variability that allows measuring the degree of variability of a subsequence is introduced, and an unsupervised Time Series Relevance Determination method is proposed. The latter allows ranking the time series of a dataset in terms of their relevance for the clustering of subsequences. The fully Bayesian modelling of the GTM, as well as its implementation through a variational Bayesian approach, namely the Variational Bayesian GTM (VBGTM), constitute novel and significant contributions of this thesis. Despite the fact that the risk of data overfitting in standard GTM was known from inception, this thesis, to the best of our knowledge, provides the first complete solution to this problem. The Variational Bayesian GTM-TT (VBGTM-TT), an elegant solution to deal with the data overfitting problem in GTM Through Time for the analysis of multivariate time series, is also defined. The high risk of data overfitting due to the elevated number of free parameters was one of the most relevant weakness of the original GTM Through Time model. Several experiments using artificial and real data show that the proposed VBGTM and VBGTM-TT models outperform their standard counterparts, GTM and GTM-TT, respectively, in terms of generalization capabilities and data visualization. Finally, some of the many possible novel extensions of GTM to be developed within the proposed Variational Bayesian framework, focusing on unsupervised relevance determination in some detail, are also outlined in the closing chapters of the thesis.