The aim of this course is to explore the analyzing methods under Multivariate techniques such as Multivariate multiple regressions, Correspondence Analysis, Multivariate Mixture Distribution, Clustering, Independent Component Analysis and Bayesian Multivariate Regression and Factor Analysis.

After completing this course, the students should

understand all the features of Multivariate multiple regression analysis;

apply and fit appropriate Multivariate multiple regression model according to the nature of data;

understand all the features of Multivariate Mixture Distribution and appropriate clustering;

understand how to separate the mixing signals through ICA; and

understand how to apply Bayesian method in Regression and Factor Analysis.

At the end of the course, the students will be able to know how to apply appropriate multivariate techniques to the data from a wide range of application area e.g., Neurology, Electroencephalographic recordings, Study of human brain, image analysis, Signal separation, Wireless communications, speech recognition, gene expression profile analysis, radiometric sky map, Volcanic eruption and climate change, Financial Econometrics, Business statistics, classifying and grouping sectors, Categorical data Management, Industrial Management and statistics, Bayesian prediction, noise analysis and so on.

Multivariate Regression Analysis: simple, multiple and multivariate multiple linear regression models. Assumptions. Parameter estimations and multivariate prediction. The distribution of likelihood ratio for the multivariate multiple regression model.  Likelihood ratio test (LRT) including other multivariate test procedures. Relationship with canonical correlation analysis (CCA). Interpretation and conclusion.

Correspondence Analysis: Concept of correspondence analysis (CA). Algebraic development of correspondence analysis. Multiple correspondence analysis (MCA).  Validation techniques in MCA. Similarities of CA and MCA with Categorical PCA and non-linear PCA. Application of CA for multiple factor analysis with contingency tables. Multiple factor analysis of mixed tables of metric and categorical data. Multi-block canonical correlation analysis for categorical variables. MCA and classification.

Multivariate Mixture Model: Definition and properties of finite mixture model. Maximum likelihood (ML) fitting of finite mixture models via EM algorithm.  Multivariate Normal Mixture models: definition, properties and ML estimation via EM algorithm. Multivariate t Mixture models: definition, properties and ML estimation via EM algorithm. Mixture of principal component analyzers and mixture of factor analyzers.

Clustering and Data Mining:  Classification, clustering and data mining. Some clustering methods: Fuzzy clustering, regression-wise clustering and clustering by finite mixture models.  Data recovery models by averaging, linear regression, PCA, factor analysis and K-mean clustering.

Bayesian Fundamentals: Statistical distributions (scalar, vector and matrix distributions). Prior distributions (vague, conjugate, generalized and correlation priors). Hyper-parameter Assessment (binomial, scalar normal, multivariate normal and matrix normal likelihoods). Bayesian Estimation Methods (marginal posterior mean, maximum a posteriori).

Bayesian Multivariate Regression: Bayesian regression model, likelihood, conjugate priors and posterior, conjugate estimation and inference, generalized priors and posterior, generalized estimation and inference, interpretation and discussion.

Bayesian Factor Analysis: Bayesian factor analysis model, likelihood, conjugate priors and posterior, conjugate estimation and inference, generalized priors and posterior, generalized estimation and inference, interpretation and discussion.

Introduction ICA/BSS: Independent component analysis (ICA) or blind source separation (BSS) model, PCA verses ICA. Estimation and inference, interpretation and discussion, Application.

1)

2)

3)

4)

5)

6)

7)

8)

10)

11)