Principal Components Analysis

Abstract

One of the problems with a lot of sets of multivariate data is that there are simply too many variables to make the application of the graphical techniques described in the previous chapters successful in providing an informative initial assessment of the data. And having too many variables can also cause problems for other multivariate techniques that the researcher may want to apply to the data. The possible problem of too many variables is sometimes known as the curse of dimensionality (Bellman 1961). Clearly the scatterplots, scatterplot matrices, and other graphics included in Chapter 2 are likely to be more useful when the number of variables in the data, the dimensionality of the data, is relatively small rather than large. This brings us to principal components analysis, a multivariate technique with the central aim of reducing the dimensionality of a multivariate data set while accounting for as much of the original variation as possible present in the data set. This aim is achieved by transforming to a new set of variables, the principal components, that are linear combinations of the original variables, which are uncorrelated and are ordered so that the first few of them account for most of the variation in all the original variables. In the best of all possible worlds, the result of a principal components analysis would be the creation of a small number of new variables that can be used as surrogates for the originally large number of variables and consequently provide a simpler basis for, say, graphing or summarising the data, and also perhaps when undertaking further multivariate analyses of the data.