From "Nick Cox" To Subject st: RE: PCA vs. Factor Loadings Date Wed, 16 Dec 2009 12:15:44 -0000

```I think the short answer is that you are not comparing like with like.

Loadings can be presented in various ways. See for example the help for
-pca postestimation- and then experiment with the different

The default presentation of PCA loadings is not what you want, but a
different normalisation shows that PCA and factor analysis coincide in
the limit:

. pca headroom trunk weight length displacement

Principal components/correlation                  Number of obs    =
74
Number of comp.  =
5
Trace            =
5
Rotation: (unrotated = principal)             Rho              =
1.0000

------------------------------------------------------------------------
--
Component |   Eigenvalue   Difference         Proportion
Cumulative

-------------+----------------------------------------------------------
--
Comp1 |      3.76201        3.026             0.7524
0.7524
Comp2 |      .736006      .427915             0.1472
0.8996
Comp3 |      .308091      .155465             0.0616
0.9612
Comp4 |      .152627      .111357             0.0305
0.9917
Comp5 |     .0412693            .             0.0083
1.0000

------------------------------------------------------------------------
--

Principal components (eigenvectors)

------------------------------------------------------------------------
------
Variable |    Comp1     Comp2     Comp3     Comp4     Comp5 |
Unexplained

-------------+--------------------------------------------------+-------
------
headroom |   0.3587    0.7640    0.5224   -0.1209    0.0130 |
0
trunk |   0.4334    0.3665   -0.7676    0.2914    0.0612 |
0
weight |   0.4842   -0.3329    0.0737   -0.2669    0.7603 |
0
length |   0.4863   -0.2372   -0.1050   -0.5745   -0.6051 |
0
displacement |   0.4610   -0.3390    0.3484    0.7065   -0.2279 |
0

------------------------------------------------------------------------
------

component normalization: sum of squares(column) = eigenvalue

----------------------------------------------------------------
|    Comp1     Comp2     Comp3     Comp4     Comp5
-------------+--------------------------------------------------
headroom |    .6958     .6554       .29   -.04724   .002635
trunk |    .8405     .3144    -.4261     .1138    .01243
weight |    .9392    -.2856    .04092    -.1043     .1545
length |    .9432    -.2035   -.05829    -.2245    -.1229
displacement |    .8942    -.2909     .1934      .276   -.04629
----------------------------------------------------------------

. factor   headroom trunk weight length displacement, pcf
(obs=74)

Factor analysis/correlation                        Number of obs    =
74
Method: principal-component factors            Retained factors =
1
Rotation: (unrotated)                          Number of params =
5

------------------------------------------------------------------------
--
Factor  |   Eigenvalue   Difference        Proportion
Cumulative

-------------+----------------------------------------------------------
--
Factor1  |      3.76201      3.02600            0.7524
0.7524
Factor2  |      0.73601      0.42791            0.1472
0.8996
Factor3  |      0.30809      0.15546            0.0616
0.9612
Factor4  |      0.15263      0.11136            0.0305
0.9917
Factor5  |      0.04127            .            0.0083
1.0000

------------------------------------------------------------------------
--
LR test: independent vs. saturated:  chi2(10) =  373.68 Prob>chi2 =
0.0000

---------------------------------------
Variable |  Factor1 |   Uniqueness
-------------+----------+--------------
trunk |   0.8405 |      0.2935
weight |   0.9392 |      0.1180
length |   0.9432 |      0.1103
displacement |   0.8942 |      0.2003
---------------------------------------

On the broader question, the question has some similarity with the
question of how big should a correlation be before one should pay
attention. I doubt there's an answer independent of discipline and
problem.

Nick
n.j.cox@durham.ac.uk

Michael I. Lichter

Stata that makes them systematically smaller? I get the sense (which may

be mistaken; I don't have any evidence in my hand) that in other

For example, in the example below the variable -trunk- has a component

solutions look comparable.

My question is prompted by a more fundamental question, which is how
large should a loading be before it is considered significant (in the
sense of "worthy of notice")? Texts that give advice on interpretation
seem to assume that -pca- and -factor- results are on the same scale,
getting from -pca-.

Example:

. sysuse auto
. pca trunk weight length headroom, mineigen(1)

Principal components/correlation                  Number of obs
=        74
Number of comp.
=         1
Trace
=         4
Rotation: (unrotated = principal)             Rho              =
0.7551

------------------------------------------------------------------------
--
Component |   Eigenvalue   Difference         Proportion
Cumulative

-------------+----------------------------------------------------------
--
Comp1 |      3.02027      2.36822             0.7551
0.7551
Comp2 |      .652053       .37494             0.1630
0.9181
Comp3 |      .277113      .226551             0.0693
0.9874
Comp4 |     .0505616            .             0.0126
1.0000

------------------------------------------------------------------------
--

Principal components (eigenvectors)

--------------------------------------
Variable |    Comp1 | Unexplained
-------------+----------+-------------
trunk |   0.5068 |       .2243
weight |   0.5221 |       .1768
length |   0.5361 |       .1319
--------------------------------------
. factor trunk weight length headroom, pcf
(obs=74)

Factor analysis/correlation                        Number of obs
=       74
Method: principal-component factors            Retained factors
=        1
Rotation: (unrotated)                          Number of params
=        4

------------------------------------------------------------------------
--
Factor  |   Eigenvalue   Difference        Proportion
Cumulative

-------------+----------------------------------------------------------
--
Factor1  |      3.02027      2.36822            0.7551
0.7551
Factor2  |      0.65205      0.37494            0.1630
0.9181
Factor3  |      0.27711      0.22655            0.0693
0.9874
Factor4  |      0.05056            .            0.0126
1.0000

------------------------------------------------------------------------
--
LR test: independent vs. saturated:  chi2(6)  =  257.89 Prob>chi2 =
0.0000

---------------------------------------
Variable |  Factor1 |   Uniqueness
-------------+----------+--------------
trunk |   0.8807 |      0.2243
weight |   0.9073 |      0.1768
length |   0.9317 |      0.1319