# Re: st: Aren't distinct factors from factor analysis or PCA orthogonal to each other?

 From Robert A Yaffee To statalist@hsphsun2.harvard.edu Subject Re: st: Aren't distinct factors from factor analysis or PCA orthogonal to each other? Date Mon, 17 Aug 2009 14:05:03 -0400

```Distinction of factors can from differential eigenvalues and  high loadings.
Such distinct factors may be correlated as a result of an oblique rotation.
They need not be orthogonal to possess distinction, although orthgonality
implies no correlation between them, which could make them distinct.

There are several approaches to hierarchical factor analysis. In some
approaches, a second order factor analysis would qualify.  In others, an
oblique rotation must be used either in the first or second step.  In others,
a confirmatory factor analysis is used.  The latter is available as Rabe Hasketh's gllamm
on SSC or as cfa from Stas Kolenikov.

Regards,
Bob

Robert A. Yaffee, Ph.D.
Research Professor
Silver School of Social Work
New York University

Biosketch: http://homepages.nyu.edu/~ray1/Biosketch2009.pdf

CV:  http://homepages.nyu.edu/~ray1/vita.pdf

----- Original Message -----
From: Robert A Yaffee <bob.yaffee@nyu.edu>
Date: Monday, August 17, 2009 12:30 pm
Subject: Re: st: Aren't distinct factors from factor analysis or PCA orthogonal to each other?
To: statalist@hsphsun2.harvard.edu

> Distinct factors can be determined more by a finding of simple
> structure from a rotation rather than by an orthogonal rotation
>  orthogonal rotation.  Distinct factors may be correlated with one
> another and may be ascertained through an oblique
>  rotation.
>
>  Diana,
>       It is possible to perform a kind of second-order or hierarchical
> factor analysis with state, providing you have
>  enough cases and variables to permit such a solution.
>       If you prefer the type of hierarchical factor analysis that
> employs confirmatory factor analysis for the final model, you
>  could use the cfa program developed by Stas Kolenkov  or possibly
> gllamm by Sophia Rabe-Hasketh and Anders Skrondal.
>     Regards,
>           Bob Yaffee
>
>
>  Robert A. Yaffee, Ph.D.
>  Research Professor
>  Silver School of Social Work
>  New York University
>
>  Biosketch: http://homepages.nyu.edu/~ray1/Biosketch2009.pdf
>
>  CV:  http://homepages.nyu.edu/~ray1/vita.pdf
>
>  ----- Original Message -----
>  From: kornbrot <d.e.kornbrot@herts.ac.uk>
>  Date: Monday, August 17, 2009 11:49 am
>  Subject: Re: st: Aren't distinct factors from factor analysis or PCA
> orthogonal to each other?
>  To: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
>
>
>  > Is it possible to hierarchical EFA in stata?
>  >  Are there do  files?
>  >  Best
>  >  diana
>  >
>  >
>  >  On 17/08/2009 07:43, "Maarten buis" <maartenbuis@yahoo.co.uk> wrote:
>  >
>  >  > --- On Mon, 17/8/09, kokootchke wrote:
>  >  >> > I am new to factor analysis and I am trying to use it to
>  >  >> > decompose a big matrix of economic, financial, and political
>  >  >> > variables for many countries. So I run
>  >  >> >
>  >  >> > factor var1-var100
>  >  >> >
>  >  >> > and then I look that the first 3 factors explain most of
>  >  >> > the variation in that matrix, so then I want to use these
>  >  >> > three factors to see whether another variable (a measure of
>  >  >> > the "risk" of the country) is explained by these three
>  >  >> > factors. So I do:
>  >  >> >
>  >  >> > predict factor1 factor2 factor3
>  >  >> > reg risk factor1 factor2 factor3
>  >  >> >
>  >  >> > and I obtain very strongly significant estimates.
>  >  >> >
>  >  >> > My first question is: if I understand correctly, these
>  >  >> > factors should be orthogonal from each other. If that's the
>  >  >> > case, a regression such as:
>  >  >> >
>  >  >> > reg risk factor1
>  >  >> >
>  >  >> > should NOT give me a different coefficient for factor1
>  >  >> > compared to the factor1 coefficient in the first regression
>  >  >> > that includes all three factors, right? This is because if I
>  >  >> > omit factor2 and factor3, these things would go into the
>  >  >> > error term of my regression, but they wouldn't be adding any
>  >  >> > correlation between the error term and factor1, so the
>  >  >> > factor1 coefficient shouldn't change.
>  >  >> >
>  >  >> > Or should I?
>  >  >> >
>  >  >> > In my case, it does. Why is this, can you please help me
>  >  >> > understand?
>  >  >
>  >  > This is definately true for principle components analysis:
>  >  >
>  >  > *------------- begin example --------------------
>  >  > sysuse auto, clear
>  >  > pca weight price rep78 turn length displacement
>  >  > predict sc1 sc2 sc3
>  >  > corr sc*
>  >  > reg mpg sc*
>  >  > reg mpg sc1
>  >  > *-------------- end example ---------------------
>  >  >
>  >  > The way I keep pca and factor analysis apart is that pca
>  >  > is more "mechanical" (It finds orthogonal vectors) while
>  >  > factor analysis is more "theoretical" (there is a latent
>  >  > variable influencing the observed variables). I don't
>  >  > use either of these very often, so I need those simple
>  >  > rules of thumb. Those who use it more often can
>  >  >
>  >  > Hope this helps,
>  >  > Maarten
>  >  >
>  >  > -----------------------------------------
>  >  > Maarten L. Buis
>  >  > Institut fuer Soziologie
>  >  > Universitaet Tuebingen
>  >  > Wilhelmstrasse 36
>  >  > 72074 Tuebingen
>  >  > Germany
>  >  >
>  >  > http://home.fsw.vu.nl/m.buis/
>  >  > -----------------------------------------
>  >  >
>  >  >
>  >  >
>  >  >
>  >  > *
>  >  > *   For searches and help try:
>  >  > *   http://www.stata.com/help.cgi?search
>  >  > *   http://www.stata.com/support/statalist/faq
>  >  > *   http://www.ats.ucla.edu/stat/stata/
>  >
>  >  Professor Diana Kornbrot
>  >  email:  d.e.kornbrot@herts.ac.uk
>  >  web:    http://web.me.com/kornbrot/KornbrotHome.html
>  >  Work
>  >  School of Psychology
>  >   University of Hertfordshire
>  >   College Lane, Hatfield, Hertfordshire AL10 9AB, UK
>  >     voice:   +44 (0) 170 728 4626
>  >     fax:     +44 (0) 170 728 5073
>  >  Home
>  >   19 Elmhurst Avenue
>  >   London N2 0LT, UK
>  >      voice:   +44 (0) 208 883  3657
>  >      mobile: +44 (0) 796 890 2102
>  >      fax:      +44 (0) 870 706 4997
>  >
>  >
>  >
>  >
>  >
>  >
>  >
>
>  *
>  *   For searches and help try:
>  *   http://www.stata.com/help.cgi?search
>  *   http://www.stata.com/support/statalist/faq
>  *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```