Re: st: Identifying the best scale without a "gold standard"

[Cameron McIntosh <cnm100@hotmail.com>]

Hi Paul,

Sorry for not getting back to you sooner. I agree with Ronan that predictive validity and sensitivity are also important criteria for a 'good scale', so I might postpone any firm judgment until you could collect such data to further assess the scales.
As to your comment about finding one eigenvalue > 1.0 in your data, this is now seen as a rather tenuous means to get at the number of non-trivial factors. Instead, you might try a parallel or Hull analysis to address that question:

Lorenzo-Seva, U., Timmerman, M.E., & Kiers, H.A.L. (2011). The Hull Method for Selecting the Number of Common Factors. Multivariate Behavioral Research, 46(2), 340-364.http://psico.fcep.urv.es/utilitats/factor/Description.html

Crawford, A.V., Green, S.B., Levy, R., Lo, W.J., Scott, L., Svetina, D., & Thompson, M.S. (2010). Evaluation of parallel analysis methods for determining the number of factors. Educational and Psychological Measurement, 70(6), 885-901.

Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality Assessment of Ordered Polytomous Items with Parallel Analysis. Psychological Methods, 16, Epub ahead of print.http://www.ncbi.nlm.nih.gov/pubmed/21500916

Hayton, J.C., Allen, D.G., & Scarpello, V. (2004). Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis. Organizational Research Methods, 7(2), 191-205.http://orm.sagepub.com/content/7/2/191.full.pdf+html ;

That said, in my view EFA is one of the most deceptive and lax statistical procedures around, and I wouldn't use its output as a justification not to try an alternative structure such as a second-order model. I would just directly test the hypothesized model - the one I believed generated the data - and possibly some alternatives.

Best, 
Cam

> From: paul.seed@kcl.ac.uk
> To: statalist@hsphsun2.harvard.edu
> Date: Wed, 16 Nov 2011 11:48:49 +0000
> Subject: Re: st: Identifying the best scale without a "gold standard"
> 
> Thank you Ronan.  A good point, well made.
> 
> Paul T Seed MSc CStat CSci, Senior Lecturer in Medical Statistics, 
> King's College London, Division of Women's Health
> (& Department of Primary Care and Public Health Sciences)
> St Thomas' Hospital, Westminster Bridge Road, London SE1 7EH
> 
> On Tue, 15 Nov 2011 09:53:29 +0000 Ronan Conroy <rconroy@rcsi.ie> wrote: 
> >Subject: Re: st: Identifying the best scale without a "gold standard"
> 
> >On 2011 Samh 14, at 20:53, Seed, Paul wrote:
> >> The research problem is to identify the best single scale for measuring breathlessness 
> >> from the six candidates.  I was therefore interested in a valid test for 
> >> identifying agreement of individual measures with a latent factor
> >> to which they all contributed.
> >
> >The definition of 'best scale' is not without its difficulty. In most cases, scales are expected to make both longitudinal and cross-sectional measurements. >For this reason, the ability to detect difference between groups that ought to be different, and to detect change in individuals when such change can >reasonably be expected is also important. 
> >
> >Ronán Conroy
> >rconroy@rcsi.ie
> >Associate Professor
> >Division of Population Health Sciences
> >Royal College of Surgeons in Ireland
> >Beaux Lane House
> >Dublin 2
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
 		 	   		   		 	   		  
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/