st: RE: Why do Stata Cronbach's Alpha values not match SAS?

["Gramig, Benjamin M" <bgramig@purdue.edu>]

Thank you Joseph. The suggestion to walk through a worked example like the one contained in the SAS manual is a very good one. I didn't have any errors in my data statements in SAS, but by walking through this process and thinking about the other things you mentioned I have been able to determine why I was seeing such big differences.

The key difference is in how SAS handles variables with negative signs (correlations) when compared to Stata.  Stata automatically reverses the coding of the variables with opposite signs from the group, whereas SAS does not.  

Using the -alpha- command in Stata with the -, item asis- options will give you the same output as the SAS command -PROC CORR...alpha-.  

SAS's -nomiss- option works equivalent to Stata's -, casewise- option.  

SAS doesn't appear to have an option that automatically reverses the sign for negatively correlated variables, so I guess I would need to recode the variables manually in my data to calculate the alpha reliability statistic for the scale equivalently in SAS.

I feel I still need to use SAS for the accompanying PCA of the items in the scale because I cannot figure out how to output factor loadings or apply rotations following -polychoric pca-. It is my understanding that for ordinal/Likert data I need to use polychoric correlations (or another technique) to perform PCA correctly.

Many thanks,
Ben Gramig

-----Original Message-----
From: Joseph Coveney [mailto:jcoveney@bigplanet.com] 
Sent: Thursday, April 28, 2011 12:23 AM
To: statalist@hsphsun2.harvard.edu
Cc: Gramig, Benjamin M
Subject: Re: Why do Stata Cronbach's Alpha values not match SAS?


Benjamin Gramig wrote:

I have used the alpha, item- command in Stata to calculate Cronbachs alpha to
evaluate scale reliability of a group of Likert scale survey questions and
received the following Stata output:

[omitted]

When I attempt to use SAS to calculate the same Alpha reliability  value using
the -PROC CORR data alpha- command I get much lower values (pasted below) for
items in the scale as well as the overall alpha for the entire scale.  The
qualitative results in terms of sign and relative magnitude of alphas calculated
when removing individual items from the scale are consistent with Stata, but not
the magnitudes. I have read the manuals for both pieces of software and it is
not clear to me that there are differences in what is being reported by both
software packages.  

Has anyone else encountered this drastic difference?  I assume that there is
something systematically different about what is going on in the two packages to
calculate the reported values, but I couldn't determine what this difference
was.

It should be noted that I turned to SAS to be able to use polychoric
correlations in a PCA with a full set of diagnostics, outputs and rotations
available.  This seems necessary for my ordinal data, despite ignoring this in
the comparison of alpha calculations shared here. I did explore 
-polychoricpca- in Stata before deciding to use SAS.

Any insights are greatly appreciated,
[omitted]

--------------------------------------------------------------------------------

As far as insights for the differences between software, my first suspicion is
that you're analyzing different data.  That DATA step can be a tricky thing if
you're accustomed to Stata, and conversely those who are at-home with SAS's way
of going about data management can be tripped up by trying to emulate it in
Stata.

A second insight based upon experience:  when in doubt, compare results with a
worked example.  Choosing an example from SAS and running it in Stata (see
below) shows that the two packages will give the same results when fed the same
dataset and options (in this case, listwise deletion of observations with at
least one missing value).  There is one feature not explored in the worked
example below:  I don't know how SAS handles sign flips as compared to Stata's
-alpha-.  The worked example below doesn't have any flipped scales and, as you
mentioned, your data do.  So, that's something to look into further if you can
find a worked example that has reversed-sense variables.

As far as your choosing SAS over Stata for principal component analysis with
categorical data, I don't have any insight.  But I have a few comments:  1. I
assume principal component analysis is a considered decision--that it's what you
really want to do. 2. You can do any rotation in Stata that I'm aware of, at
least for exploratory factor analysis. 3. Based upon my experience with M-Plus,
at least a couple of the popular (must-show) diagnostic tests appear not to work
well with categorical variables.  4. At least for exploratory factor analysis, I
didn't notice any practical gain (can-do-science-better) in moving away from
Stata because of categorical variables.

Joseph Coveney

version 11.2

clear *
set more off

input float Weight double (Length3 HtPct WidthPct)
/*
omitted:  get the dataset (and the SAS output to compare
Stata's) from 
http://support.sas.com/documentation/cdl/en/procstat/63104/HTML/default/viewer.h
tm#procstat_corr_sect032.htm
*/
end
generate double Weight3 = Weight^(1/3)
generate double Height = HtPct * Length3 / 100
generate double Width = WidthPct *  Length3 / 100
drop HtPct WidthPct
label data "Fish Measurement Data"

generate byte listwise_missing = 0
foreach var of varlist Weight3 Length3 Height Width {
	replace listwise_missing = 1 if mi(`var')
}
summarize Weight3 Length3 Height Width if !listwise_missing
pwcorr Weight3 Length3 Height Width, casewise // NOMISS

alpha Weight3 Length3 Height Width, casewise item
alpha Weight3 Length3 Height Width, casewise item std

exit



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/