Re: st: RE: Mean test in a Likert Scale

[Nick Cox <njcoxstata@gmail.com>]

I mostly disagree with David here. In particular, his proposal to
collapse the Likert scales just throws away information in an
arbitrary manner.

I don't think his advice is even consistent. If it's OK to treat means
of Bernoulli distributions as valid arguments for a t test, why is not
OK to treat means of Likert scales as if they were?

It's true that the reference case for a t test is two paired normal
distributions, and Likert scales can not be normal if only because
they are _not_ continuous, but there is always a judgment call on
whether summaries of the data will in practice work similarly.

A fair question is what exactly kind of advice is Leonor seeking? The
question presumably isn't really whether it is possible -- clearly it
is possible -- but perhaps somewhere between "Is it correct?" and "Is
it a good idea?"

Leonor's question appears to have the flavour of "I gather that this
is wrong. but is there a sense in which this is allowed?" The long
answer has to be that Leonor should tell us much more about the data
and the problem in hand if a good answer is to be given. If means make
sense as summary statistics, then comparing means with a t test is
likely to work well, but watch out.

David is clearly right in alluding to a purist literature in which you
are told as a matter of doctrine that ordinal data shouldn't be
summarised by means and so mean-based tests are also invalid. When
acting as academics, the same people work with grade-point averages
just like anybody else, at least in my experience.

There is also a pragmatist literature which points out that despite
all that, the sinful practice usually works well. Compare the t-test
with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the
P-values and z- or t-statistics will point to the same substantive
conclusion and indicate just about the same quantitative effect. It's
also likely that doing both tests will be needed because some reviewer
has been indoctrinated against t-tests here, and especially if anyone
is working with a rigid threshold (e.g. a 5% significance level).

Also, the behaviour of t-tests in cases like this can always be
examined by simulation, so no-one need be limited by textbook dogma
(or wickedness).

Nick

On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <dradwin@mprinc.com> wrote:
> Leonor,
>
> No, you can't correctly calculate the mean of an ordinal-level measure
> like the Likert scale you describe, although plenty of people do it
> anyway.
>
> But you can use -ttest- with these data if you first collapse each
> variable to a dichotomous (dummy) variable, because the mean of a
> dichotomous variable is identical to the proportion where the value is 1.
> As a guess, you might set the highest two values to 1, the lowest two
> values to 0, and the middle value to missing to calculate the proportion
> agreeing or somewhat agreeing.
>
> David
> --
> David Radwin
> Senior Research Associate
> MPR Associates, Inc.
> 2150 Shattuck Ave., Suite 800
> Berkeley, CA 94704
> Phone: 510-849-4942
> Fax: 510-849-0794
>
> www.mprinc.com
>
>
>> -----Original Message-----
>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
>> statalist@hsphsun2.harvard.edu] On Behalf Of Leonor Saravia
>> Sent: Thursday, August 30, 2012 3:23 PM
>> To: statalist@hsphsun2.harvard.edu
>> Subject: st: Mean test in a Likert Scale
>>
>> Hello,
>>
>> I'm working with a survey that presents 26 questions and each of them
>> has as possible answer a 5 point Likert scale from Desagree (1) to
>> Agree (5). This survey was applyed for a treatment and a control
>> group.
>>
>> As far as I know, it is possible to analyze the information given only
>> by the proportions of each answer; for instance, 25% agrees, 50%
>> desagree, or so.
>>
>> I have two questions that maybe one of you have had before:
>>
>> a) Is it possible to calculate the mean score of a sample (treatment
>> or control group) - adding the individual answers - when one is
>> working with a Likert scale?
>>
>> b) If it is possible to calculate a mean score of a sample when using
>> a Likert scale, to compare the answers of the treatment versus the
>> control group, is it well done if I use the 'ttest' command?
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/