Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: RE: Mean test in a Likert Scale
"David Radwin" <email@example.com>
RE: st: RE: Mean test in a Likert Scale
Fri, 31 Aug 2012 08:38:52 -0700 (PDT)
I agree with Nick that "don't use means for ordinal data" is a purist
stance, even if it is more honored in the breach than the observance.
I also agree that collapsing Likert scales as described below is throwing
away information. But I don't think it is arbitrary to consider "agree"
and "strongly agree" to be one category and "disagree" and "strongly
disagree" to be the other category.
There is a separate question of why one would measure agreement on a 5
point (or 7 point or 101 point or whatever) scale only to later collapse
it to yes/no, but analysts don't always have the ability to design the
Senior Research Associate
MPR Associates, Inc.
2150 Shattuck Ave., Suite 800
Berkeley, CA 94704
> -----Original Message-----
> From: firstname.lastname@example.org [mailto:owner-
> email@example.com] On Behalf Of Nick Cox
> Sent: Friday, August 31, 2012 12:46 AM
> To: firstname.lastname@example.org
> Subject: Re: st: RE: Mean test in a Likert Scale
> Thanks for the extra detail, but I don't think it lets me add much to
> my own advice, except that with a sample size of 20000 very small
> differences are likely to seem significant by any particular
> significance test.
> On Fri, Aug 31, 2012 at 5:08 AM, Leonor Saravia <email@example.com>
> > Dear Nick and David,
> > I really appreciate your reply, thank you.
> > I read carefully your answers to my questions and as Nick says, my
> > first question pointed to the fact that there could be the sence in
> > which computing the mean score of a Likert scale is allowed. I have
> > seen practical studies were the mean of this kind of scales are
> > calculated and interpreted. However, there is also literature that
> > indicates that, as the Likert scales are an ordinal-level measure, you
> > should not calculate the mean of it. So, I am confused because I do
> > not understand whether calculating and interpreting the mean of a
> > Likert scale is correct or not.
> > The data I have is desagregated by individual (20000 observations) of
> > a treatment and a control group, and has the answer for each of the 26
> > questions, a number between 1 and 5, which are the values of a 5 point
> > Likert scale from Disagree (1) to Agree (5).
> > For instance, the first question (Q1) is: "Chilean people find
> > entrepreneurial activities socialy valuable" and the possible answers
> > are:
> > 1 - Strongly disagree
> > 2 - Disagree
> > 3 - Nor agree nor disagree
> > 4 - Agree
> > 5 - Strongly agree
> > So, the database has this structure:
> > Observation Group Q1 Q2 ..... Q25 Q26
> > 1 Treatment 1 5 ...... 3 1
> > 2 Control 3 1 ....... 2 5
> > .
> > .
> > 19999 Control 5 2 ........ 4 3
> > 20000 Treatment 3 2 ......... 5 4
> > From this, one could calculate the mean of Q1 for the treatment and
> > control group, but I do not know if the number obtained can be
> > interpreted and even more, if one can test mean differences between
> > both groups.
> > Thank you very much for your help and advice.
> > Best regards,
> > Leonor
> >>I mostly disagree with David here. In particular, his proposal to
> >>collapse the Likert scales just throws away information in an
> >>arbitrary manner.
> >>I don't think his advice is even consistent. If it's OK to treat means
> >>of Bernoulli distributions as valid arguments for a t test, why is not
> >>OK to treat means of Likert scales as if they were?
> >>It's true that the reference case for a t test is two paired normal
> >>distributions, and Likert scales can not be normal if only because
> >>they are _not_ continuous, but there is always a judgment call on
> >>whether summaries of the data will in practice work similarly.
> >>A fair question is what exactly kind of advice is Leonor seeking? The
> >>question presumably isn't really whether it is possible -- clearly it
> >>is possible -- but perhaps somewhere between "Is it correct?" and "Is
> >>it a good idea?"
> >>Leonor's question appears to have the flavour of "I gather that this
> >>is wrong. but is there a sense in which this is allowed?" The long
> >>answer has to be that Leonor should tell us much more about the data
> >>and the problem in hand if a good answer is to be given. If means make
> >>sense as summary statistics, then comparing means with a t test is
> >>likely to work well, but watch out.
> >>David is clearly right in alluding to a purist literature in which you
> >>are told as a matter of doctrine that ordinal data shouldn't be
> >>summarised by means and so mean-based tests are also invalid. When
> >>acting as academics, the same people work with grade-point averages
> >>just like anybody else, at least in my experience.
> >>There is also a pragmatist literature which points out that despite
> >>all that, the sinful practice usually works well. Compare the t-test
> >>with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the
> >>P-values and z- or t-statistics will point to the same substantive
> >>conclusion and indicate just about the same quantitative effect. It's
> >>also likely that doing both tests will be needed because some reviewer
> >>has been indoctrinated against t-tests here, and especially if anyone
> >>is working with a rigid threshold (e.g. a 5% significance level).
> >>Also, the behaviour of t-tests in cases like this can always be
> >>examined by simulation, so no-one need be limited by textbook dogma
> >>(or wickedness).
> > On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <firstname.lastname@example.org>
> >> Leonor,
> >>No, you can't correctly calculate the mean of an ordinal-level measure
> >>like the Likert scale you describe, although plenty of people do it
> >> anyway.
> >> But you can use -ttest- with these data if you first collapse each
> >> variable to a dichotomous (dummy) variable, because the mean of a
> >> dichotomous variable is identical to the proportion where the value
> >> As a guess, you might set the highest two values to 1, the lowest two
> >> values to 0, and the middle value to missing to calculate the
> >> agreeing or somewhat agreeing.
> >> David
> >> --
> >> David Radwin
> >> Senior Research Associate
> >> MPR Associates, Inc.
> >> 2150 Shattuck Ave., Suite 800
> >> Berkeley, CA 94704
> >> Phone: 510-849-4942
> >> Fax: 510-849-0794
> >> www.mprinc.com
> >>> -----Original Message-----
> >>> From: email@example.com [mailto:owner-
> >>> firstname.lastname@example.org] On Behalf Of Leonor Saravia
> >>> Sent: Thursday, August 30, 2012 3:23 PM
> >>> To: email@example.com
> >>> Subject: st: Mean test in a Likert Scale
> >>> Hello,
> >>> I'm working with a survey that presents 26 questions and each of
> >>> has as possible answer a 5 point Likert scale from Desagree (1) to
> >>> Agree (5). This survey was applyed for a treatment and a control
> >>> group.
> >>> As far as I know, it is possible to analyze the information given
> >>> by the proportions of each answer; for instance, 25% agrees, 50%
> >>> desagree, or so.
> >>> I have two questions that maybe one of you have had before:
> >>> a) Is it possible to calculate the mean score of a sample (treatment
> >>> or control group) - adding the individual answers - when one is
> >>> working with a Likert scale?
> >>> b) If it is possible to calculate a mean score of a sample when
> >>> a Likert scale, to compare the answers of the treatment versus the
> >>> control group, is it well done if I use the 'ttest' command?
* For searches and help try: