Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"David Radwin" <dradwin@mprinc.com> |

To |
<statalist@hsphsun2.harvard.edu> |

Subject |
RE: st: RE: Mean test in a Likert Scale |

Date |
Fri, 31 Aug 2012 08:38:52 -0700 (PDT) |

I agree with Nick that "don't use means for ordinal data" is a purist stance, even if it is more honored in the breach than the observance. I also agree that collapsing Likert scales as described below is throwing away information. But I don't think it is arbitrary to consider "agree" and "strongly agree" to be one category and "disagree" and "strongly disagree" to be the other category. There is a separate question of why one would measure agreement on a 5 point (or 7 point or 101 point or whatever) scale only to later collapse it to yes/no, but analysts don't always have the ability to design the survey themselves. David -- David Radwin Senior Research Associate MPR Associates, Inc. 2150 Shattuck Ave., Suite 800 Berkeley, CA 94704 Phone: 510-849-4942 Fax: 510-849-0794 www.mprinc.com > -----Original Message----- > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner- > statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox > Sent: Friday, August 31, 2012 12:46 AM > To: statalist@hsphsun2.harvard.edu > Subject: Re: st: RE: Mean test in a Likert Scale > > Thanks for the extra detail, but I don't think it lets me add much to > my own advice, except that with a sample size of 20000 very small > differences are likely to seem significant by any particular > significance test. > > Nick > > On Fri, Aug 31, 2012 at 5:08 AM, Leonor Saravia <lmisaravia@gmail.com> > wrote: > > Dear Nick and David, > > > > I really appreciate your reply, thank you. > > > > I read carefully your answers to my questions and as Nick says, my > > first question pointed to the fact that there could be the sence in > > which computing the mean score of a Likert scale is allowed. I have > > seen practical studies were the mean of this kind of scales are > > calculated and interpreted. However, there is also literature that > > indicates that, as the Likert scales are an ordinal-level measure, you > > should not calculate the mean of it. So, I am confused because I do > > not understand whether calculating and interpreting the mean of a > > Likert scale is correct or not. > > > > The data I have is desagregated by individual (20000 observations) of > > a treatment and a control group, and has the answer for each of the 26 > > questions, a number between 1 and 5, which are the values of a 5 point > > Likert scale from Disagree (1) to Agree (5). > > > > For instance, the first question (Q1) is: "Chilean people find > > entrepreneurial activities socialy valuable" and the possible answers > > are: > > > > 1 - Strongly disagree > > 2 - Disagree > > 3 - Nor agree nor disagree > > 4 - Agree > > 5 - Strongly agree > > > > So, the database has this structure: > > > > Observation Group Q1 Q2 ..... Q25 Q26 > > 1 Treatment 1 5 ...... 3 1 > > 2 Control 3 1 ....... 2 5 > > . > > . > > 19999 Control 5 2 ........ 4 3 > > 20000 Treatment 3 2 ......... 5 4 > > > > > > From this, one could calculate the mean of Q1 for the treatment and > > control group, but I do not know if the number obtained can be > > interpreted and even more, if one can test mean differences between > > both groups. > > > > Thank you very much for your help and advice. > > > > Best regards, > > > > Leonor > > > >>I mostly disagree with David here. In particular, his proposal to > >>collapse the Likert scales just throws away information in an > >>arbitrary manner. > > > >>I don't think his advice is even consistent. If it's OK to treat means > >>of Bernoulli distributions as valid arguments for a t test, why is not > >>OK to treat means of Likert scales as if they were? > > > >>It's true that the reference case for a t test is two paired normal > >>distributions, and Likert scales can not be normal if only because > >>they are _not_ continuous, but there is always a judgment call on > >>whether summaries of the data will in practice work similarly. > > > >>A fair question is what exactly kind of advice is Leonor seeking? The > >>question presumably isn't really whether it is possible -- clearly it > >>is possible -- but perhaps somewhere between "Is it correct?" and "Is > >>it a good idea?" > > > >>Leonor's question appears to have the flavour of "I gather that this > >>is wrong. but is there a sense in which this is allowed?" The long > >>answer has to be that Leonor should tell us much more about the data > >>and the problem in hand if a good answer is to be given. If means make > >>sense as summary statistics, then comparing means with a t test is > >>likely to work well, but watch out. > > > >>David is clearly right in alluding to a purist literature in which you > >>are told as a matter of doctrine that ordinal data shouldn't be > >>summarised by means and so mean-based tests are also invalid. When > >>acting as academics, the same people work with grade-point averages > >>just like anybody else, at least in my experience. > > > >>There is also a pragmatist literature which points out that despite > >>all that, the sinful practice usually works well. Compare the t-test > >>with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the > >>P-values and z- or t-statistics will point to the same substantive > >>conclusion and indicate just about the same quantitative effect. It's > >>also likely that doing both tests will be needed because some reviewer > >>has been indoctrinated against t-tests here, and especially if anyone > >>is working with a rigid threshold (e.g. a 5% significance level). > > > >>Also, the behaviour of t-tests in cases like this can always be > >>examined by simulation, so no-one need be limited by textbook dogma > >>(or wickedness). > > > >>Nick > > > > On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <dradwin@mprinc.com> > wrote: > >> Leonor, > >> > >>No, you can't correctly calculate the mean of an ordinal-level measure > >>like the Likert scale you describe, although plenty of people do it > >> anyway. > >> > >> But you can use -ttest- with these data if you first collapse each > >> variable to a dichotomous (dummy) variable, because the mean of a > >> dichotomous variable is identical to the proportion where the value is > 1. > >> As a guess, you might set the highest two values to 1, the lowest two > >> values to 0, and the middle value to missing to calculate the > proportion > >> agreeing or somewhat agreeing. > >> > >> David > >> -- > >> David Radwin > >> Senior Research Associate > >> MPR Associates, Inc. > >> 2150 Shattuck Ave., Suite 800 > >> Berkeley, CA 94704 > >> Phone: 510-849-4942 > >> Fax: 510-849-0794 > >> > >> www.mprinc.com > >> > >> > >>> -----Original Message----- > >>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner- > >>> statalist@hsphsun2.harvard.edu] On Behalf Of Leonor Saravia > >>> Sent: Thursday, August 30, 2012 3:23 PM > >>> To: statalist@hsphsun2.harvard.edu > >>> Subject: st: Mean test in a Likert Scale > >>> > >>> Hello, > >>> > >>> I'm working with a survey that presents 26 questions and each of them > >>> has as possible answer a 5 point Likert scale from Desagree (1) to > >>> Agree (5). This survey was applyed for a treatment and a control > >>> group. > >>> > >>> As far as I know, it is possible to analyze the information given only > >>> by the proportions of each answer; for instance, 25% agrees, 50% > >>> desagree, or so. > >>> > >>> I have two questions that maybe one of you have had before: > >>> > >>> a) Is it possible to calculate the mean score of a sample (treatment > >>> or control group) - adding the individual answers - when one is > >>> working with a Likert scale? > >>> > >>> b) If it is possible to calculate a mean score of a sample when using > >>> a Likert scale, to compare the answers of the treatment versus the > >>> control group, is it well done if I use the 'ttest' command? * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: RE: Mean test in a Likert Scale***From:*Rob Ploutz-Snyder <robploutzsnyder@gmail.com>

**References**:**Re: st: RE: Mean test in a Likert Scale***From:*Leonor Saravia <lmisaravia@gmail.com>

**Re: st: RE: Mean test in a Likert Scale***From:*Nick Cox <njcoxstata@gmail.com>

- Prev by Date:
**Re: st: wordcount() undocumented limitation?** - Next by Date:
**Re: st: wordcount() undocumented limitation?** - Previous by thread:
**Re: st: RE: Mean test in a Likert Scale** - Next by thread:
**Re: st: RE: Mean test in a Likert Scale** - Index(es):