Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: statalist-digest V4 #4807 (st: reliability with -icc- ) - Statistics as APPLIED science

From   Nick Cox <>
Subject   Re: statalist-digest V4 #4807 (st: reliability with -icc- ) - Statistics as APPLIED science
Date   Fri, 1 Mar 2013 09:55:14 +0000

Thanks for the extra context. We can all agree that what you should be
doing and how to do it in Stata [NB] are different questions. We're
not examiners or reviewers, but my guess is that the same kind of
questions would recur if this were part of a thesis, report or
submitted paper.


On Fri, Mar 1, 2013 at 2:37 AM, Lenny Lesser <> wrote:

> Thank you all for your comments.
> Surely their is a methods and conceptual issue.
> There is also a STATA question, which I was mostly discussing here.
> The problem with our scale is that is wasn't meant for smartphone
> apps.  We actually developed another shorter scale for this purpose
> that we tested along side this.  The shorter scale had more agreement
> between raters and ranked the apps similarly.  The reason the first
> scale was so poor is that it asked detailed questions that were easy
> to miss in looking at a smartphone app.  They were also repetitive and
> similar.
> For the STATA question, it appears that we've found the right solution.

> On Thu, Feb 28, 2013 at 5:37 AM, JVerkuilen (Gmail)

>> On Thu, Feb 28, 2013 at 6:10 AM, Nick Cox <> wrote:
>>> I do agree broadly with Allan, whether or not that is surprising.
>> Me too, actually, though I think both Nick and I were misquoted and
>> taken out of context. I know the first thing I did with the raw data
>> was qnorm and graph box....
>>> A wilder idea is that rater 4 who gave no score higher than 3 either
>>> never knew or somehow forgot that scores could be up to 100 and just
>>> used a 5-point scale. Even if #4 did know that, #4 is so out-of-line
>>> that including them remains dubious, although doing computations with
>>> and without #4 remains manageable.
>>> In any case if the highest score is 18, then something else is going
>>> on that needs to be spelled out, if only as context.
>> 100% agree and that's certainly consistent with a good bit of the
>> discussion on that thread, but there were quite a bit of discussions
>> not strictly aimed at Lenny's original problem but at the broader
>> question of ICC estimation using Stata, which is the nature of this
>> listserv.
>>> Only the original poster can add more context than we already have. In
>>> any field that I know about this dataset would be too small to be
>>> publishable, except as a toy dataset to make points about method
>>> (which I take it is Jay's motive here).
>> Yes, that's my exact motive. I'm writing a paper on "small sample"
>> problems with estimation of reliability coefficients, which are quite
>> common in practice.
*   For searches and help try:

© Copyright 1996–2016 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index