Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: statalist-digest V4 #4807 (st: reliability with -icc- ) - Statistics as APPLIED science


From   Lenny Lesser <lenny3200@gmail.com>
To   statalist <statalist@hsphsun2.harvard.edu>
Subject   Re: statalist-digest V4 #4807 (st: reliability with -icc- ) - Statistics as APPLIED science
Date   Thu, 28 Feb 2013 18:37:23 -0800

Hi,
Thank you all for your comments.
Surely their is a methods and conceptual issue.
There is also a STATA question, which I was mostly discussing here.

The problem with our scale is that is wasn't meant for smartphone
apps.  We actually developed another shorter scale for this purpose
that we tested along side this.  The shorter scale had more agreement
between raters and ranked the apps similarly.  The reason the first
scale was so poor is that it asked detailed questions that were easy
to miss in looking at a smartphone app.  They were also repetitive and
similar.

For the STATA question, it appears that we've found the right solution.
Thank you all,
Lenny

On Thu, Feb 28, 2013 at 5:37 AM, JVerkuilen (Gmail)
<jvverkuilen@gmail.com> wrote:
> On Thu, Feb 28, 2013 at 6:10 AM, Nick Cox <njcoxstata@gmail.com> wrote:
>> I do agree broadly with Allan, whether or not that is surprising.
>
> Me too, actually, though I think both Nick and I were misquoted and
> taken out of context. I know the first thing I did with the raw data
> was qnorm and graph box....
>
>
>> A wilder idea is that rater 4 who gave no score higher than 3 either
>> never knew or somehow forgot that scores could be up to 100 and just
>> used a 5-point scale. Even if #4 did know that, #4 is so out-of-line
>> that including them remains dubious, although doing computations with
>> and without #4 remains manageable.
>>
>> In any case if the highest score is 18, then something else is going
>> on that needs to be spelled out, if only as context.
>
> 100% agree and that's certainly consistent with a good bit of the
> discussion on that thread, but there were quite a bit of discussions
> not strictly aimed at Lenny's original problem but at the broader
> question of ICC estimation using Stata, which is the nature of this
> listserv.
>
>
>
>> Only the original poster can add more context than we already have. In
>> any field that I know about this dataset would be too small to be
>> publishable, except as a toy dataset to make points about method
>> (which I take it is Jay's motive here).
>
> Yes, that's my exact motive. I'm writing a paper on "small sample"
> problems with estimation of reliability coefficients, which are quite
> common in practice.
>
>
> --
> JVVerkuilen, PhD
> jvverkuilen@gmail.com
>
> "It is like a finger pointing away to the moon. Do not concentrate on
> the finger or you will miss all that heavenly glory." --Bruce Lee,
> Enter the Dragon (1973)
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index