Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: reliability with -icc- and -estat icc-


From   "JVerkuilen (Gmail)" <jvverkuilen@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: reliability with -icc- and -estat icc-
Date   Wed, 27 Feb 2013 10:33:53 -0500

On Wed, Feb 27, 2013 at 10:06 AM,  <jcoveney@bigplanet.com> wrote:
>
> When in doubt, try going back to a reference source (
> www.hongik.edu/~ym480/Shrout-Fleiss-ICC.pdf ) and manually computing
> the ICC.  According to the source, “ICC is the correlation between
> one measurement . . . on a target and another measurement obtained on
> that target.”  In your case, targets are smartphone software.

Correct.



> By the way, Rater #4 is providing valuable information about rater
> reliability, and so I recommend against eliminating her scores from
> the ICC computation.

I think my interpretation is that Rater 4 is an outlier. The question
is what does that outlier status mean? It's very clear that she's
driving most of the ICC estimate so the notion of a good ICC estimate
from these data themselves is suspect. This is a useful study outcome.


> My take on all that would be that your volunteers need better
> training on evaluating smartphone software in the manner that you
> want it done.  Perhaps you and your colleagues could provide more
> explicit instructions on what you’re are looking for in measuring the
> characteristic(s) of the software that you’re trying to measure.

Absolutely true, and is probably the best takeaway of this exercise.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index