Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Interrater Agreement and Kap Command


From   Morten Hesse <mh@crf.au.dk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Interrater Agreement and Kap Command
Date   Tue, 13 Jul 2010 23:53:35 +0200

Dear Frank
You wouldn't want to use the ICC if your values are not interval scale. What I know of SPSS that would be the ICC for an interval scale. There is a similar statistic for logistic regression (the rho statistic), but in my experience, in cases where kappa values suggest moderate agreement between any two observations, the rho statistic derived from xtlogit or MLE logistic regression in other software will show very high values of agreement, and I do not believe that they represent reliability estimates that are realistic, although I do not have the expertise to say so (any thoughts, anyone else). Therefore, you would want to use Fleiss's kappa. I am not sure if that or a similar statistic is available for Stata. Percent agreement should be fairly easy to estimate using an excell spreadsheet. One question: If you say 49 variables, would that refer to different outcomes pertaining to the same units of observation?
Greetings
Morten

Frank Gallo skrev:
Dear All:

I am a beginner with Stata, and I prefer to use Stata instead of SPSS
to address the following situation. I want to evaluate interrater
agreement. I have more than two raters (94), more than two ratings (49
variables), and varying number of raters (some missing values on the
variables). The data are in the form to use the kap command, however,
there is no statistic for testing kappa > 0 and no results reported
regarding the percentage of agreement. Is there some alternative that
I can use in Stata to obtain these results? I looked through the
archives and could not find one. SPSS offers the ICC option. I
appreciate any thoughts.

Best,
Frank
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index