Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: Interrater Agreement and Kap Command

From   "Visintainer, Paul" <>
To   "''" <>
Subject   RE: st: Interrater Agreement and Kap Command
Date   Wed, 14 Jul 2010 09:24:01 -0400

You can try -icc23-, which is downloadable in Stata (type: findit icc23).  It will compute type 2 and type 3 intra-class correlations, using error components derived from ANOVA.


Paul F. Visintainer, PhD
Baystate Medical Center
Springfield, MA 01199

-----Original Message-----
From: [] On Behalf Of Frank Gallo
Sent: Tuesday, July 13, 2010 8:28 PM
To: statalist
Subject: Re: st: Interrater Agreement and Kap Command

Hi Morten,

Thank you for your response. Let me clarify my situation. The ratings
are interval-like. The data file is in the following format: columns
(raters, n = 94) and rows (variables, n = 49). The raters gave
responses that ranged from 1 to 9 on the variables. I am looking for
something similar to the ICC (model: two-way mixed, alpha; type:
consistency) available in SPSS. I hope this helps.


On Tue, Jul 13, 2010 at 5:53 PM, Morten Hesse <> wrote:
> Dear Frank
> You wouldn't want to use the ICC if your values are not interval scale. What
> I know of SPSS that would be the ICC for an interval scale. There is a
> similar statistic for logistic regression (the rho statistic), but in my
> experience, in cases where kappa values suggest moderate agreement between
> any two observations, the rho statistic derived from xtlogit or MLE logistic
> regression in other software will show very high values of agreement, and I
> do not believe that they represent reliability estimates that are realistic,
> although I do not have the expertise to say so (any thoughts, anyone else).
> Therefore, you would want to use Fleiss's kappa. I am not sure if that or a
> similar statistic is available for Stata.
> Percent agreement should be fairly easy to estimate using an excell
> spreadsheet.
> One question: If you say 49 variables, would that refer to different
> outcomes pertaining to the same units of observation?
> Greetings
> Morten
> Frank Gallo skrev:
>> Dear All:
>> I am a beginner with Stata, and I prefer to use Stata instead of SPSS
>> to address the following situation. I want to evaluate interrater
>> agreement. I have more than two raters (94), more than two ratings (49
>> variables), and varying number of raters (some missing values on the
>> variables). The data are in the form to use the kap command, however,
>> there is no statistic for testing kappa > 0 and no results reported
>> regarding the percentage of agreement. Is there some alternative that
>> I can use in Stata to obtain these results? I looked through the
>> archives and could not find one. SPSS offers the ICC option. I
>> appreciate any thoughts.
>> Best,
>> Frank
>> *
>> *   For searches and help try:
>> *
>> *
>> *
> *
> *   For searches and help try:
> *
> *
> *

*   For searches and help try:

Please view our annual report at

CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please reply to the sender immediately or by telephone at 413-794-0000 and destroy all copies of this communication and any attachments. For further information regarding Baystate Health's privacy policy, please visit our Internet site at

*   For searches and help try:

© Copyright 1996–2016 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index