Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

# Re: st: RE: Kappa query - unequal distribution of scores

 From Nick Cox To statalist@hsphsun2.harvard.edu Subject Re: st: RE: Kappa query - unequal distribution of scores Date Thu, 22 Sep 2011 11:43:54 +0100

```. bootstrap r(kappa), nodots reps(10000): kap rev1 rev2

[...]

Bootstrap results                               Number of obs      =        83
Replications       =     10000

command:  kap rev1 rev2
_bs_1:  r(kappa)

------------------------------------------------------------------------------
|   Observed   Bootstrap                         Normal-based
|      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
_bs_1 |    .020649   .0983171     0.21   0.834     -.172049     .213347
------------------------------------------------------------------------------

The s.e. is a bit higher. In this case, as kappa is almost 0, it is
pretty much immaterial what s.e. you get.

Nick

On Thu, Sep 22, 2011 at 10:36 AM, Nick Cox <njcoxstata@gmail.com> wrote:
> I'd think about bootstrapping here.

> On Thu, Sep 22, 2011 at 10:06 AM, Downs, Sara
> <Sara.Downs@ahvla.gsi.gov.uk> wrote:
>> Hi,
>>
>> I am trying to calculate agreement between 2 distributions of scores.
>> There are 3 possible scores: One score predominates and the other 2 are
>> extremely rare :
>>
>>  tab rev1 rev2
>>
>>           |               rev2
>>      rev1 |         0          1          2 |     Total
>> -----------+---------------------------------+----------
>>         0 |         1          6          0 |         7
>>         1 |        13         57          3 |        73
>>         2 |         0          2          1 |         3
>> -----------+---------------------------------+----------
>>     Total |        14         65          4 |        83
>>
>>
>> .         kap rev1 rev2
>>
>>             Expected
>> Agreement   Agreement     Kappa   Std. Err.         Z      Prob>Z
>> -----------------------------------------------------------------
>>  71.08%      70.47%     0.0206     0.0852       0.24      0.4042
>>
>>
>> I understand that the usual kappa statistic is not reliable in this
>> situation .   Is there any way in STATA that I can adjust for the high
>> expected frequency. I am using STATA 10.1.
>>
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```