Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

# RE: st: RE: Kappa query - unequal distribution of scores

 From "Downs, Sara" <[email protected]> To <[email protected]> Subject RE: st: RE: Kappa query - unequal distribution of scores Date Tue, 27 Sep 2011 18:17:34 +0100

```Hi Nick,

Thanks for the advice which was helpful.

I also found these references that suggest that if the marginal totals are unbalanced the kappa statistic may provide an under or over estimate of association:
Feinstein & Cicchetti (1990) J Clin Epidemiol, vol 43, no 6, 543-549
Cicchetti & Feinstein (1990) J Clin Epidemiol, vol 43, no 6, 551-558
Sara

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Nick Cox
Sent: 22 September 2011 11:44
To: [email protected]
Subject: Re: st: RE: Kappa query - unequal distribution of scores

. bootstrap r(kappa), nodots reps(10000): kap rev1 rev2

[...]

Bootstrap results                               Number of obs      =        83
Replications       =     10000

command:  kap rev1 rev2
_bs_1:  r(kappa)

------------------------------------------------------------------------------
|   Observed   Bootstrap                         Normal-based
|      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------
-------------+------
_bs_1 |    .020649   .0983171     0.21   0.834     -.172049     .213347
------------------------------------------------------------------------------

The s.e. is a bit higher. In this case, as kappa is almost 0, it is pretty much immaterial what s.e. you get.

Nick

On Thu, Sep 22, 2011 at 10:36 AM, Nick Cox <[email protected]> wrote:
> I'd think about bootstrapping here.

> On Thu, Sep 22, 2011 at 10:06 AM, Downs, Sara
> <[email protected]> wrote:
>> Hi,
>>
>> I am trying to calculate agreement between 2 distributions of scores.
>> There are 3 possible scores: One score predominates and the other 2
>> are extremely rare :
>>
>>  tab rev1 rev2
>>
>>           |               rev2
>>      rev1 |         0          1          2 |     Total
>> -----------+---------------------------------+----------
>>         0 |         1          6          0 |         7
>>         1 |        13         57          3 |        73
>>         2 |         0          2          1 |         3
>> -----------+---------------------------------+----------
>>     Total |        14         65          4 |        83
>>
>>
>> .         kap rev1 rev2
>>
>>             Expected
>> Agreement   Agreement     Kappa   Std. Err.         Z      Prob>Z
>> -----------------------------------------------------------------
>>  71.08%      70.47%     0.0206     0.0852       0.24      0.4042
>>
>>
>> I understand that the usual kappa statistic is not reliable in this
>> situation .   Is there any way in STATA that I can adjust for the
>> high expected frequency. I am using STATA 10.1.
>>
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Animal Health and Veterinary Laboratories Agency (AHVLA)

This email and any attachments is intended for the named recipient only.
If you have received it in error you have no authority to use, disclose,
store or copy any of its contents and you should destroy it and inform the sender.
Whilst this email and associated attachments will have been checked for known viruses whilst within AHVLA systems we can accept no responsibility once it has left our systems.
Communications on AHVLA computer systems may be monitored and/or recorded to secure the effective operation of the system and for other lawful purposes.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```

© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index