# st: Calculating weighted kappa for multiple raters

 From "Karen Wright" To Subject st: Calculating weighted kappa for multiple raters Date Thu, 03 Dec 2009 16:45:58 +0000

```Dear list,

I have a problem that is perhaps more to do withprogramming than anything else.  i am not experieced in programming stata and so have given up writing an ado-file program for whta I am trying to do, but unfortunately I am stil encountering problems!

I am trying to calculate weighted kappa for multiple raters, i have attached a small word document with the equation.  My problem occurs when I am trying to calculate marginal totals.  I am trying to create a total of the frequency for each rater, within each category and multiply these together, as shown in the equation.

The current method i am employing to do this is :

forval a=1/`r' {
loc A_`a' = r(sum)
loc B_`a' = r(sum)
loc C_`a' = r(sum)
}
forval a=1/4 {
forval b=1/4 {
forval c=1/4 {
}
}
}
egen den = rowtotal(d_111-d_`r'`r'`r')
drop d_*
summ(den)
loc denom = r(sum)
di `denom'

(i have attached my full code in a do-file)

But this becomes very lengthy and produces too many variables to do for any sufficient amount of raters.  Eventually i hope to analyse gradings from >100 raters, and yet with my current methods stata cannot compute it for only 8!  too many variables created.

Is there another way anyone with much better stata programming experience than me can see to proceed?

Thank you, any help is much appreciated!
Karen

The Institute of Cancer Research: Royal Cancer Hospital, a charitable Company Limited by Guarantee, Registered in England under Company No. 534147 with its Registered Office at 123 Old Brompton Road, London SW7 3RP.

This e-mail message is confidential and for use by the addressee only.  If the message is received by anyone other than the addressee, please return the message to the sender by replying to it and then delete the message from your computer and network.```

Attachment: Weighted kappa equation.doc
Description: MS-Word document