Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: RE: bootstrap confidence intervals for multiple rater kappa


From   [email protected]
To   [email protected]
Subject   Re: st: RE: bootstrap confidence intervals for multiple rater kappa
Date   Fri, 23 Jun 2006 09:24:54 +1000

Thankyou very much for your help, David. It was most appreciated.
Bernadette

Quoting David Harrison <[email protected]>:

> Bernadette
> 
> You can calculate an analytical confidence interval for kappa for
> multiple raters and a binary rating using the method of Zou & Donner
> (Biometrics 2004) - the commands -kappci- and -kappaci- to do this
> calculation (for -kap- and -kappa- formatted data, respectively) are
> included in my package -kaputil- available from SSC (-ssc describe
> kaputil-).
> 
> Alternatively, you may want to take a look at the command -kapci-, from
> Stata J 4-4 (-findit kapci-), which does bootstrap cis for kappa,
> although my recollection is that the data need to be in the format for
> -kap- for this one.
> 
> You should note, though, that both -kap- and -kappa- require one row per
> subject rather than one row per rater. You can get from one form to the
> other with two -reshape-s, e.g. suppose you have a variable rater taking
> values 1-149 and 14 variables rating1-rating14 for the ratings of the 14
> videos...
> 
> reshape long rating, i(rater) j(video)
> reshape wide rating, i(video) j(rater)
> 
> Now you have a variable video taking values 1-14 and 149 variables
> rating1-rating149 for the ratings of the 149 raters, and you can use
> -kap- (or -kappci- or -kapci- or do your own bootstrapping).
> 
> Hope this helps
> 
> David
> 
> 
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> [email protected]
> Sent: 20 June 2006 05:18
> To: [email protected]
> Subject: st: bootstrap confidence intervals for multiple rater kappa
> 
> I have a dataset where 149 raters have classified 14 video scenarios
> into a binary (1,0) classification system on two occasions, once before
> and once after an education intervention. I'm interested in seeing
> whether the education intervention improved the agreement of the raters
> in classifying the video scenarios. I can use the "kappa" command on a
> table summarising this dataset to calculate the kappa statistic across
> the 14 scenarios on each occasion. I would now like to calculate
> bootstrapped confidence intervals for these and also compare kappa
> statistics between the two assessment occasions. 
> However, I would like the bootstrap procedure to resample raters rather
> than the 14 video scenarios. Can anyone tell me how I can calculate
> kappa in the case of more than two raters using a dataset where each
> rater and their classifications of the video scenarios are presented as
> an individual row in the dataset so that these can be resampled when I
> perform the bootstrap procedure?
> Bernadette Massey
> 
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> ______________________________________________________________________
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 



*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index