Statalist The Stata Listserver

[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: RE: bootstrap confidence intervals for multiple rater kappa

Subject   Re: st: RE: bootstrap confidence intervals for multiple rater kappa
Date   Fri, 23 Jun 2006 09:24:54 +1000

Thankyou very much for your help, David. It was most appreciated.

Quoting David Harrison <>:

> Bernadette
> You can calculate an analytical confidence interval for kappa for
> multiple raters and a binary rating using the method of Zou & Donner
> (Biometrics 2004) - the commands -kappci- and -kappaci- to do this
> calculation (for -kap- and -kappa- formatted data, respectively) are
> included in my package -kaputil- available from SSC (-ssc describe
> kaputil-).
> Alternatively, you may want to take a look at the command -kapci-, from
> Stata J 4-4 (-findit kapci-), which does bootstrap cis for kappa,
> although my recollection is that the data need to be in the format for
> -kap- for this one.
> You should note, though, that both -kap- and -kappa- require one row per
> subject rather than one row per rater. You can get from one form to the
> other with two -reshape-s, e.g. suppose you have a variable rater taking
> values 1-149 and 14 variables rating1-rating14 for the ratings of the 14
> videos...
> reshape long rating, i(rater) j(video)
> reshape wide rating, i(video) j(rater)
> Now you have a variable video taking values 1-14 and 149 variables
> rating1-rating149 for the ratings of the 149 raters, and you can use
> -kap- (or -kappci- or -kapci- or do your own bootstrapping).
> Hope this helps
> David
> -----Original Message-----
> From:
> [] On Behalf Of
> Sent: 20 June 2006 05:18
> To:
> Subject: st: bootstrap confidence intervals for multiple rater kappa
> I have a dataset where 149 raters have classified 14 video scenarios
> into a binary (1,0) classification system on two occasions, once before
> and once after an education intervention. I'm interested in seeing
> whether the education intervention improved the agreement of the raters
> in classifying the video scenarios. I can use the "kappa" command on a
> table summarising this dataset to calculate the kappa statistic across
> the 14 scenarios on each occasion. I would now like to calculate
> bootstrapped confidence intervals for these and also compare kappa
> statistics between the two assessment occasions. 
> However, I would like the bootstrap procedure to resample raters rather
> than the 14 video scenarios. Can anyone tell me how I can calculate
> kappa in the case of more than two raters using a dataset where each
> rater and their classifications of the video scenarios are presented as
> an individual row in the dataset so that these can be resampled when I
> perform the bootstrap procedure?
> Bernadette Massey
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit 
> ______________________________________________________________________
> *
> *   For searches and help try:
> *
> *
> *

*   For searches and help try:

© Copyright 1996–2022 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index