[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
sjsamuels@gmail.com |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: Kappa Weights |

Date |
Wed, 4 Nov 2009 16:51:31 -0500 |

Theron's description is vague and confusing. I think he has a 2 x 2 (a b \ c d) table of rater/test agreements (+/-) Conditional kappa positive for the would compute kappa that compares observed and expected agreements in the "a" cell alone Theron's statement that the drug test has only one rating (true positive) contradicts what he elsewhere says about the data: that there were a large number of negatives. I think that he means that the tests are all positive by some gold standard. But he might mean that it was rated positive by one rater or another. Or he just might means that he is interested only in the positive agreements. Now in principle, either rater (test?) could give the drug a negative rating, unless one of them *is* the gold standard. If that is the case, Ronan is correct: Kappa is inappropriate and Theron should just give the proportion of the time that the second rater is right. Apparently, in Theron's table, only one rater declared that any samples were negative. If so, conditional kappa is zero (observed a = expected a) and his computation is finished. None of Stata's commands will compute conditional kappa directly, but it is possible to estimate it indirectly with -mvreg- followed by -nlcom- There are four multinomial cells in the 2 x 2 table. Number them 1,2,3,4, to correspond to the 11, 12, 21, 22 cells, respectively 1. Create a four line data set, with one line for each of the four table cells, a cell number, and the count of observations in that cell. 2. Create 0-1 indicator variables for each observation to indicate the table cell from which it comes, say x1 x2 x3 x4, using -tab, gen()- or by hand. 3. Create a constant variable k = 1. 3. Run the command mvreg x1 x2 x3= k [fweight = count] (If one were interested in the negative conditional kappa, one would use x2 x3 x4 as predictors). 4. The constants for x1 and x2 x3 are just the sample proportions in the a, b, and c cells. and can be accessed as [x1]_cons [x2]_cons [x3]_cons 6. Positive Conditional kappa is a function of these three proportions. Use -nlcom- to estimate it. 7. To get valid standard errors, -jackknife- or -bootstrap- -mvreg- and follow by -nlcom- -Steve On Wed, Nov 4, 2009 at 10:52 AM, Jackson, Theron Keith (UMSL-Student) <tkjgx2@umsl.edu> wrote: > The drug test has only one rating (True Positives). Due to the large majority of the sample being (True Negatives) the Kappa will be biased (Bishop et al., 1975; Magura et al, 1987; Magura et al, 1996; Sherman et al., 1992; Patton, et al., 2005). > > "A more appropriate indicator of agreement between self-reports and urinalysis for most of the studies reviewed is a conditional kappa coefficient...The self-reports of subjects with negative urinalyses are ignored in the computation of conditional Kappa." (Magura et al., 1987 p. 734). > > It appears to me that there is no direct way to compute conditional kappa; however, I should be able to weight kappa so that those who test positive are given all the weight of the sample. > Ronan Conroy wrote: > > Are you telling us that you have only three participants? Then the > advice is to get more. > > Or are you telling us that the drug test has only one rating > (positive) because it is correct? In this case, your whole sample is > made up of true positives. In a case like this, kappa is > inappropriate. The level of agreement reduces to the proportion of > self-reports (rating 2) which are positive. > > Steven Samuels sjsamuels@gmail.com 18 Cantine's Island Saugerties NY 12477 USA 845-246-0774 * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**st: Kappa Weights***From:*sjsamuels@gmail.com

**References**:**st: Kappa Weights***From:*"Jackson, Theron Keith (UMSL-Student)" <tkjgx2@umsl.edu>

**Re: st: Kappa Weights***From:*Ronan Conroy <rconroy@rcsi.ie>

**RE: st: Kappa Weights***From:*"Jackson, Theron Keith (UMSL-Student)" <tkjgx2@umsl.edu>

- Prev by Date:
**st: Re: RSS feed for Statalist** - Next by Date:
**st: Kappa Weights** - Previous by thread:
**RE: st: Kappa Weights** - Next by thread:
**st: Kappa Weights** - Index(es):

© Copyright 1996–2014 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |