Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: "adjusted" kappas

Subject   st: "adjusted" kappas
Date   Thu, 18 Aug 2005 17:23:51 -0500

t 01:50 PM 8/17/2005, wrote:


We submitted a manuscript comparing two observers measuring 10 dicotomous
variables.  We measured kappa, to assess concordance between observers.  A
reviewer asked that we calculate an overall value (composite) of the
kappas, for all 10 variables.   (Fleiss, Stat Methods for Rates and
Proportions, 2nd or 3rd edition).

I couldn't find such a function in stata (overall kappa).  Would there be
another way to calculate this other than by hand?

Also, the study was completed at 5 different hospitals.  The reviewer
asked that we take into account hospital to hospital variation when
calculating kappas.  I wasn't aware that kappas could be adjusted for the
5 hospitals, other than generating a large matrix of kappas corresponding
to the different hospitals.  Would there be a way to take into account the
variation in agreement among the hospitals when estimating the kappas?

Thank you!

Richard Lenhardt, MD, MPH
Assistant Professor of Medicine
Division of Pulmonary and Critical Care Medicine
Rush University Medical Center
Chicago, IL

Hospital can be used as a stratification factor.
You say you have 10 dichotomous variables that are being rated. This is 
what you can do.

Take variable Y1 first. Compute stratum-specific kappas (for each hospital 
separately). Then, compute a weighted-average combined kappa. Whether 
that's reasonable depends on whether you are estimating the same kappa 
parameter in all the strata -- eyeballing and a homogeneity test might 
help here. In flavor, all this is like stratification and computation of 
an adjusted MH odds ratio. I don't have Fleiss in front of me but the 
specifics are in there.

Then do the same thing for the rest of your variables, Y2, ..., Y10.

You now have 10 "adjusted" kappas (controlling for hospital).

Should you compute an overall adjusted kappa across all 10 ratings? Maybe 
yes, maybe no.

Could you do it? In principle, you could use the same procedure to get a 
weighted kappa from all those 10 kappas. The trouble is that the 
individual kappas are correlated beyond the fact that you are using the 
same raters -- you are also probably using the same patients. I think the 
only feasible way to get a handle on such a beast is through the 

Hope this helps.


The documents accompanying this transmission may contain confidential 
health or business information. This information is intended for the use 
of the individual or entity named above. If you have received this 
information in error, please notify the sender immediately and arrange for 
the return or destruction of these documents.

Constantine Daskalakis, ScD
Assistant Professor,
Thomas Jefferson University, Division of Biostatistics,
211 S. 9th St., Suite 602, Philadelphia, PA 19107
Tel: 215-955-5695
Fax: 215-503-3804

Thank you for your help, now I understand what to do with stata.  This 
brings me to a follow up question (for anyone): 

To calculate an "adjusted" kappa, based on 5 hospitals, according to 
Fleiss, one would sum the hospital kappas, weighted by the inverse of the 
square of the hospital kappa standard errors. 

     adjusted kappa for given item =      SUM  kappahosp(i)              / 
     SUM 1/se(kappahosp(i))^2
 se(kappahosp(i))^2        /

I'm trying to write a do program that enables this with the following 

Study ID   HospID     A1    B1     A2    B2    ....
1               1       Y     Y        Y     N 
2               1          N     N        Y     Y 
 and so on 

statsby "kap A1 B1" k1=r(kappa) se1=R(se), by(hospid)saving(temp1) replace
use temp1, clear

gen adjustedkappa= 

I can't seem to figure out how to generate a variable "adjustedkappa" 
whereby the variable is the sum of the 5 kappas calculated earlier for 
each individual hospital. 

I'd like to do the same for the the std error term and the 95% confidence 
interval.  The std error = square root(SUM 1/se(kappahosp(i))^2)

Finally, is there a way to program this, so that I could simply write: 

statsby "kap X Y"  k=r(kappa) se=R(se), by(hospid)saving(temp1) replace

so I don't have to re-write a statsby line for each item that was observed 
(A1/B2, A2/B2, A3/B3, ...)

Thanks so much! 

Richard Lenhardt, MD, MPH 
Assistant Professor of Medicine
Division of Pulmonary and Critical Care Medicine
Rush University Medical Center
Chicago, IL 

*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index