[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
Re: st: composite kappa & intercenter variability in kappas
At 01:50 PM 8/17/2005, Richard_Lenhardt@rush.edu wrote:
We submitted a manuscript comparing two observers measuring 10 dicotomous
variables. We measured kappa, to assess concordance between observers. A
reviewer asked that we calculate an overall value (composite) of the
kappas, for all 10 variables. (Fleiss, Stat Methods for Rates and
Proportions, 2nd or 3rd edition).
I couldn't find such a function in stata (overall kappa). Would there be
another way to calculate this other than by hand?
Also, the study was completed at 5 different hospitals. The reviewer
asked that we take into account hospital to hospital variation when
calculating kappas. I wasn't aware that kappas could be adjusted for the
5 hospitals, other than generating a large matrix of kappas corresponding
to the different hospitals. Would there be a way to take into account the
variation in agreement among the hospitals when estimating the kappas?
Richard Lenhardt, MD, MPH
Assistant Professor of Medicine
Division of Pulmonary and Critical Care Medicine
Rush University Medical Center
Hospital can be used as a stratification factor.
You say you have 10 dichotomous variables that are being rated. This is
what you can do.
Take variable Y1 first. Compute stratum-specific kappas (for each hospital
separately). Then, compute a weighted-average combined kappa. Whether
that's reasonable depends on whether you are estimating the same kappa
parameter in all the strata -- eyeballing and a homogeneity test might help
here. In flavor, all this is like stratification and computation of an
adjusted MH odds ratio. I don't have Fleiss in front of me but the
specifics are in there.
Then do the same thing for the rest of your variables, Y2, ..., Y10.
You now have 10 "adjusted" kappas (controlling for hospital).
Should you compute an overall adjusted kappa across all 10 ratings? Maybe
yes, maybe no.
Could you do it? In principle, you could use the same procedure to get a
weighted kappa from all those 10 kappas. The trouble is that the individual
kappas are correlated beyond the fact that you are using the same raters --
you are also probably using the same patients. I think the only feasible
way to get a handle on such a beast is through the bootstrap.
Hope this helps.
The documents accompanying this transmission may contain confidential
health or business information. This information is intended for the use of
the individual or entity named above. If you have received this information
in error, please notify the sender immediately and arrange for the return
or destruction of these documents.
Constantine Daskalakis, ScD
Thomas Jefferson University, Division of Biostatistics,
211 S. 9th St., Suite 602, Philadelphia, PA 19107
* For searches and help try: