Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Garry Anderson <g.anderson@unimelb.edu.au> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
st: RE: Re: Kappa for multiple raters and paired body parts |

Date |
Fri, 10 Dec 2010 18:51:25 +1100 |

Dear Statalist, Upon Joseph's suggestion I have attempted a cross-classified random effects model. However the intraclass correlation coefficeint of 0.95 seems to be too high because the inter-rater kappa for the right side eye is 0.79 and 0.71 for the left side eye. I think my syntax for the cross-classified model is not correct. Any suggestions as to obtaning the variance components for patient and rater would be welcome. [patient = id]. . xtmelogit y1ff side || _all:R.id || rater: ,var Note: factor variables specified; option laplace assumed Refining starting values: Iteration 0: log likelihood = -532.18211 (not concave) Iteration 1: log likelihood = -529.2315 Iteration 2: log likelihood = -486.754 Performing gradient-based optimization: Iteration 0: log likelihood = -486.754 (not concave) Iteration 1: log likelihood = -448.85311 (not concave) Iteration 2: log likelihood = -436.40142 (not concave) Iteration 3: log likelihood = -425.19299 numerical derivatives are approximate flat or discontinuous region encountered Iteration 4: log likelihood = -416.02159 Iteration 5: log likelihood = -414.48618 (not concave) Iteration 6: log likelihood = -414.25055 (not concave) Iteration 7: log likelihood = -414.18432 Iteration 8: log likelihood = -414.12319 Iteration 9: log likelihood = -414.12255 Iteration 10: log likelihood = -414.12255 Mixed-effects logistic regression Number of obs = 1336 ------------------------------------------------------------------------ -- | No. of Observations per Group Integration Group Variable | Groups Minimum Average Maximum Points ----------------+------------------------------------------------------- -- _all | 1 1336 1336.0 1336 1 rater | 4 334 334.0 334 1 ------------------------------------------------------------------------ -- Wald chi2(1) = 3.57 Log likelihood = -414.12255 Prob > chi2 = 0.0590 ------------------------------------------------------------------------ ------ y1ff | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------- ------ side | -.4258961 .2255334 -1.89 0.059 -.8679335 .0161413 _cons | -7.073155 .8871375 -7.97 0.000 -8.811913 -5.334398 ------------------------------------------------------------------------ ------ ------------------------------------------------------------------------ ------ Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval] -----------------------------+------------------------------------------ ------ _all: Identity | var(R.id) | 60.34364 20.16911 31.3421 116.181 -----------------------------+------------------------------------------ ------ rater: Identity | var(_cons) | .0083921 .0378825 1.21e-06 58.37682 ------------------------------------------------------------------------ ------ LR test vs. logistic regression: chi2(2) = 596.12 Prob > chi2 = 0.0000 Note: LR test is conservative and provided only for reference. Note: log-likelihood calculations are based on the Laplacian approximation. . . matrix list e(b) e(b)[1,4] eq1: eq1: lns1_1_1: lns2_1_1: side _cons _cons _cons y1 -.42589608 -7.0731554 2.0500278 -2.390231 . nlcom (exp([lns1_1_1]:_cons)^2) / ((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_pi^2/3) _nl_1: (exp([lns1_1_1]:_cons)^2) / ((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_p > i^2/3) ------------------------------------------------------------------------ ------ y1ff | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------- ------ _nl_1 | .9481747 .0163522 57.98 0.000 .9161249 .9802245 ------------------------------------------------------------------------ ------ . Kind regards, Garry -----Original Message----- From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Joseph Coveney Sent: Saturday, 20 November 2010 7:43 PM To: statalist@hsphsun2.harvard.edu Subject: st: Re: Kappa for multiple raters and paired body parts Garry Anderson wrote: I wish to estimate a single kappa (SE or 95%CI) when there are 4 raters that each rate the left and right eyes of about 150 patients. The response for each eye is binary. Estimation of kappa (SE) can be done separately for the left eye and the right eye using -kap- or -kapci-, however I am unsure as to how to include both eyes and take account of the non-independence of eyes. Schouten (1993) describes the methodology for two raters. Schouten HJA (1993) Estimating kappa from binocular data and comparing marginal probabilities. Statistics in Medicine 12: 2207-2217. Any suggestions would be appreciated. ------------------------------------------------------------------------ -------- Well, kappa for binary scores is an intraclass correlation coefficient (ICC). How about using -xtmelogit- to fit a cross-classified random-effects model to the data with i.side (right or left eye) as a fixed effect, and then use the patients' and raters' variance components, along with the logistically distributed residual (pi^2 / 3), to compute the ICC (patients' divided by the sum of patients', raters' and residual)? You can get the (transformed) variance components from the parameter vector, e(b). I'm guessing that bootstrapping is the best bet for the confidence interval. But -nlcom- is also worth looking into for this. Joseph Coveney * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

- Prev by Date:
**Re: st: RE: RE: do file: t-score, dfuller, to sw regress** - Next by Date:
**st: Margins after mi sqreg** - Previous by thread:
**st: How to caculate the 90th percentile and 50th percentile of the wage distribution in my data set using stata?** - Next by thread:
**st: Margins after mi sqreg** - Index(es):