Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
st: RE: Re: Kappa for multiple raters and paired body parts
From
Garry Anderson <[email protected]>
To
[email protected]
Subject
st: RE: Re: Kappa for multiple raters and paired body parts
Date
Fri, 10 Dec 2010 18:51:25 +1100
Dear Statalist,
Upon Joseph's suggestion I have attempted a cross-classified random
effects model. However the intraclass correlation coefficeint of 0.95
seems to be too high because the inter-rater kappa for the right side
eye is 0.79 and 0.71 for the left side eye. I think my syntax for the
cross-classified model is not correct. Any suggestions as to obtaning
the variance components for patient and rater would be welcome. [patient
= id].
. xtmelogit y1ff side || _all:R.id || rater: ,var
Note: factor variables specified; option laplace assumed
Refining starting values:
Iteration 0: log likelihood = -532.18211 (not concave)
Iteration 1: log likelihood = -529.2315
Iteration 2: log likelihood = -486.754
Performing gradient-based optimization:
Iteration 0: log likelihood = -486.754 (not concave)
Iteration 1: log likelihood = -448.85311 (not concave)
Iteration 2: log likelihood = -436.40142 (not concave)
Iteration 3: log likelihood = -425.19299
numerical derivatives are approximate
flat or discontinuous region encountered
Iteration 4: log likelihood = -416.02159
Iteration 5: log likelihood = -414.48618 (not concave)
Iteration 6: log likelihood = -414.25055 (not concave)
Iteration 7: log likelihood = -414.18432
Iteration 8: log likelihood = -414.12319
Iteration 9: log likelihood = -414.12255
Iteration 10: log likelihood = -414.12255
Mixed-effects logistic regression Number of obs =
1336
------------------------------------------------------------------------
--
| No. of Observations per Group
Integration
Group Variable | Groups Minimum Average Maximum Points
----------------+-------------------------------------------------------
--
_all | 1 1336 1336.0 1336 1
rater | 4 334 334.0 334 1
------------------------------------------------------------------------
--
Wald chi2(1) =
3.57
Log likelihood = -414.12255 Prob > chi2 =
0.0590
------------------------------------------------------------------------
------
y1ff | Coef. Std. Err. z P>|z| [95% Conf.
Interval]
-------------+----------------------------------------------------------
------
side | -.4258961 .2255334 -1.89 0.059 -.8679335
.0161413
_cons | -7.073155 .8871375 -7.97 0.000 -8.811913
-5.334398
------------------------------------------------------------------------
------
------------------------------------------------------------------------
------
Random-effects Parameters | Estimate Std. Err. [95% Conf.
Interval]
-----------------------------+------------------------------------------
------
_all: Identity |
var(R.id) | 60.34364 20.16911 31.3421
116.181
-----------------------------+------------------------------------------
------
rater: Identity |
var(_cons) | .0083921 .0378825 1.21e-06
58.37682
------------------------------------------------------------------------
------
LR test vs. logistic regression: chi2(2) = 596.12 Prob > chi2 =
0.0000
Note: LR test is conservative and provided only for reference.
Note: log-likelihood calculations are based on the Laplacian
approximation.
.
. matrix list e(b)
e(b)[1,4]
eq1: eq1: lns1_1_1: lns2_1_1:
side _cons _cons _cons
y1 -.42589608 -7.0731554 2.0500278 -2.390231
. nlcom (exp([lns1_1_1]:_cons)^2) /
((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_pi^2/3)
_nl_1: (exp([lns1_1_1]:_cons)^2) /
((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_p
> i^2/3)
------------------------------------------------------------------------
------
y1ff | Coef. Std. Err. z P>|z| [95% Conf.
Interval]
-------------+----------------------------------------------------------
------
_nl_1 | .9481747 .0163522 57.98 0.000 .9161249
.9802245
------------------------------------------------------------------------
------
.
Kind regards, Garry
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Joseph
Coveney
Sent: Saturday, 20 November 2010 7:43 PM
To: [email protected]
Subject: st: Re: Kappa for multiple raters and paired body parts
Garry Anderson wrote:
I wish to estimate a single kappa (SE or 95%CI) when there are 4 raters
that each rate the left and right eyes of about 150 patients. The
response for each eye is binary. Estimation of kappa (SE) can be done
separately for the left eye and the right eye using -kap- or -kapci-,
however I am unsure as to how to include both eyes and take account of
the non-independence of eyes. Schouten (1993) describes the methodology
for two raters.
Schouten HJA (1993) Estimating kappa from binocular data and comparing
marginal probabilities. Statistics in Medicine 12: 2207-2217.
Any suggestions would be appreciated.
------------------------------------------------------------------------
--------
Well, kappa for binary scores is an intraclass correlation coefficient
(ICC).
How about using -xtmelogit- to fit a cross-classified random-effects
model to the data with i.side (right or left eye) as a fixed effect, and
then use the patients' and raters' variance components, along with the
logistically distributed residual (pi^2 / 3), to compute the ICC
(patients' divided by the sum of patients', raters' and residual)? You
can get the (transformed) variance components from the parameter vector,
e(b). I'm guessing that bootstrapping is the best bet for the
confidence interval. But -nlcom- is also worth looking into for this.
Joseph Coveney
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/