Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: RE: Re: Kappa for multiple raters and paired body parts


From   Garry Anderson <g.anderson@unimelb.edu.au>
To   statalist@hsphsun2.harvard.edu
Subject   st: RE: Re: Kappa for multiple raters and paired body parts
Date   Fri, 10 Dec 2010 18:51:25 +1100

Dear Statalist,

Upon Joseph's suggestion I have attempted a cross-classified random
effects model. However the intraclass correlation coefficeint of 0.95
seems to be too high because the inter-rater kappa for the right side
eye is 0.79 and 0.71 for the left side eye.  I think my syntax for the
cross-classified model is not correct. Any suggestions as to obtaning
the variance components for patient and rater would be welcome. [patient
= id].

. xtmelogit   y1ff    side  || _all:R.id || rater: ,var

Note: factor variables specified; option laplace assumed

Refining starting values: 

Iteration 0:   log likelihood = -532.18211  (not concave)
Iteration 1:   log likelihood =  -529.2315  
Iteration 2:   log likelihood =   -486.754  

Performing gradient-based optimization: 

Iteration 0:   log likelihood =   -486.754  (not concave)
Iteration 1:   log likelihood = -448.85311  (not concave)
Iteration 2:   log likelihood = -436.40142  (not concave)
Iteration 3:   log likelihood = -425.19299  
numerical derivatives are approximate
flat or discontinuous region encountered
Iteration 4:   log likelihood = -416.02159  
Iteration 5:   log likelihood = -414.48618  (not concave)
Iteration 6:   log likelihood = -414.25055  (not concave)
Iteration 7:   log likelihood = -414.18432  
Iteration 8:   log likelihood = -414.12319  
Iteration 9:   log likelihood = -414.12255  
Iteration 10:  log likelihood = -414.12255  

Mixed-effects logistic regression               Number of obs      =
1336

------------------------------------------------------------------------
--
                |   No. of       Observations per Group
Integration
 Group Variable |   Groups    Minimum    Average    Maximum      Points
----------------+-------------------------------------------------------
--
           _all |        1       1336     1336.0       1336           1
          rater |        4        334      334.0        334           1
------------------------------------------------------------------------
--

                                                Wald chi2(1)       =
3.57
Log likelihood = -414.12255                     Prob > chi2        =
0.0590

------------------------------------------------------------------------
------
        y1ff |      Coef.   Std. Err.      z    P>|z|     [95% Conf.
Interval]
-------------+----------------------------------------------------------
------
        side |  -.4258961   .2255334    -1.89   0.059    -.8679335
.0161413
       _cons |  -7.073155   .8871375    -7.97   0.000    -8.811913
-5.334398
------------------------------------------------------------------------
------

------------------------------------------------------------------------
------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf.
Interval]
-----------------------------+------------------------------------------
------
_all: Identity               |
                   var(R.id) |   60.34364   20.16911       31.3421
116.181
-----------------------------+------------------------------------------
------
rater: Identity              |
                  var(_cons) |   .0083921   .0378825      1.21e-06
58.37682
------------------------------------------------------------------------
------
LR test vs. logistic regression:     chi2(2) =   596.12   Prob > chi2 =
0.0000

Note: LR test is conservative and provided only for reference.
Note: log-likelihood calculations are based on the Laplacian
approximation.

. 
. matrix list e(b)

e(b)[1,4]
           eq1:        eq1:   lns1_1_1:   lns2_1_1:
          side       _cons       _cons       _cons
y1  -.42589608  -7.0731554   2.0500278   -2.390231

. nlcom (exp([lns1_1_1]:_cons)^2) /
((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_pi^2/3)

       _nl_1:  (exp([lns1_1_1]:_cons)^2) /
((exp([lns1_1_1]:_cons)^2)+(exp([lns2_1_1]:_cons)^2)+_p
> i^2/3)

------------------------------------------------------------------------
------
        y1ff |      Coef.   Std. Err.      z    P>|z|     [95% Conf.
Interval]
-------------+----------------------------------------------------------
------
       _nl_1 |   .9481747   .0163522    57.98   0.000     .9161249
.9802245
------------------------------------------------------------------------
------

. 


Kind regards, Garry


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Joseph
Coveney
Sent: Saturday, 20 November 2010 7:43 PM
To: statalist@hsphsun2.harvard.edu
Subject: st: Re: Kappa for multiple raters and paired body parts

Garry Anderson wrote:

I wish to estimate a single kappa (SE or 95%CI) when there are 4 raters
that each rate the left and right eyes of about 150 patients. The
response for each eye is binary. Estimation of kappa (SE) can be done
separately for the left eye and the right eye using -kap- or -kapci-,
however I am unsure as to how to include both eyes and take account of
the non-independence of eyes. Schouten (1993) describes the methodology
for two raters.

Schouten HJA (1993) Estimating kappa from binocular data and comparing
marginal probabilities. Statistics in Medicine 12: 2207-2217.

Any suggestions would be appreciated.

------------------------------------------------------------------------
--------

Well, kappa for binary scores is an intraclass correlation coefficient
(ICC).  
How about using -xtmelogit- to fit a cross-classified random-effects
model to the data with i.side (right or left eye) as a fixed effect, and
then use the patients' and raters' variance components, along with the
logistically distributed residual (pi^2 / 3), to compute the ICC
(patients' divided by the sum of patients', raters' and residual)?  You
can get the (transformed) variance components from the parameter vector,
e(b).  I'm guessing that bootstrapping is the best bet for the
confidence interval.  But -nlcom- is also worth looking into for this.

Joseph Coveney


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index