Emma
You should take a look at
http://www.stata.com/support/faqs/stat/anova2.html
The problem may lie with your data structure. Consider using a long
rather than wide format, with subjects forming the rows. Each row
should be something like
subjectid judgeid occasionid score
i.e some rehape-ing may be helpful.
Then score is the appropriate outcome variable for the anova.
You may also find the contributed command wsanova (within-subjects
anova) easier to comprehend than anova, though I don't think it gives
icc by default.
Hope some of this helps!
Steve
Date: Mon, 22 Sep 2003 16:47:54 +0100
From: Emma <[email protected]>
Subject: Re: st: RE: inter and intra rater
Dear Nick, (and statalist)
Many thanks for your prompt response to my initial question.
Earlier I sent the question below to statalist, but somewhat
confusingly I
only sent a subset of the data for the small disease group, below is
the
data for the entire set. where rater is the individual who took the
measure, 6 individuals, subject 1 had small disease, subject 2 had
moderate
disease and subject 3 had large disease. and repeats 1 and 2 were the
two
repeated measurements taken by the raters.
ratersubject 1subject 1subject 2subject 2subject 3subject 3
repeat1repeat2repeat1repeat2repeat1repeat2
JT22.522.535.5356867.5
KD22.522.536.535.567.568.5
WD22.522.536.536.56768
NC222335.5366767
RP22.52236.536.56868.5
ES21.52336356867.5
I wondered if anyone has any suggestions to look at the three subjects
together, whilst checking for inter rater and intra rater reliability,
Kindest regards and thanks,
Emma
At 16:18 22/09/03 +0100, you wrote:
>Emma
>
> > I wonder if someone could help me with the following, I am
> > looking to
> > calculate intra rater and inter rater correlations on the
> > following data
> > set. A continuous variable has been measured by 6 trainees on three
> > different subjects (the three subjects have differing
> > degrees of disease:
> > small, moderate and large). All 6 trainees repeated the
> > measurements on
> > the three subjects on two separate occasions.
> >
> > I have applied ICC as a measure for the agreement between
> > the 6 raters;
> > however STATA outputs 0* for the ICC whereas the data
> > appears to show a
> > reasonable agreement in measures between raters:
> >
> > rater time1 time2
> > JT 22.5 22.5
> > KD 22.5 22.5
> > WD 22.5 22.5
> > NC 22 23
> > RP 22.5 22
> > ES 21.5 23
> >
> >
> > . loneway var1 rater
> >
> > One-way Analysis of Variance for var1: measure2
> >
> > Number of
> > obs = 12
>
> > R-squared = 0.0870
> >
> > Source SS df MS
> > F Prob > F
> > ------------------------------------------------------------
> > -------------
> > Between rater .16666667 5 .03333333
> > 0.11 0.9845
> > Within rater 1.75 6 .29166667
> > ------------------------------------------------------------
> > -------------
> > Total 1.9166667 11 .17424242
> >
> > Intraclass Asy.
> > correlation S.E. [95% Conf. Interval]
> > ------------------------------------------------
> > 0.00000* 0.42817 0.00000 0.83921
> >
> > Estimated SD of rater effect .
> > Estimated SD within rater .5400617
> > Est. reliability of a rater mean 0.00000*
> > (evaluated at n=2.00)
> >
> > (*) Truncated at zero.
> >
> >
> > Also I wondered what measure I should use to consider the
> > agreement between
> > the repeat measures (should this be ICC also, and if so how
> > should the data
> > be set up, apologies to ask a basic question). Finally, is
> > there anyway to
> > consider the three subjects data together, i.e to combine
> > the info for the
> > small, moderate and large.
>
>One comment only, as I am mostly in the dark
>here: it is not clear to me how the subset of data
>you give is related to the analysis you
>report or to the problem you describe.
>
>Nevertheless, focusing on that subset alone,
>although the "agreement" between raters is close
>in the sense that all ratings are 21.5-23, which presumably
>is some small fraction of the possible range, the
>correlation (classic sense) is nevertheless
>strong and _negative_.
>
>However, another way of thinking about it is that
>your data points may collectively be one big blob.
>
>I guess wildly that this is consistent
>with small ICC.
>
>Nick
>[email protected]
----------------------
Stephen McKay
PFRC, School of Geographical Sciences
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
