# Re: st: RE: Interobserver agreement for continuous variables

 From Roger Newson To statalist@hsphsun2.harvard.edu Subject Re: st: RE: Interobserver agreement for continuous variables Date Fri, 18 Oct 2002 12:26:40 +0100

```At 21:43 17/10/02 -0700, Nitin wrote:
```
```Hi Nick and Scott,
Thank you for the response. But the concord and other
tests contained in it are only for 2 observers/raters.
I am looking for something that calculates
interobserver agreement for a continuous variable WHEN
THERE ARE MORE THAN 3 RATERS AND THE number of raters
vary for each subject ?
I look forward to hearing from you.
Nitin
```
I personally would use a variation on the intra-class Kendall's tau-a, which can be calculated (with confidence limits) using my -somersd- package, downloadable from SSC. (Type -ssc describe somersd- or -net search somersd- within Stata to find how to download.) First, create a data set with 1 obs for every ordered pair of measurements on the same subject by different observers. Such a data set might have variables as follows:

subjid Subject ID
obsid1 First observer ID
obsid2 Second observer ID
y1 First observer rating
y2 Second observer rating

Such a data set might be created using the -joinby- package. Note that each combination of Observer A, Observer B and Patient X should appear twice in this data set, once with Observer A as first observer and Observer B as second observer, and once with Observer B as first observer and Observer A as second observer. Once you have created this data set, type

somersd y1 y2,taua tdist cluster(subjid)

and Stata will calculate confidence limits for a Kendall tau-a between -y1- and -y2-, clustered by -subjid-. This tau-a is a difference between 2 probabilities, namely the probability of concordance (two raters order two subjects in the same way) and the probability of discordance (two raters order two subjects in the opposite way). For instance, if Kendall's tau-a is 0.70, then this means that, if 2 observers are given 2 randomly chosen subjects and asked which one to rate more highly, then the observers are 70% more likely to agree than to disagree.

For more information on Kendall's tau-a, a possible reference is Newson (2002), which in turn has many more references.

I hope this helps.

Roger

References

Newson R. Parameters behind "non-parametric" statistics: Kendall's tau, Somers' D and median differences. The Stata Journal 2002; 2(1): 45-64.

--
Roger Newson
Lecturer in Medical Statistics
Department of Public Health Sciences
King's College London
5th Floor, Capital House
42 Weston Street
London SE1 3QD
United Kingdom

Tel: 020 7848 6648 International +44 20 7848 6648
Fax: 020 7848 6620 International +44 20 7848 6620
or 020 7848 6605 International +44 20 7848 6605
Email: roger.newson@kcl.ac.uk

Opinions expressed are those of the author, not the institution.

*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/