Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Kappa weights & category detection

From   Ronan Conroy <>
To   "" <>
Subject   Re: st: Kappa weights & category detection
Date   Fri, 11 Feb 2011 10:43:31 +0000

The main limitation is that of your data. The potential range of psychiatric diagnosis is large, and you can tap very little of this with a small study. 

That said, rather than weighting Kappa, which would produce an arbitrary scoring system, I would report my results under

1. Agreement on headline diagnosis
Under this header, only clinically significant disagreements would count. So if you are using DSM-IV, if both raters diagnosed depressive disorder of any sort (any 369 code) the sub-classification would be ignored. 

2. Agreement in detail
Precise agreement on diagnosis

Rather than subsuming all disagreement into a global statistic, it would be interesting to see what were the commonest disagreements (you would be surprised by substantial disagreement about obsessive compulsive disorder, but I think we would all expect some variation in mood disorders). 

Sometimes an overall statistic can obscure the interesting questions.


On 10 Feb 2011, at 20:25, James Kirkbride wrote:

> Hello Stata Users
> I am trying to calculate some Kappa values for two raters who have each 
> rated the same 10 case notes to produce a mental health diagnosis. Their 
> diagnosis can be one of 16 possible categories. Obviously not all 16 
> categories are used, so I specify a weight matrix to let stata know 
> there are 16 possible categories that could have been chosen.
> However I am unsure stata has taken this into account as when I run the 
> analysis I get the same kappa statistic with or without the weight 
> matrix/absolute command specified (output below).
> Any help on this would be greatly appreciated. Is stata taking into 
> account the full possibility of all 16 categories?
> (I realise 10 is a small number of subjects, but this particular study 
> is resource limited).

Ronán Conroy
Associate Professor
Division of Population Health Sciences
Royal College of Surgeons in Ireland
Beaux Lane House
Dublin 2

*   For searches and help try:

© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index