Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Kappa weights & category detection

From   James Kirkbride <>
Subject   Re: st: Kappa weights & category detection
Date   Fri, 11 Feb 2011 10:37:32 -0500

Dear Ronan

Many thanks for your feedback. I agree with you entirely and we are in fact using collapsed categories of diagnostic groups which make clinical sense. This is part of a training program to ensure our psychiatrists (rating case notes using opcrit criteria) are doing roughly the same thing.

Thanks very much for your salient advice on how to proceed.

Part of my concern with kappa in stata is whether it acknowledges all the categories (including those not rated) -- it was unclear to me whether this was taken into account.

Thanks again


On 11/02/2011 5:43 AM, Ronan Conroy wrote:
The main limitation is that of your data. The potential range of psychiatric diagnosis is large, and you can tap very little of this with a small study.

That said, rather than weighting Kappa, which would produce an arbitrary scoring system, I would report my results under

1. Agreement on headline diagnosis
Under this header, only clinically significant disagreements would count. So if you are using DSM-IV, if both raters diagnosed depressive disorder of any sort (any 369 code) the sub-classification would be ignored.

2. Agreement in detail
Precise agreement on diagnosis

Rather than subsuming all disagreement into a global statistic, it would be interesting to see what were the commonest disagreements (you would be surprised by substantial disagreement about obsessive compulsive disorder, but I think we would all expect some variation in mood disorders).

Sometimes an overall statistic can obscure the interesting questions.


On 10 Feb 2011, at 20:25, James Kirkbride wrote:

Hello Stata Users

I am trying to calculate some Kappa values for two raters who have each
rated the same 10 case notes to produce a mental health diagnosis. Their
diagnosis can be one of 16 possible categories. Obviously not all 16
categories are used, so I specify a weight matrix to let stata know
there are 16 possible categories that could have been chosen.

However I am unsure stata has taken this into account as when I run the
analysis I get the same kappa statistic with or without the weight
matrix/absolute command specified (output below).

Any help on this would be greatly appreciated. Is stata taking into
account the full possibility of all 16 categories?

(I realise 10 is a small number of subjects, but this particular study
is resource limited).
Ronán Conroy
Associate Professor
Division of Population Health Sciences
Royal College of Surgeons in Ireland
Beaux Lane House
Dublin 2

*   For searches and help try:


James Kirkbride Ph.D.

Sir Henry Wellcome Research Fellow
Department of Psychiatry
University of Cambridge
01223 336 798

*   For searches and help try:

© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index