Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Kappa weights & category detection


From   Ronan Conroy <rconroy@rcsi.ie>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   Re: st: Kappa weights & category detection
Date   Fri, 11 Feb 2011 10:43:31 +0000

The main limitation is that of your data. The potential range of psychiatric diagnosis is large, and you can tap very little of this with a small study. 

That said, rather than weighting Kappa, which would produce an arbitrary scoring system, I would report my results under

1. Agreement on headline diagnosis
Under this header, only clinically significant disagreements would count. So if you are using DSM-IV, if both raters diagnosed depressive disorder of any sort (any 369 code) the sub-classification would be ignored. 

2. Agreement in detail
Precise agreement on diagnosis

Rather than subsuming all disagreement into a global statistic, it would be interesting to see what were the commonest disagreements (you would be surprised by substantial disagreement about obsessive compulsive disorder, but I think we would all expect some variation in mood disorders). 



Sometimes an overall statistic can obscure the interesting questions.

r


On 10 Feb 2011, at 20:25, James Kirkbride wrote:

> Hello Stata Users
> 
> I am trying to calculate some Kappa values for two raters who have each 
> rated the same 10 case notes to produce a mental health diagnosis. Their 
> diagnosis can be one of 16 possible categories. Obviously not all 16 
> categories are used, so I specify a weight matrix to let stata know 
> there are 16 possible categories that could have been chosen.
> 
> However I am unsure stata has taken this into account as when I run the 
> analysis I get the same kappa statistic with or without the weight 
> matrix/absolute command specified (output below).
> 
> Any help on this would be greatly appreciated. Is stata taking into 
> account the full possibility of all 16 categories?
> 
> (I realise 10 is a small number of subjects, but this particular study 
> is resource limited).

Ronán Conroy
rconroy@rcsi.ie
Associate Professor
Division of Population Health Sciences
Royal College of Surgeons in Ireland
Beaux Lane House
Dublin 2


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index