Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Kappa weights & category detection
From
Ronan Conroy <[email protected]>
To
"[email protected]" <[email protected]>
Subject
Re: st: Kappa weights & category detection
Date
Fri, 11 Feb 2011 10:43:31 +0000
The main limitation is that of your data. The potential range of psychiatric diagnosis is large, and you can tap very little of this with a small study.
That said, rather than weighting Kappa, which would produce an arbitrary scoring system, I would report my results under
1. Agreement on headline diagnosis
Under this header, only clinically significant disagreements would count. So if you are using DSM-IV, if both raters diagnosed depressive disorder of any sort (any 369 code) the sub-classification would be ignored.
2. Agreement in detail
Precise agreement on diagnosis
Rather than subsuming all disagreement into a global statistic, it would be interesting to see what were the commonest disagreements (you would be surprised by substantial disagreement about obsessive compulsive disorder, but I think we would all expect some variation in mood disorders).
Sometimes an overall statistic can obscure the interesting questions.
r
On 10 Feb 2011, at 20:25, James Kirkbride wrote:
> Hello Stata Users
>
> I am trying to calculate some Kappa values for two raters who have each
> rated the same 10 case notes to produce a mental health diagnosis. Their
> diagnosis can be one of 16 possible categories. Obviously not all 16
> categories are used, so I specify a weight matrix to let stata know
> there are 16 possible categories that could have been chosen.
>
> However I am unsure stata has taken this into account as when I run the
> analysis I get the same kappa statistic with or without the weight
> matrix/absolute command specified (output below).
>
> Any help on this would be greatly appreciated. Is stata taking into
> account the full possibility of all 16 categories?
>
> (I realise 10 is a small number of subjects, but this particular study
> is resource limited).
Ronán Conroy
[email protected]
Associate Professor
Division of Population Health Sciences
Royal College of Surgeons in Ireland
Beaux Lane House
Dublin 2
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/