Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Nick Cox" <n.j.cox@durham.ac.uk> |

To |
<statalist@hsphsun2.harvard.edu> |

Subject |
RE: st: AW: Non-reported s.e. for kappa |

Date |
Wed, 14 Apr 2010 18:39:17 +0100 |

On a quite different note, I am curious. How much practical use is a single kappa measure with multiple raters? If this were my problem I would be much more interested in looking at the fine structure of disagreement, e.g. whether raters fall into distinct groups, using some sort of multivariate method. Imagine instead of a correlation matrix a single super-correlation.... It wouldn't be much use, so far as I can see. Nick n.j.cox@durham.ac.uk Steve Samuels You are welcome, Karen. I'm flattered to be mistaken for Martin. With n = 26, you might also try -bootstrap-. On Wed, Apr 14, 2010 at 12:20 PM, Karen Wright <Karen.Wright@icr.ac.uk> wrote: > Deepest apologies, thanks Steve for the jackknife suggestion. > Although thanks also to Martin for your input! >>>> Steve Samuels <sjsamuels@gmail.com> 14/04/2010 17:18 >>> > Correction, to get the pseudo-values _jk_1: > > jackknife r(kappa), mse keep: kap id* > ci _jk_1 // First order unbiased estimates > > > Steve > On Wed, Apr 14, 2010 at 12:03 PM, Steve Samuels <sjsamuels@gmail.com> > wrote: >> Martin- >> For Karen's set up (idj contains the subject rating for rater j), >> there is no official example that I can see. p615 shows counts of >> ratings, with each variable being a rating category. Also, for > p615,, >> the command should be "kappa", not "kap", because the data are >> counts of assignments to three categories by five raters. This was >> terrible syntax decision: depending on the use of "kap" or "kappa" > to >> distinguish between two data setups. >> >> One suggestion for a standard error: >> ************************************************* >> version 10.0 >> clear >> input case id2 id3 id6 id8 id9 id10 >> 1 3 3 3 3 3 3 >> 2 1 1 1 1 1 1 >> 3 2 2 1 3 1 1 >> 4 3 3 3 3 3 3 >> 5 1 1 1 1 1 1 >> end >> kap id* >> jackknife r(kappa), mse: kap id* >> ci _jk_1 // First order unbiased estimates >> ************************************* >> >> n = 5 subjects doesn't seem like enough to draw many conclusions. >> It's interesting that the raters disagreed for only one subject. >> >> Steve >> >> Steve >> >> On Wed, Apr 14, 2010 at 10:52 AM, Martin Weiss <martin.weiss1@gmx.de> > wrote: >>> >>> <> >>> >>> You do not need an imaginary dataset, as the official example does > not >>> return the se, either: >>> >>> >>> ************* >>> webuse p615, clear >>> kap cat* >>> ret li >>> ************* >>> >>> The help file could make this fact clear, I think. BTW, a command > reporting >>> a whole table inevitably runs into the problem which statistics it > should >>> return. Just see what -summ- with a -varlist- hands back: The > statistics for >>> the last variable in that -varlist-. The alternative would be to > return a >>> -matrix-, of course... >>> >>> >>> HTH >>> Martin >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: owner-statalist@hsphsun2.harvard.edu >>> [mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Karen > Wright >>> Gesendet: Mittwoch, 14. April 2010 16:38 >>> An: statalist@hsphsun2.harvard.edu >>> Betreff: st: Non-reported s.e. for kappa >>> >>> Dear All, >>> I have been working with kappa statistics and I have come across a >>> peculiarity that I don't fully understand. Searching the help files > and the >>> statalist archive did not shed any light on the matter > unfortunately. >>> After running a simple composite kappa on multiple readers (3 > categories) I >>> then type -di r(se)- and obtain nothing! The results have not been > cleared >>> from memory as typing -di r(kappa)- displays the kappa value. Does > anyone >>> know why this is? >>> I am aware that test statistics cannot be calculated when the number > of >>> ratings per subject vary. Is there something similar happening > here?! >>> >>> I have shown below a snippet of the data and the commands used. > Snippet of >>> data contains the diagnosis (=1,2, or 3) for 6 readers (id2-id10) on > 5 >>> cases. [Done using Stata10 on Windows.] >>> >>> Thank you for taking the time to read this, >>> Karen >>> >>> ************************************************* >>> version 10.0 >>> >>> input case id2 id3 id6 id8 id9 id10 >>> 1 3 3 3 3 3 3 >>> 2 1 1 1 1 1 1 >>> 3 2 2 1 3 1 1 >>> 4 3 3 3 3 3 3 >>> 5 1 1 1 1 1 1 >>> end >>> >>> kap id* >>> di r(se) >>> di r(kappa) >>> >>> ************************************************* >>> >>> >>> >>> The Institute of Cancer Research: Royal Cancer Hospital, a > charitable >>> Company Limited by Guarantee, Registered in England under Company > No. 534147 >>> with its Registered Office at 123 Old Brompton Road, London SW7 > 3RP. >>> >>> This e-mail message is confidential and for use by the addressee > only. If >>> the message is received by anyone other than the addressee, please > return >>> the message to the sender by replying to it and then delete the > message from >>> your computer and network. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**RE: st: AW: Non-reported s.e. for kappa***From:*"Karen Wright" <Karen.Wright@icr.ac.uk>

**References**:**st: Non-reported s.e. for kappa***From:*"Karen Wright" <Karen.Wright@icr.ac.uk>

**Re: st: AW: Non-reported s.e. for kappa***From:*Steve Samuels <sjsamuels@gmail.com>

**Re: st: AW: Non-reported s.e. for kappa***From:*Steve Samuels <sjsamuels@gmail.com>

**Re: st: AW: Non-reported s.e. for kappa***From:*"Karen Wright" <Karen.Wright@icr.ac.uk>

**Re: st: AW: Non-reported s.e. for kappa***From:*Steve Samuels <sjsamuels@gmail.com>

- Prev by Date:
**Re: st: AW: Non-reported s.e. for kappa** - Next by Date:
**Re: st: how to do fixed effects with weights? (problem using svy: with xtreg, fe)** - Previous by thread:
**Re: st: AW: Non-reported s.e. for kappa** - Next by thread:
**RE: st: AW: Non-reported s.e. for kappa** - Index(es):