Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

Re: st: New to Stata; wish to calculate sample size for kappa

 From Steven Samuels To statalist@hsphsun2.harvard.edu Subject Re: st: New to Stata; wish to calculate sample size for kappa Date Mon, 2 May 2011 23:01:40 -0400

```George-
but it also applies to the binary outcome with two raters.

I don't think that kappa exists for your situation  All the examples that I've seen require multiple  scenarios for the raters to assess. For a single video, you can only describe the proportions I mentioned and functions of those proportions; kappa doesn't appear to be such a function.

Certainly statistics other than kappa that can measure agreement. A distribution in which one of the proportions is close to 1 and the others are small indicates excellent agreement.  One measure which exploits this is:

r  = 1 - (entropy of observed distribution of p's)/(maximum possible entropy).  where the denominator is the entropy corresponding to p1 = 1/3; p2 = 1/3; p3 = 1/3. See Wikipedia articles on "entropy".  This measure doesn't take the ordinal nature of your scale into account. For example, it would have the same value for (1/2, 1/2, 0) and (1/2, 0, 1/2), when the former clearly shows better agreement. I have no experience or references in the use of entropy-based statistics for rater agreement, so you have some research to do.

Good luck!

Steve
sjsamuels@gmail.com

On May 2, 2011, at 8:59 PM, volsmd99 wrote:

My goal is to calculate sample size for my scenario. I should mention that in addition to being new to stata, i am also new to kappa.

For my case, I had the raters look at a video for which they could classify their opinion into three categories. Their choices were 0, 1, or 2.

My Ko value is 0.80 (the level at which I considered Kappa to be significant)  and my KL is 0.60 (minimum threshold value). Supposing a 95% confidence interval, I wish to calculate sample size.

I have come across the package sskdlg but it seems to only allow sample size calculation with a binary outcome and with only two raters.

thanks

On May 2, 2011, at 3:12 PM, Steven Samuels wrote:

> Welcome to Stata and Statalist!
>
> I don't really understand your question. If you have a number of raters and one subject,  you can estimate the proportion of raters who assign each category to the subject. Call them p1, p2, p3, with p1 + p2 + p3 =1.  How is kappa to be defined?
>
> Steve
> sjsamuels@gmail.com
>
>
>
> On May 1, 2011, at 7:24 PM, volsmd99 wrote:
>
> Hi,
>
> I was previously a user of SAS and have recently switch to stata.
>
> My situation involves multiple raters on a 3 point ordinal scale. I have one subject. I am unsure which or if there is a package that will allow me to do this.
>
> thanks, George
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```