Re: st: New to Stata; wish to calculate sample size for kappa

[Steven Samuels <sjsamuels@gmail.com>]

George-
In addition, to -sskdlg- is the later -sskapp-, downloadable from SSC ("ssc download sskkapp")
but it also applies to the binary outcome with two raters.

I don't think that kappa exists for your situation  All the examples that I've seen require multiple  scenarios for the raters to assess. For a single video, you can only describe the proportions I mentioned and functions of those proportions; kappa doesn't appear to be such a function. 

Certainly statistics other than kappa that can measure agreement. A distribution in which one of the proportions is close to 1 and the others are small indicates excellent agreement.  One measure which exploits this is: 

 r  = 1 - (entropy of observed distribution of p's)/(maximum possible entropy).  where the denominator is the entropy corresponding to p1 = 1/3; p2 = 1/3; p3 = 1/3. See Wikipedia articles on "entropy".  This measure doesn't take the ordinal nature of your scale into account. For example, it would have the same value for (1/2, 1/2, 0) and (1/2, 0, 1/2), when the former clearly shows better agreement. I have no experience or references in the use of entropy-based statistics for rater agreement, so you have some research to do.  

Good luck!

Steve
sjsamuels@gmail.com



On May 2, 2011, at 8:59 PM, volsmd99 wrote:

My goal is to calculate sample size for my scenario. I should mention that in addition to being new to stata, i am also new to kappa.

For my case, I had the raters look at a video for which they could classify their opinion into three categories. Their choices were 0, 1, or 2. 

My Ko value is 0.80 (the level at which I considered Kappa to be significant)  and my KL is 0.60 (minimum threshold value). Supposing a 95% confidence interval, I wish to calculate sample size. 

I have come across the package sskdlg but it seems to only allow sample size calculation with a binary outcome and with only two raters.

thanks 


On May 2, 2011, at 3:12 PM, Steven Samuels wrote:

> Welcome to Stata and Statalist!
> 
> I don't really understand your question. If you have a number of raters and one subject,  you can estimate the proportion of raters who assign each category to the subject. Call them p1, p2, p3, with p1 + p2 + p3 =1.  How is kappa to be defined?
> 
> Steve
> sjsamuels@gmail.com
> 
> 
> 
> On May 1, 2011, at 7:24 PM, volsmd99 wrote:
> 
> Hi,
> 
> I was previously a user of SAS and have recently switch to stata.
> 
> My situation involves multiple raters on a 3 point ordinal scale. I have one subject. I am unsure which or if there is a package that will allow me to do this.
> 
> thanks, George
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/