.- help for ^sskdlg^ (STB-58: sxd3) .- Sample size for the kappa-statistic measure of interrater agreement ------------------------------------------------------------------- Description ----------- ^sskdlg^ determines the sample size for the kappa-statistic measure of interrater agreement when there are two unique raters evaluating a binary event. It is geared towards calculating a sample size from a precision oriented perspective, i.e., choosing a sample size so that the standard error of the estimate and the ensuing limits for a 100(1-alpha)% confidence interval do not exceed stipulated values. The program is based on the asymptotic variance presented by Fleiss, Cohen and Everitt (1969) (see also Fleiss, 1981, equations 13.15 - 13.18) and follows the procedure outlined by Cantor (1996). This procedure is based on the quantity Q, which, being equal to the variance of kappa times the sample size, enables the latter to be solved out and calculated. In order to calculate the sample size, five parameters have to be specified. (See below under "Using the dialog box") ^sskdlg^ is interactive since it is implemented through a dialog box. Several outputs are available by checking options in the dialog box. ^sskdlg^ provides graphical displays with interactive X-axis calibrations. Using the dialog box -------------------- 1. Call the dialog box by typing ^sskdlg^ in the command window. 2. Input values for the 5 basic parameters needed to obtain a sample size (or leave defaults). Expected kappa (k) - specifies the estimate of kappa the researcher expects to find. ^sskdlg^ uses the default value of 0 which presumes calculating a standard error when both ratings are independent. Although more realistic values are possible and should be encouraged, this default value, albeit extremely conservative, is suitable for projecting the sample size of a study that, ultimately, would be analysed using the traditional equation as in @kap@. Depending on the specification of other conditions, it can take any value within the permissible bounds, never exceeding -1 to 1. If the specified kappa is incompatible with the selected marginals (proportion of positives expected by each rater) or is outside the plausible range, ^sskdlg^ outputs a warning and the sample size is not calculated. The proportion one expects the 1st observer to rate as positives (p1) (or the 1st observation in the case or a test-retest reproducibility study). ^sskdlg^ uses the default value of 0.1 when the dialog box is opened. The proportion one expects the 2nd observer to rate as positives (p2). As in (p1) for the 2nd rater (or observation). Absolute precision (d) - the minimum envisaged difference between kappa and its lower (or upper) 1-alpha confidence limit. Acceptable values range from > 0 to a limit where, given a set of marginal values (p1 and p2), values of d entailing k - d below the minimum possible value of kappa (kmin) or k + d above the maximum possible value of kappa (kmax) are disallowed. If both conditions are satisfied, ^sskdlg^ returns a message warning that the sample size will not be calculated. If only one of those occurs, the user is warned that there is a partial incompatibility between the stated values of d and k. Yet, the sample size is calculated since d is compatible with at least one of kappa's boundry values. Confidence level (CI) (%) for the confidence interval; default is 95%. 3. Optionals Checking "Show value of Q" reports Q for the given set of parameters. Checking "Show maximum sample size" request the calculation of the largest possible value of Q and the corresponding sample size. This is important when one is not prepared to make any prior assumption concerning kappa. Output also indicates the maximum possible value of kappa, given the preassigned marginals. Checking "Sample size for d" and filling in the desired value requests a unique value of the absolute precision (see item 2d above), given the other selected inputs. This enables the user to work backwards by finding out the precision corresponding to a preset (possible) sample size. 4. Graphical displays Clicking the Graph Q button requests a graphical display of the Q values according to a range of kappa values. Default range is 0 to 1.0 when dialog box is opened. The editting boxes (e.g., indicated by "X:kappas") can be used to change the range of the x-axis. Change values for zooming in or out. Note that negative values are allowed although this should be unusual in the context of reliability studies. The position of the maximum possible value of kappa reported in 3b can also be visualized in the graph by checking "k_max". Also note that when the specified parameters (k, p1, p2, d and CI) preclude the calculation of the sample size, Graph Q will not be (re)displayed. Reset the values in order to enable the graph. Clicking the Graph S button requests a graphical display analogous to Graph Q but plots sample size instead. Use the same editting boxes as for 4a to control the x-axis (zooming). "k_max" may also be checked. The same restrictions as in Graph Q apply here too. Clicking the Graph P button requests a graphical display of sample sizes according to the proportion of positives measured by raters 1 and 2 when both are expected to find the same value and given the specified values k, d and CI. X-axis default range is 0 to 1.0. Nevertheless, Graph P will only show a range compatible with plausible sample sizes, since some combinations of specified parameters are impossible. For zooming in or out, edit boxes indicated by "X:prop." can be used. Clicking the Graph D button requests a graphical display of values of absolute precision for a range of pre-specified sample sizes. This display is an extention of the option "Sample size for d". Sample size range can be controlled using the edit boxes by "X:ssize". Checking "Keep variables" retains in memory all variables used for displaying Graphs Q, S, P and D when leaving the dialog box. This enables the user to redraw new graphs graphs at his/her own discretion. Note that values kept in memory are those specified (on display) at the time of exit. This option requires at least running the program once or running a viable configuration of parameters after an improper one precluded a calculation. This is because the underlying data is cleared in this situation in order to avoid a mismatch between the parameters on screen (dialog box) and the data in memory generated by a former viable run. Saved values ------------ The following items are saved in the global ^S_^# macros. ^S_1^ sample size calculated ^S_2^ sample size at maximum Q ^S_3^ value of Q, where Q = variance(kappa) * N ^S_4^ maximum value of Q ^S_5^ specified value of minimum kappa worth finding ^S_6^ maximum value of kappa, given specified marginal probalities ^S_7^ minimum value of kappa, given specified marginal probalities ^S_8^ specified marginal probability of a positive event calculated by Rater 1 ^S_9^ specified marginal probability of a positive event calculated by Rater 2 ^S_10^ specified absolute precision ^S_11^ z value for confidence intervals Author ------ Michael E. Reichenheim, IMS/UERJ, Brazil, michael@@ims.uerj.br Also see -------- STB: STB-58: sxd3 On-line: help for @kap@, @kappa@