Ronan Conroy and others,
The drug test has only one rating (True Positives). Due to the large majority of the sample being (True Negatives) the Kappa will be biased (Bishop et al., 1975; Magura et al, 1987; Magura et al, 1996; Sherman et al., 1992; Patton, et al., 2005).
"A more appropriate indicator of agreement between self-reports and urinalysis for most of the studies reviewed is a conditional kappa coefficient...The self-reports of subjects with negative urinalyses are ignored in the computation of conditional Kappa." (Magura et al., 1987 p. 734).
It appears to me that there is no direct way to compute conditional kappa; however, I should be able to weight kappa so that those who test positive are given all the weight of the sample.
Theron
Are you telling us that you have only three participants? Then the
advice is to get more.
Or are you telling us that the drug test has only one rating
(positive) because it is correct? In this case, your whole sample is
made up of true positives. In a case like this, kappa is
inappropriate. The level of agreement reduces to the proportion of
self-reports (rating 2) which are positive.
Ronan Conroy
<<winmail.dat>>
