Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

re: st: Checking reliability of a measurement device


From   David Airey <david.airey@vanderbilt.edu>
To   statalist@hsphsun2.harvard.edu
Subject   re: st: Checking reliability of a measurement device
Date   Fri, 26 Nov 2004 09:55:56 -0600

Hi Statalisters,

a Dentistry PhD student did some measurements on 12
teeth with varying conditions and he asked me how
could he show that the device used for the
measurements is reliable. More specifically each one
of the 12 teeth has been measured by this device by 2
raters (a and b) X 2 time points (week 1 and week 2) X
6 relative positions =   24 measurements. The goal is
to show that the discrepancies among measurements are
not statistically significant.
That is interesting.

I'm not sure whether the student is interested in reliability (variance component estimation?) or the student wishes to compare performance of a new method to an accepted method (mean comparisons with certain assumptions about the variance between methods?).

If the latter, maybe you want a planned one-sided comparison against a "gold standard". You are only interested in whether your new method mean is higher (or lower) than an accepted method's mean performance. You might look at Hsu (1996) Multiple Comparisons: Theory and methods (Chapman and Hall).

If the former, don't your hypotheses center around the rater means and or variances? There is a lot of literature on rater reliability. Seems like the design has a lot of information within two raters, but very little between rater information from the population of possible raters.

In your model, level of variation to think about is the person. Is each tooth from a separate individual? Teeth are correlated with person if not.



My first thought was to use a 2-level model
(measurements nested within tooth) and test the rater,
time, position effects and maybe their interractions.
Something like

xi:xtreg length i.rater i.time i.position,i(tooth)

assuming that a random intercept structure is adequate
for this experiment.
The problem is that this is the first time I deal with
a problem where the goal is to "prove" that some
factors have no significant effects, thus I am not
very confident with the aforementioned method.

Any thoughts?

Nikos Pantazis

Biostatistician
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index