[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Increasing variance of dependent variable, logit, inter-rater agreement

From   Steven Samuels <>
Subject   Re: st: Increasing variance of dependent variable, logit, inter-rater agreement
Date   Sat, 28 Feb 2009 19:45:56 -0500


All this detail is welcome and clear. I don't really know how to model all of this simultaneously, or, even if there would be any benefit in doing so. I hope that others will read your description and chime in.

Some thoughts: I've read the abstracts of the Feinstein-Cichetti articles, and I think that your original idea of predicting positive agreement from a regression model is good. Be sure to use a flexible model for age. I think that you need a model with more variability than logistic assumes. Consider -hetprob- , which fits a probit model. If you have vehicles that were retested over time, also consider longitudinal data methods (-xt- prefix) If the remote sensing device was not recalibrated between individual observations, you probably also have non-independent errors for observations taken on the same device at the same time. If you used different remote sensing devices to retest the same vehicle of the same age, then you can add random- or fixed- device terms to a predictive model. If you know about environmental conditions that would have affected errors in the remote-sensing, be sure to add those as predictors. With so many observations, you can afford to divide your data, develop your model on one piece, and test on the other.

Best wishes,


On Feb 27, 2009, at 8:28 PM, Supnithadnaporn, Anupit wrote:


Dear Steven,

I appreciate your reply to my post. I am sorry if my explanation is too long.

Thank you,

Please give more detail about what is being assessed. Is there a gold
standard, measured or latent, for what these technologies are trying
to agree upon?

The subject of my study is the in-used vehicles. In some areas of the US, there is a regulation that requires a vehicle to be tested for its emission. In the past, this instrument measured the real tailpipe emission. The test
is typically performed at the commercial inspection station. If the
amount of emission surpasses the threshold standard, the vehicle fails. The owner of failing vehicle has to repair his/her vehicle until it meets the standard level otherwise he/she cannot renew the vehicle registration.

However, this tailpipe-test technology has been replaced by the new one called OBD II test. This test no longer measures the tailpipe emission. Instead, it gives the fail result if there is an error codes relating to the emission
control part of the vehicle.

Despite the different technologies measuring different things, they share the common goal of the regulation. That is to identify the high-polluting vehicles.

* What is the first technology that measures characteristics and
arrives at a pass-fail?  How does it make this decision? Was age one
of these characteristics?

So, the first technology is the OBD II that detects the error codes and yield the pass-fail result which is the *nominal level*. Having certain error codes means that the vehicle is likely to emit high level of pollution beyond the
standards. As the vehicle become older, it is likely to pollute more.
Moreover, the OBD II which is the computer unit of the vehicle is likely to malfunction. If the OBD II is malfunction, it can give either the false-pass
or false-fail result.

* How was the cut point y2b arrived at?

Fortunately, the regulator also has set up several unobtrusive monitoring stations on road. Basically, this technology uses the remote- sensing device (RSD) to measure the real tailpipe emission from numerous vehicles running pass by. This is the second technology in my analysis. It measures the real tailpipe emission which is the *interval level*. And the threshold is based on the EPA regulation set for particular type of vehicle make, model year,
and weight - *the cut point of y2b*.

* You say that the variability of y2a increases with age.  Is the
level of y2a related to age?

Correct. As a vehicle is getting older, its emission level is likely to be high due to deterioration. Moreover, its emission can vary vastly different from one measurement (by RSD) to the other. This is what I am trying to
take into account in my analysis

My data is a pooled-cross section time series of 4 years.
My unit of analysis is a matched pair of a vehicle tested by OBD II and
measured by RSD on road in the same year-testing cycle.
My hypothesis is that the OBD-RSD agreement is greater for the older vehicle
fleets. My sample size ~ 80,000 observations.
Of the total, 72% is classified as 'agree'
For 28% of 'disagree' group, around 90% is the Fail-RSD, Pass-OBD.

During the early analysis, I split the vehicles into different age groups from 3-9 years. I obtain Kappa for each group and compare them. However, I run into problem of Kappa when the prevalence (the disagree cases for
each age-group) is small.

Cicchetti DV, Feinstein AR. High agreement but low Kappa: II Resolving
the paradoxes. J Clin Epidemiol 1990; 43:551-8

*   For searches and help try:

*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index