Re: st: RE: Poisson Regression

[brendan.halpin@ul.ie (Brendan Halpin)]

On Mon, Feb 14 2011, Visintainer, Paul wrote:

> My frustration is that when the outcome is common and logistic
> regression is used, there virtually no discussion of clinical relevance
> -- only statistical significance, (e.g., is a significant odds ratio of
> 2.5 clinically relevant? Perhaps if the base risk is 2%; perhaps not if
> the base risk 73%. 

Your underlying point about substantive significance is well taken, but
this is a bad example. Here is a simulation (code below):

Case 1:
           |        outcome
     class |        No        Yes |     Total
-----------+----------------------+----------
  Controls |       980         20 |     1,000 
 Treatment |       951         49 |     1,000 
-----------+----------------------+----------
     Total |     1,931         69 |     2,000 

Case 1:
RR  2.450
OR  2.525
N extra outcomes   29

Case 2:
           |        outcome
     class |        No        Yes |     Total
-----------+----------------------+----------
  Controls |       270        730 |     1,000 
 Treatment |       129        871 |     1,000 
-----------+----------------------+----------
     Total |       399      1,601 |     2,000 

RR  1.193
OR  2.497
N extra outcomes  141


In both cases the OR is 2.5, and the rate for controls is respectively
2% and 73%. The RR is much lower with the 73% base rate. However, the
"clinical" significance is *higher* with the 73% base rate, with 14.1%
excess "outcomes" in the treatment group compared with 2.9% when the
base rate is 2%.

In other words, the relative rate seems a poorer, not a better estimate
of clinical significance than the odds ratio. (In fact, a probit model
looks even better with a 2% effect of 0.40 and a 73% effect of
0.52.) 

Brendan

--8<-----
clear
input class outcome n
0 0 980
0 1  20
1 0 951
1 1  49
end

label define class 0 "Controls" 1 "Treatment"
label define outcome 0 "No" 1 "Yes"
label values class class
label values outcome outcome

noi tab class outcome [freq=n], matcell(t)
scalar relrate = (t[2,2]/(t[2,1]+t[2,2]))/(t[1,2]/(t[1,1]+t[1,2]))
scalar OR      = (t[2,2]/(t[2,1]       ))/(t[1,2]/(t[1,1]       ))
scalar D       = t[2,2] - t[1,2]

noi di "Case 1:" _newline "RR " %6.3f relrate _newline "OR " %6.3f OR _newline "N extra outcomes" %5.0f D
expand n
noi probit outcome class

clear
input class outcome n
0 0 270
0 1 730
1 0 129
1 1 871
end

label define class 0 "Controls" 1 "Treatment"
label define outcome 0 "No" 1 "Yes"
label values class class
label values outcome outcome

noi tab class outcome [freq=n], matcell(t)
scalar relrate = (t[2,2]/(t[2,1]+t[2,2]))/(t[1,2]/(t[1,1]+t[1,2]))
scalar OR      = (t[2,2]/(t[2,1]       ))/(t[1,2]/(t[1,1]       ))
scalar D       = t[2,2] - t[1,2]

noi di "Case 2:" _newline "RR " %6.3f relrate _newline "OR " %6.3f OR _newline "N extra outcomes" %5.0f D
expand n
noi probit outcome class

--8<-----
-- 
Brendan Halpin,  Department of Sociology,  University of Limerick,  Ireland
Tel: w +353-61-213147 f +353-61-202569 h +353-61-338562; Room F1-009 x 3147
mailto:brendan.halpin@ul.ie  http://www.ul.ie/sociology/brendan.halpin.html
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/