# st: Does my statistic for "net proportion of subjects with improved prediction" already exist?

 From "Daniel Waxman" <[email protected]> To <[email protected]> Subject st: Does my statistic for "net proportion of subjects with improved prediction" already exist? Date Thu, 27 Sep 2007 12:31:22 -0400

```Statalist,

I am studying the effect of adding a biomarker to an existing model and want
to describe the effect of that model vis-�-vis the number of subjects with
improved predictions in the �new model� vs. the �old model�.  While there is
an extensive literature on this topic, most of it divides the outcome into
risk categories (i.e. predicted risk of 0-5%, 5-10%, etc.), something that I
am not so interested in doing.

An intuitive way to look at this would be to look at the net number of
subjects who are assigned a higher predicted probability with the new model
among those with the outcome in question, plus the net number assigned a
lower probability among those who did not have the outcome.  The ratio of
this number to the total # of subjects would then be the proportion of
patients with improved predictions (and would range from zero to 1).  See
example below.

My question:  Did I just reinvent the wheel?  (e.g. is this equivalent to
some existing statistic?)  Does anybody see any logical problem with looking
at this as one measure of the effect of adding a predictor to an existing
model?

Thanks,
Daniel Waxman

**** example: (where zlog is continuous, zero is dichotomous, new_marker is
the dichotomous new marker, and there is no missing data) ***

. logistic outcome zlog zero
. predict p_old

. logistic outcome zlog zero new_marker
. predict p_new

. count if e(sample)
. gen N=r(N)

. egen number_up_outcome=total(p_new>p_old & outcome)
. egen number_down_outcome=total(p_new<p_old & outcome)

. egen number_up_no_outcome=total(p_new>p_old & !outcome)
. egen number_down_no_outcome=total(p_new<p_old & !outcome)

. gen net_proportion_improved=
((number_up_outcome-number_down_outcome)+(number_down_no_outcome-number_up_n
o_outcome))/N

No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.488 / Virus Database: 269.13.32/1033 - Release Date: 9/27/2007
11:06 AM

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```

• Follow-Ups: