Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

# Re: st: Comparing risk scores

 From Nick Cox To statalist@hsphsun2.harvard.edu Subject Re: st: Comparing risk scores Date Tue, 18 Oct 2011 13:50:00 +0100

```I'd change the clinicians.

Nick

On Tue, Oct 18, 2011 at 1:35 PM, K Jensen <k.x.jensen@gmail.com> wrote:
> The clinicians I am working with are ADAMANT that they want a simple
> scale based on ticking boxes and adding the number of ticks, and
> nothing more complicated...
>
> If we work within these constraints, how is it best to compare the
> possible scores?
>
> Thanks
>
> Karin
>
> On 18 October 2011 13:22, Richard Goldstein <richgold@ix.netcom.com> wrote:
>> Karin,
>>
>> I suggest you might want to read Sullivan, LM, et al. (2004),
>> "Presentation of multivariate data for clinical use: the Framingham
>> study risk score functions," _Statistics in Medicine_, 23: 1631-1660,
>> which describes how the Framingham people came up with their risk scores
>>
>> Rich
>>
>> On 10/18/11 8:18 AM, K Jensen wrote:
>>> Hi Nick
>>>
>>> Thanks for your reply.  It's actually a bit more complicated than
>>> that.  We are trying to construct a "best" single score that would be
>>> simple and used clinically.  The elements that are summed to make the
>>> score (0,1,2,3 etc) are derived from various clinical measurements.
>>> They are dichotomised by choosing the cutpoint that maximises the sum
>>> of sensitivity+specificity.  Only those binary variables significant
>>> in a univariate logistic regression are proposed for the model.
>>>
>>> I am wanting to choose the "best" model, that is useful for
>>> clinicians.  If we had 7 binary variables, say, I would look at all
>>> possibilities of choosing different combinations of the sums of them.
>>> E.g. 1, 2, 3, 4, 5, 6, 7,1+2,1+3,1+4,1+5,1+6,1+7, 2+3, 2+4,... up to
>>> 1+2+3+4+5+6+7.  I would like to use the optimal score based on this
>>> method, but don't know how to measure optimality.
>>>
>>> Best wishes,
>>>
>>> Karin
>>>
>>> On 18 October 2011 12:36, Nick Cox <njcoxstata@gmail.com> wrote:
>>>> I would recast this as a -logit- or -logistic- problem in which your
>>>> outcome is dead or alive. Depending on how you think about your
>>>> scores, they define predictors to be treated as they come or
>>>> predictors to be treated as a set of indicator variables (or in some
>>>> cases both).
>>>>
>>>>  I don't think you are restricted to using one score or the other as predictor.
>>>>
>>>> Nick
>>>>
>>>> On Tue, Oct 18, 2011 at 12:11 PM, K Jensen <k.x.jensen@gmail.com> wrote:
>>>>> Maybe this is more of a stats question than a Stata one, but there are
>>>>> such a lot of good brains here...
>>>>>
>>>>> We are constructing point scores to indicate severity of risk  Death
>>>>> is the outcome. What is the best way of measuring the usefulness of
>>>>> the score?  The aim is to show a good gradient of risk.  Say the
>>>>> results for two different scores were:
>>>>>
>>>>> 0        12    136      9.9%      145
>>>>> 1        18    126     15.4%      144
>>>>> 2        18     62     26.2%       81
>>>>> 3        10      9     57.1%       20
>>>>> 4         2      0    100  %        3
>>>>> -------------------------------------
>>>>> Total:   60    333                393
>>>>>
>>>>> 0         8    174      4.6%      182
>>>>> 1        21    143     12.8%      164
>>>>> 2        22     19     53.7%       41
>>>>> 3         5      1     83.3%        6
>>>>> -------------------------------------
>>>>> TOTAL:   60    333                393
>>>>>
>>>>> Which is the better score?  What is the best way to measure its
>>>>> predictive power?  I understand that ROC type analysis doesn't really
>>>>> apply here.  Some measure of R-squared?  AIC?
>>>>>
>>>>> Thankyou
>>>>>
>>>>> Karin
>>>>>
>>>>> PS) I have made up the data, so the numbers don't quite add up.  It is
>>>>> meant to be two different, competing scores on the same people.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```