Re: st: Comparing risk scores

[K Jensen <k.x.jensen@gmail.com>]

The clinicians I am working with are ADAMANT that they want a simple
scale based on ticking boxes and adding the number of ticks, and
nothing more complicated...

If we work within these constraints, how is it best to compare the
possible scores?

Thanks

Karin

On 18 October 2011 13:22, Richard Goldstein <richgold@ix.netcom.com> wrote:
> Karin,
>
> I suggest you might want to read Sullivan, LM, et al. (2004),
> "Presentation of multivariate data for clinical use: the Framingham
> study risk score functions," _Statistics in Medicine_, 23: 1631-1660,
> which describes how the Framingham people came up with their risk scores
>
> Rich
>
> On 10/18/11 8:18 AM, K Jensen wrote:
>> Hi Nick
>>
>> Thanks for your reply.  It's actually a bit more complicated than
>> that.  We are trying to construct a "best" single score that would be
>> simple and used clinically.  The elements that are summed to make the
>> score (0,1,2,3 etc) are derived from various clinical measurements.
>> They are dichotomised by choosing the cutpoint that maximises the sum
>> of sensitivity+specificity.  Only those binary variables significant
>> in a univariate logistic regression are proposed for the model.
>>
>> I am wanting to choose the "best" model, that is useful for
>> clinicians.  If we had 7 binary variables, say, I would look at all
>> possibilities of choosing different combinations of the sums of them.
>> E.g. 1, 2, 3, 4, 5, 6, 7,1+2,1+3,1+4,1+5,1+6,1+7, 2+3, 2+4,... up to
>> 1+2+3+4+5+6+7.  I would like to use the optimal score based on this
>> method, but don't know how to measure optimality.
>>
>> Best wishes,
>>
>> Karin
>>
>> On 18 October 2011 12:36, Nick Cox <njcoxstata@gmail.com> wrote:
>>> I would recast this as a -logit- or -logistic- problem in which your
>>> outcome is dead or alive. Depending on how you think about your
>>> scores, they define predictors to be treated as they come or
>>> predictors to be treated as a set of indicator variables (or in some
>>> cases both).
>>>
>>>  I don't think you are restricted to using one score or the other as predictor.
>>>
>>> Nick
>>>
>>> On Tue, Oct 18, 2011 at 12:11 PM, K Jensen <k.x.jensen@gmail.com> wrote:
>>>> Maybe this is more of a stats question than a Stata one, but there are
>>>> such a lot of good brains here...
>>>>
>>>> We are constructing point scores to indicate severity of risk  Death
>>>> is the outcome. What is the best way of measuring the usefulness of
>>>> the score?  The aim is to show a good gradient of risk.  Say the
>>>> results for two different scores were:
>>>>
>>>> Score  Dead  Alive    %dead    Totals
>>>> 0        12    136      9.9%      145
>>>> 1        18    126     15.4%      144
>>>> 2        18     62     26.2%       81
>>>> 3        10      9     57.1%       20
>>>> 4         2      0    100  %        3
>>>> -------------------------------------
>>>> Total:   60    333                393
>>>>
>>>> Score  Dead  Alive    %dead    Totals
>>>> 0         8    174      4.6%      182
>>>> 1        21    143     12.8%      164
>>>> 2        22     19     53.7%       41
>>>> 3         5      1     83.3%        6
>>>> -------------------------------------
>>>> TOTAL:   60    333                393
>>>>
>>>> Which is the better score?  What is the best way to measure its
>>>> predictive power?  I understand that ROC type analysis doesn't really
>>>> apply here.  Some measure of R-squared?  AIC?
>>>>
>>>> Thankyou
>>>>
>>>> Karin
>>>>
>>>> PS) I have made up the data, so the numbers don't quite add up.  It is
>>>> meant to be two different, competing scores on the same people.
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/