Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Comparing risk scores


From   K Jensen <k.x.jensen@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Comparing risk scores
Date   Tue, 18 Oct 2011 14:20:37 +0100

Thanks, Jesper.

This is sort of binary data, in that it is in-hospital mortality and
the length of stay doesn't vary that much by patient and is relatively
short.  We are treating it as binary.  Any tips on binary data?

Thankyou

Karin

On 18 October 2011 13:52, Jesper Lindhardsen <JESLIN01@geh.regionh.dk> wrote:
> Hi there,
>
> A popular, although perhaps rough, method is to use logist models and determine the best models by AUC. However in my view , this seem to impose restrictions on the data you analyse regarding censoring and equal follow-up. Since (I gather) your data you use to make the risk scores is survival data, perhaps the approach by Newson could be applied (Newson RB. Comparing the predictive power of survival models using Harrell ' s c or Somers ' D. Stata Journal - http://www.imperial.ac.uk/nhli/r.newson/papers/predsurv.pdf), where comparison of Harrells C of Cox regression is used to compare the prediction of the scheme.
>
> Incidentially, I guess generalisability is always an issue and the risk scores should be tried out in other cohorts than the one you use to develop the model.
>
> HTH
>
> Jesper
>
> Jesper Lindhardsen
> MD, PhD candidate
> Department of Cardiovascular Research
> Copenhagen University Hospital, Gentofte
> Denmark
>
>
>
>
>
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of K Jensen
> Sent: 18 October 2011 14:36
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Comparing risk scores
>
> The clinicians I am working with are ADAMANT that they want a simple
> scale based on ticking boxes and adding the number of ticks, and
> nothing more complicated...
>
> If we work within these constraints, how is it best to compare the
> possible scores?
>
> Thanks
>
> Karin
>
> On 18 October 2011 13:22, Richard Goldstein <richgold@ix.netcom.com> wrote:
>> Karin,
>>
>> I suggest you might want to read Sullivan, LM, et al. (2004),
>> "Presentation of multivariate data for clinical use: the Framingham
>> study risk score functions," _Statistics in Medicine_, 23: 1631-1660,
>> which describes how the Framingham people came up with their risk scores
>>
>> Rich
>>
>> On 10/18/11 8:18 AM, K Jensen wrote:
>>> Hi Nick
>>>
>>> Thanks for your reply.  It's actually a bit more complicated than
>>> that.  We are trying to construct a "best" single score that would be
>>> simple and used clinically.  The elements that are summed to make the
>>> score (0,1,2,3 etc) are derived from various clinical measurements.
>>> They are dichotomised by choosing the cutpoint that maximises the sum
>>> of sensitivity+specificity.  Only those binary variables significant
>>> in a univariate logistic regression are proposed for the model.
>>>
>>> I am wanting to choose the "best" model, that is useful for
>>> clinicians.  If we had 7 binary variables, say, I would look at all
>>> possibilities of choosing different combinations of the sums of them.
>>> E.g. 1, 2, 3, 4, 5, 6, 7,1+2,1+3,1+4,1+5,1+6,1+7, 2+3, 2+4,... up to
>>> 1+2+3+4+5+6+7.  I would like to use the optimal score based on this
>>> method, but don't know how to measure optimality.
>>>
>>> Best wishes,
>>>
>>> Karin
>>>
>>> On 18 October 2011 12:36, Nick Cox <njcoxstata@gmail.com> wrote:
>>>> I would recast this as a -logit- or -logistic- problem in which your
>>>> outcome is dead or alive. Depending on how you think about your
>>>> scores, they define predictors to be treated as they come or
>>>> predictors to be treated as a set of indicator variables (or in some
>>>> cases both).
>>>>
>>>>  I don't think you are restricted to using one score or the other as predictor.
>>>>
>>>> Nick
>>>>
>>>> On Tue, Oct 18, 2011 at 12:11 PM, K Jensen <k.x.jensen@gmail.com> wrote:
>>>>> Maybe this is more of a stats question than a Stata one, but there are
>>>>> such a lot of good brains here...
>>>>>
>>>>> We are constructing point scores to indicate severity of risk  Death
>>>>> is the outcome. What is the best way of measuring the usefulness of
>>>>> the score?  The aim is to show a good gradient of risk.  Say the
>>>>> results for two different scores were:
>>>>>
>>>>> Score  Dead  Alive    %dead    Totals
>>>>> 0        12    136      9.9%      145
>>>>> 1        18    126     15.4%      144
>>>>> 2        18     62     26.2%       81
>>>>> 3        10      9     57.1%       20
>>>>> 4         2      0    100  %        3
>>>>> -------------------------------------
>>>>> Total:   60    333                393
>>>>>
>>>>> Score  Dead  Alive    %dead    Totals
>>>>> 0         8    174      4.6%      182
>>>>> 1        21    143     12.8%      164
>>>>> 2        22     19     53.7%       41
>>>>> 3         5      1     83.3%        6
>>>>> -------------------------------------
>>>>> TOTAL:   60    333                393
>>>>>
>>>>> Which is the better score?  What is the best way to measure its
>>>>> predictive power?  I understand that ROC type analysis doesn't really
>>>>> apply here.  Some measure of R-squared?  AIC?
>>>>>
>>>>> Thankyou
>>>>>
>>>>> Karin
>>>>>
>>>>> PS) I have made up the data, so the numbers don't quite add up.  It is
>>>>> meant to be two different, competing scores on the same people.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index