Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | Nick Cox <njcoxstata@gmail.com> |
| To | statalist@hsphsun2.harvard.edu |
| Subject | Re: st: Comparing risk scores |
| Date | Tue, 18 Oct 2011 13:50:00 +0100 |
I'd change the clinicians. Nick On Tue, Oct 18, 2011 at 1:35 PM, K Jensen <k.x.jensen@gmail.com> wrote: > The clinicians I am working with are ADAMANT that they want a simple > scale based on ticking boxes and adding the number of ticks, and > nothing more complicated... > > If we work within these constraints, how is it best to compare the > possible scores? > > Thanks > > Karin > > On 18 October 2011 13:22, Richard Goldstein <richgold@ix.netcom.com> wrote: >> Karin, >> >> I suggest you might want to read Sullivan, LM, et al. (2004), >> "Presentation of multivariate data for clinical use: the Framingham >> study risk score functions," _Statistics in Medicine_, 23: 1631-1660, >> which describes how the Framingham people came up with their risk scores >> >> Rich >> >> On 10/18/11 8:18 AM, K Jensen wrote: >>> Hi Nick >>> >>> Thanks for your reply. It's actually a bit more complicated than >>> that. We are trying to construct a "best" single score that would be >>> simple and used clinically. The elements that are summed to make the >>> score (0,1,2,3 etc) are derived from various clinical measurements. >>> They are dichotomised by choosing the cutpoint that maximises the sum >>> of sensitivity+specificity. Only those binary variables significant >>> in a univariate logistic regression are proposed for the model. >>> >>> I am wanting to choose the "best" model, that is useful for >>> clinicians. If we had 7 binary variables, say, I would look at all >>> possibilities of choosing different combinations of the sums of them. >>> E.g. 1, 2, 3, 4, 5, 6, 7,1+2,1+3,1+4,1+5,1+6,1+7, 2+3, 2+4,... up to >>> 1+2+3+4+5+6+7. I would like to use the optimal score based on this >>> method, but don't know how to measure optimality. >>> >>> Best wishes, >>> >>> Karin >>> >>> On 18 October 2011 12:36, Nick Cox <njcoxstata@gmail.com> wrote: >>>> I would recast this as a -logit- or -logistic- problem in which your >>>> outcome is dead or alive. Depending on how you think about your >>>> scores, they define predictors to be treated as they come or >>>> predictors to be treated as a set of indicator variables (or in some >>>> cases both). >>>> >>>> I don't think you are restricted to using one score or the other as predictor. >>>> >>>> Nick >>>> >>>> On Tue, Oct 18, 2011 at 12:11 PM, K Jensen <k.x.jensen@gmail.com> wrote: >>>>> Maybe this is more of a stats question than a Stata one, but there are >>>>> such a lot of good brains here... >>>>> >>>>> We are constructing point scores to indicate severity of risk Death >>>>> is the outcome. What is the best way of measuring the usefulness of >>>>> the score? The aim is to show a good gradient of risk. Say the >>>>> results for two different scores were: >>>>> >>>>> Score Dead Alive %dead Totals >>>>> 0 12 136 9.9% 145 >>>>> 1 18 126 15.4% 144 >>>>> 2 18 62 26.2% 81 >>>>> 3 10 9 57.1% 20 >>>>> 4 2 0 100 % 3 >>>>> ------------------------------------- >>>>> Total: 60 333 393 >>>>> >>>>> Score Dead Alive %dead Totals >>>>> 0 8 174 4.6% 182 >>>>> 1 21 143 12.8% 164 >>>>> 2 22 19 53.7% 41 >>>>> 3 5 1 83.3% 6 >>>>> ------------------------------------- >>>>> TOTAL: 60 333 393 >>>>> >>>>> Which is the better score? What is the best way to measure its >>>>> predictive power? I understand that ROC type analysis doesn't really >>>>> apply here. Some measure of R-squared? AIC? >>>>> >>>>> Thankyou >>>>> >>>>> Karin >>>>> >>>>> PS) I have made up the data, so the numbers don't quite add up. It is >>>>> meant to be two different, competing scores on the same people. >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/