Statalist The Stata Listserver

[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: -mlowess- available on SSC

From   "Nick Cox" <>
To   <>
Subject   RE: st: -mlowess- available on SSC
Date   Thu, 19 Oct 2006 11:23:52 +0100

I don't have any special expertise here, just 
a bunch of prejudices based on reading, thinking
and practice with various datasets. As Bill 
Gould reminded the list yesterday, I am a geographer,
not a statistician. 

I think the main problem with smoothing is that
it is too easy, and so leaves many hard questions 
unanswered. Of course, in many moods I go for "easy"
not "difficult" every time, especially if my 
colleagues or students report themselves happy 
that a successful smooth is capturing a pattern that
they can tell a story about in a presentation or

I like the idea that when fitting the data we 
should listen to what the data say, especially 
when existing theory is weak and/or we can't take 
formal models too seriously, except as simplifications
or conveniences. Who doesn't? So we should be 
open to indications of curvature, non-monotonicity, 
etc., etc. -- even to kinks and jumps, although
smooths won't always capture those well. 

But any kind of smooth -- bivariate or multivariate -- 
can leave awkward issues unanswered: 

1. Sure, we got this nice smooth with this method
and these data. But what about a different degree
of smoothing or a different smoothing method? 
As an old joke tells us, "The great thing 
about standards is that there are so many to choose
form." Same story with smoothing methods. 

2. How do I report my results other except as a graph? 
How are others in my field expected to relate my 
findings to theirs? For all the rigidity of many 
parametric models, the apparatus of estimates, 
standard errors, figures of merit, etc., etc., 
does give us a basis for comparisons, and comparisons
are much of the stuff of data analysis if you go
beyond what John Nelder called the cult of the
isolated study. 

J. A. Nelder (1986) 
Statistics, science and technology. 
Journal of the Royal Statistical Society A 
149: 109-121. 

3. Anyone can buy into some homespun philosophy
such as "Nature is wobbly and wiggly, not
straight". But being open to wobbles and 
wiggles in your data doesn't necessarily 
get you any closer to interesting generalisations
about nature (or society, or whatever). You 
can end up reporting lots of idiosyncratic behaviour, 
however well supported (pun intended) it is in 
a particular dataset. 

For those and other reasons, when I think about this
territory I am reminded of Nestroy's dark comment
that it is the nature of every advance to appear
greater than it really is. 
> thanks indeed Nick for your responses to all these
> smoothing questions.
> i have noticed that you warn of the dangers of taking
> these Stata smoothers too seriously, or of using them
> for anything other than exploratory work. also, in the
> concurrent -fractileplot- thread you mention that
> S-plus and R offer a much richer modelling
> environment.
> for those of us without the expertise required to make
> these judgements, would you care to explain further
> why you stress that smoothers such as -mlowess- or
> -mrunning- should not be taken too seriously? which
> deficiencies concern you?
> consider a simple application where all the x
> variables are known a priori (so that no exploratory
> work is required to determine the appropriate x's),
> and where all the y's & the x's are measured without
> error: if theory does not offer any priors re the
> functional form (other than local smoothness) are
> there any other estimators that would you take
> seriously? or do you think that smoothing science has
> not yet advanced to that point?

*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index