Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: RE: st: Elimination of outliers


From   Richard Williams <richardwilliams.ndu@gmail.com>
To   statalist@hsphsun2.harvard.edu, <statalist@hsphsun2.harvard.edu>
Subject   RE: RE: st: Elimination of outliers
Date   Mon, 06 Jun 2011 13:23:14 -0500

At 10:42 AM 6/6/2011, Jeff wrote:
"Outliers and influential observations should not routinely be deleted
or automatically down-weighted because they are not necessarily bad
observations.  On the contrary, if they are correct, they may be the
most informative points in the data.  For example, they may indicate
that the data did not come from a normal population or that the model is
not linear."  "Regression Analysis By Example", 3rd Edition, Chatterjee,
Hadi, Price.

Jeffrey B. Wolpin

In general, I think you should look at outliers before automatically eliminating them. It might be (a) they are coding errors, which you can either fix or use as a justification for dropping the case (b) they may lead you to redefine the population of interest, e.g. maybe you find your model works well for one group but a different model is required for another (c) you may be able to identify relevant variables that should be added to the model, and the cases won't be outliers anymore.

Also, there are different kinds of outliers. A univariate outlier might have an atypically high value on Y. But, it might also have an atypically high value on X, so it won't be an outlier on the regression line. It sounded to me like the original poster wanted to drop univariate outliers.

I don't remember the exact details, but there is some planet whose orbit did not quite fit the theories of the day. Somebody said that, if social scientists were doing things, they would have dropped that planet as an outlier and we never would have discovered the theory of relativity. :)


-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
HOME:   (574)289-5227
EMAIL:  Richard.A.Williams.5@ND.Edu
WWW:    http://www.nd.edu/~rwilliam

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index