Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: positive interaction - negative covariance


From   David Hoaglin <dchoaglin@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: positive interaction - negative covariance
Date   Sat, 23 Feb 2013 15:34:39 -0500

The problems that arise from trying to compare confidence intervals
are more general.  They arise in situations where the estimates are
independent.  Thus, the covariance in the sampling distribution of b1
and b3 is not the real issue.

To assess the difference between two estimates, it is usually a
mistake to compare their confidence intervals.  The correct approach
is to form the appropriate confidence interval for the difference and
ask whether that confidence interval includes zero.  I often encounter
people who think that they can determine whether two estimates (e.g.,
the means of two independent samples) are different by checking
whether the two confidence intervals overlap.  They are simply wrong.
The article by Schenker and Gentleman (2001) explains.  (I said
"usually" above to exclude intervals that are constructed specifically
for use in assessing the significance of pairwise comparisons.)

David Hoaglin

Nathaniel Schenker and Jane F. Gentleman, On judging the significance
of differences by examining the overlap between confidence intervals.
The American Statistician 2001; 55(3):182-186.

On Sat, Feb 23, 2013 at 10:50 AM, Maarten Buis <maartenlbuis@gmail.com> wrote:
>>>>>> Thank you for clarifying the point. Indeed, if I compare the
>>>>>> confidence interval of b1 to the confidence interval of (b1+b3), they
>>>>>> are not statistically different. Is it the same? In a book I read, the
>>>>>> authors make this comparison.
>
> That is a common mistake. If you want to know whether the two curves
> are equal you must look at the interaction term (b3), not whether the
> confidence intervals of both curves (b1 and b1 + b3) overlap. The
> latter ignores the covariance in the sampling distribution of both
> parameters, and is thus wrong. See for example:
>
> Andrew Gelman and Hal Stern (2006) "The Difference Between
> `Significant' and `Not Significant' is not Itself Statistically
> Significant" The American Statistician, 60(4):328--331.
>
> -- Maarten

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index