Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.

# Re: st: positive interaction - negative covariance

 From David Hoaglin To statalist@hsphsun2.harvard.edu Subject Re: st: positive interaction - negative covariance Date Sat, 23 Feb 2013 15:34:39 -0500

```The problems that arise from trying to compare confidence intervals
are more general.  They arise in situations where the estimates are
independent.  Thus, the covariance in the sampling distribution of b1
and b3 is not the real issue.

To assess the difference between two estimates, it is usually a
mistake to compare their confidence intervals.  The correct approach
is to form the appropriate confidence interval for the difference and
ask whether that confidence interval includes zero.  I often encounter
people who think that they can determine whether two estimates (e.g.,
the means of two independent samples) are different by checking
whether the two confidence intervals overlap.  They are simply wrong.
The article by Schenker and Gentleman (2001) explains.  (I said
"usually" above to exclude intervals that are constructed specifically
for use in assessing the significance of pairwise comparisons.)

David Hoaglin

Nathaniel Schenker and Jane F. Gentleman, On judging the significance
of differences by examining the overlap between confidence intervals.
The American Statistician 2001; 55(3):182-186.

On Sat, Feb 23, 2013 at 10:50 AM, Maarten Buis <maartenlbuis@gmail.com> wrote:
>>>>>> Thank you for clarifying the point. Indeed, if I compare the
>>>>>> confidence interval of b1 to the confidence interval of (b1+b3), they
>>>>>> are not statistically different. Is it the same? In a book I read, the
>>>>>> authors make this comparison.
>
> That is a common mistake. If you want to know whether the two curves
> are equal you must look at the interaction term (b3), not whether the
> confidence intervals of both curves (b1 and b1 + b3) overlap. The
> latter ignores the covariance in the sampling distribution of both
> parameters, and is thus wrong. See for example:
>
> Andrew Gelman and Hal Stern (2006) "The Difference Between
> `Signiﬁcant' and `Not Signiﬁcant' is not Itself Statistically
> Signiﬁcant" The American Statistician, 60(4):328--331.
>
> -- Maarten

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/
```