st: Re: testing for significant differences between independent samplesusing svytab

["Michael Blasnik" <michael.blasnik@verizon.net>]

It is not completely clear what you did since you don't show us the Stata 
commands and output, but I have a few comments:

I guess this is a rhetorical question, but is there really anything magical 
about p=.05 that makes these two results substantively different in anyone's 
mind?

1)  95% confidence intervals can overlap even when the difference in means is 
significant at greater than 95%.  The joint probability of both values being in 
the region of overlap is different than the separate nominal probabilities. 
Therefore, I would use the second test (well, actually I wouldn't just report a 
p value...).

2) The svytab command employs an endpoint transformation for confidence 
intervals to ensure that they are between 0 and 1.  This transformation could be 
affecting your comparison of the two approaches.  But, I'm pretty sure that 
explanation 1 is the issue here since t=2.12 for the difference in means is near 
the threshold and consistent with overlapping confidence intervals.

Michael Blasnik

----- Original Message ----- 
From: "Lacey Hartman" <Lacey.Hartman@state.mn.us>
To: <statalist@hsphsun2.harvard.edu>
Sent: Monday, January 28, 2008 2:43 PM
Subject: st: testing for significant differences between independent samples 
using svytab



> I am wondering why I get different results when using these 2 approaches to 
> testing for a significant difference between two independent samples.  I am 
> also wondering which approach is more accurate and why.
>
> #1 compare confidence intervals generated from svytab row obs ci se
> I find that the confidence intervals overlap
>
> #2
> test for significant difference using the following  t-test for independent 
> samples:
>
> value2-value1/ square root of (standard error of value 1 ^2 +standard error of 
> value 2^2)
>
> this formula gets me to a value of 2.12, significant at the 95% level
>
> Again, why is there a difference and which approach is more correct?
>
> Thanks!
>
> Lacey Hartman
> Senior Research Scientist
> Health Economics Program
> MN Department of Health
> tel: 651-201-3556
> fax: 651-201-5179
> lacey.hartman@health.state.mn.us
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/