st: Re: testing for significant differences between independent samplesusing svytab

 From "Michael Blasnik" <[email protected]> To <[email protected]> Subject st: Re: testing for significant differences between independent samplesusing svytab Date Tue, 29 Jan 2008 08:15:07 -0500

It is not completely clear what you did since you don't show us the Stata commands and output, but I have a few comments:

I guess this is a rhetorical question, but is there really anything magical about p=.05 that makes these two results substantively different in anyone's mind?

1) 95% confidence intervals can overlap even when the difference in means is significant at greater than 95%. The joint probability of both values being in the region of overlap is different than the separate nominal probabilities. Therefore, I would use the second test (well, actually I wouldn't just report a p value...).

2) The svytab command employs an endpoint transformation for confidence intervals to ensure that they are between 0 and 1. This transformation could be affecting your comparison of the two approaches. But, I'm pretty sure that explanation 1 is the issue here since t=2.12 for the difference in means is near the threshold and consistent with overlapping confidence intervals.

Michael Blasnik

----- Original Message ----- From: "Lacey Hartman" <[email protected]>
To: <[email protected]>
Sent: Monday, January 28, 2008 2:43 PM
Subject: st: testing for significant differences between independent samples using svytab

I am wondering why I get different results when using these 2 approaches to testing for a significant difference between two independent samples. I am also wondering which approach is more accurate and why.

#1 compare confidence intervals generated from svytab row obs ci se
I find that the confidence intervals overlap

#2
test for significant difference using the following t-test for independent samples:

value2-value1/ square root of (standard error of value 1 ^2 +standard error of value 2^2)

this formula gets me to a value of 2.12, significant at the 95% level

Again, why is there a difference and which approach is more correct?

Thanks!

Lacey Hartman
Senior Research Scientist
Health Economics Program
MN Department of Health
tel: 651-201-3556
fax: 651-201-5179
[email protected]

```*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```

• References: