Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: testing for significant differences between independent samplesusing svytab


From   "Michael Blasnik" <[email protected]>
To   <[email protected]>
Subject   st: Re: testing for significant differences between independent samplesusing svytab
Date   Tue, 29 Jan 2008 08:15:07 -0500

It is not completely clear what you did since you don't show us the Stata commands and output, but I have a few comments:

I guess this is a rhetorical question, but is there really anything magical about p=.05 that makes these two results substantively different in anyone's mind?

1) 95% confidence intervals can overlap even when the difference in means is significant at greater than 95%. The joint probability of both values being in the region of overlap is different than the separate nominal probabilities. Therefore, I would use the second test (well, actually I wouldn't just report a p value...).

2) The svytab command employs an endpoint transformation for confidence intervals to ensure that they are between 0 and 1. This transformation could be affecting your comparison of the two approaches. But, I'm pretty sure that explanation 1 is the issue here since t=2.12 for the difference in means is near the threshold and consistent with overlapping confidence intervals.

Michael Blasnik

----- Original Message ----- From: "Lacey Hartman" <[email protected]>
To: <[email protected]>
Sent: Monday, January 28, 2008 2:43 PM
Subject: st: testing for significant differences between independent samples using svytab



I am wondering why I get different results when using these 2 approaches to testing for a significant difference between two independent samples. I am also wondering which approach is more accurate and why.

#1 compare confidence intervals generated from svytab row obs ci se
I find that the confidence intervals overlap

#2
test for significant difference using the following t-test for independent samples:

value2-value1/ square root of (standard error of value 1 ^2 +standard error of value 2^2)

this formula gets me to a value of 2.12, significant at the 95% level

Again, why is there a difference and which approach is more correct?

Thanks!

Lacey Hartman
Senior Research Scientist
Health Economics Program
MN Department of Health
tel: 651-201-3556
fax: 651-201-5179
[email protected]

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index