Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: Statistical significance vs the 95% confidence intervals -- howshould i interpret these


From   "Michael Blasnik" <[email protected]>
To   <[email protected]>
Subject   st: Re: Statistical significance vs the 95% confidence intervals -- howshould i interpret these
Date   Fri, 08 Dec 2006 12:54:00 -0500

Neither of your approaches is correct. In your first approach, you are being overly conservative by just checking whether confidence intervals overlap (but you aren't very far off). In your second approach, -ttesti- is looking for standard deviations, not standard errors. Look at the output and you will see how ttest has created a new standard error that is much smaller (approxiately s.d./sqrt(n)).

Your question would be best answered by using whatever estimation method you used to come up with the estimates and standard errors and then use Stata's -test- or -lincom- commands to test hypotheses that properly account for potential covariance. If you can't do that for some reason, then you would need to make an assumption about the covariance. If you assume that the two estimates are independent, then you can calculate a t-statistic on the difference based on the difference in the estimates and the standard error of this difference:

di (.484721-.4556235)/sqrt(.015637^2+.0138994^2)
1.3907943

So t=1.39. You can get a p-value from:

di ttail(2398,(.484721-.4556235)/sqrt(.015637^2+.0138994^2))
.08220843

(or use whatever d.f. are appropriate if 2398 is not, but it won't make any real difference here)

So you could conclude that, if the two estimates are independent, then the difference between year1 and year3 is not quite statistically significant at the .05 level.


Michael Blasnik

----- Original Message ----- From: "Columbia & Belmont Apt" <[email protected]>
To: <[email protected]>
Sent: Friday, December 08, 2006 12:25 PM
Subject: st: Statistical significance vs the 95% confidence intervals -- how should i interpret these



Dear Statalist Colleagues,
This might be a simple question but I cannot reconcile the following results.

The question is "Are the estimates for year 1 and 3 statistically different?
year estimate st.error 95% conf interval
year1 .484721 .015637 .454068 .5153741
year2 .4128893 .0145616 .3843443 .4414343
year3 .4556235 .0138994 .4283765 .4828704

1. Using the 95% confidence interval that I obtained:

Year 1 and year 3 confidence intervals overlap thus year 1&3 estimates are not statistically different at the 95% level.

2. Using ttesti they are statistically different with 99% confidence:

. ttesti 2398 .484721 .015637 2399 .4556235 .0138994
Two-sample t test with equal variances
------------------------------------------------------------------------------
| Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
x | 2398 .484721 .0003193 .015637 .4840948 .4853472
y | 2399 .4556235 .0002838 .0138994 .455067 .45618
---------+--------------------------------------------------------------------
combined | 4797 .4701692 .0002996 .0207488 .4695819 .4707565
---------+--------------------------------------------------------------------
diff | .0290975 .0004272 .02826 .029935
------------------------------------------------------------------------------
Degrees of freedom: 4795
Ho: mean(x) - mean(y) = diff = 0
Ha: diff < 0 Ha: diff != 0 Ha: diff > 0
t = 68.1143 t = 68.1143 t = 68.1143
P < t = 1.0000 P > |t| = 0.0000 P > t = 0.0000


Which one is the correct? What am I not interpreting right?
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index